redis_queued_locks 1.1.0 → 1.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +18 -3
- data/README.md +262 -60
- data/lib/redis_queued_locks/acquier/acquire_lock/delay_execution.rb +2 -2
- data/lib/redis_queued_locks/acquier/acquire_lock/try_to_lock.rb +239 -12
- data/lib/redis_queued_locks/acquier/acquire_lock/with_acq_timeout.rb +2 -2
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_expire.rb +115 -0
- data/lib/redis_queued_locks/acquier/acquire_lock.rb +200 -89
- data/lib/redis_queued_locks/acquier/clear_dead_requests.rb +3 -3
- data/lib/redis_queued_locks/acquier/extend_lock_ttl.rb +3 -3
- data/lib/redis_queued_locks/acquier/is_locked.rb +2 -2
- data/lib/redis_queued_locks/acquier/is_queued.rb +2 -2
- data/lib/redis_queued_locks/acquier/keys.rb +2 -2
- data/lib/redis_queued_locks/acquier/lock_info.rb +19 -3
- data/lib/redis_queued_locks/acquier/locks.rb +13 -4
- data/lib/redis_queued_locks/acquier/queue_info.rb +2 -2
- data/lib/redis_queued_locks/acquier/queues.rb +4 -4
- data/lib/redis_queued_locks/acquier/release_all_locks.rb +4 -4
- data/lib/redis_queued_locks/acquier/release_lock.rb +4 -4
- data/lib/redis_queued_locks/acquier.rb +1 -1
- data/lib/redis_queued_locks/client.rb +50 -22
- data/lib/redis_queued_locks/debugger/interface.rb +4 -4
- data/lib/redis_queued_locks/debugger.rb +8 -8
- data/lib/redis_queued_locks/errors.rb +10 -6
- data/lib/redis_queued_locks/instrument/active_support.rb +2 -2
- data/lib/redis_queued_locks/instrument/void_notifier.rb +2 -2
- data/lib/redis_queued_locks/instrument.rb +2 -2
- data/lib/redis_queued_locks/logging/void_logger.rb +10 -10
- data/lib/redis_queued_locks/logging.rb +10 -3
- data/lib/redis_queued_locks/resource.rb +22 -16
- data/lib/redis_queued_locks/utilities.rb +2 -2
- data/lib/redis_queued_locks/version.rb +2 -2
- data/lib/redis_queued_locks.rb +2 -2
- metadata +4 -4
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_with_expire.rb +0 -72
data/README.md
CHANGED
@@ -1,11 +1,11 @@
|
|
1
1
|
# RedisQueuedLocks · ![Gem Version](https://img.shields.io/gem/v/redis_queued_locks) ![build](https://github.com/0exp/redis_queued_locks/actions/workflows/build.yml/badge.svg??branch=master)
|
2
2
|
|
3
|
-
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "lock acquisition queue" capabilities based on the Redis Database.
|
4
|
-
|
5
|
-
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, fast failing, etc), logging and instrumentation.
|
3
|
+
<a href="https://redis.io/docs/manual/patterns/distributed-locks/">Distributed locks</a> with "prioritized lock acquisition queue" capabilities based on the Redis Database.
|
6
4
|
|
7
5
|
Each lock request is put into the request queue (each lock is hosted by it's own queue separately from other queues) and processed in order of their priority (FIFO). Each lock request lives some period of time (RTTL) (with requeue capabilities) which guarantees the request queue will never be stacked.
|
8
6
|
|
7
|
+
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttls, queue ttls, lock attempts limit, fast failing, etc), logging and instrumentation.
|
8
|
+
|
9
9
|
---
|
10
10
|
|
11
11
|
## Table of Contents
|
@@ -32,6 +32,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
32
32
|
- [locks_info](#locks_info---get-list-of-locks-with-their-info)
|
33
33
|
- [queues_info](#queues_info---get-list-of-queues-with-their-info)
|
34
34
|
- [clear_dead_requests](#clear_dead_requests)
|
35
|
+
- [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks)
|
35
36
|
- [Logging](#logging)
|
36
37
|
- [Instrumentation](#instrumentation)
|
37
38
|
- [Instrumentation Events](#instrumentation-events)
|
@@ -57,7 +58,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
|
|
57
58
|
|
58
59
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
59
60
|
|
60
|
-
- Battle-tested on huge ruby projects in production: `~
|
61
|
+
- Battle-tested on huge ruby projects in production: `~3000` locks-per-second are obtained and released on an ongoing basis;
|
61
62
|
- Works well with `hiredis` driver enabled (it is enabled by default on our projects where `redis_queued_locks` are used);
|
62
63
|
|
63
64
|
---
|
@@ -149,9 +150,22 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
149
150
|
# - should be all blocks of code are timed by default;
|
150
151
|
config.is_timed_by_default = false
|
151
152
|
|
153
|
+
# (symbol) (default: :wait_for_lock)
|
154
|
+
# - The conflict strategy mode for cases when the process that obtained the lock want to acquire this lock again;
|
155
|
+
# - Realizes "reentrant locks" abstraction (same process conflict / same process deadlock);
|
156
|
+
# - By default uses `:wait_for_lock` strategy (classic way);
|
157
|
+
# - Strategies:
|
158
|
+
# - `:work_through` - continue working under the lock <without> lock's TTL extension;
|
159
|
+
# - `:extendable_work_through` - continue working under the lock <with> lock's TTL extension;
|
160
|
+
# - `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
161
|
+
# - `:dead_locking` - fail with deadlock exception;
|
162
|
+
# - Can be customized in methods via `:conflict_strategy` attribute (see method signatures of #lock and #lock! methods);
|
163
|
+
# - See "Dead locks and Reentrant Locks" documentation section in REDME.md for details;
|
164
|
+
config.default_conflict_strategy = :wait_for_lock
|
165
|
+
|
152
166
|
# (default: 100)
|
153
|
-
# - how many items will be released at a time in
|
154
|
-
# - affects the
|
167
|
+
# - how many items will be released at a time in #clear_locks and in #clear_dead_requests (uses SCAN);
|
168
|
+
# - affects the performance of your Redis and Ruby Application (configure thoughtfully);
|
155
169
|
config.lock_release_batch_size = 100
|
156
170
|
|
157
171
|
# (default: 500)
|
@@ -185,13 +199,17 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
185
199
|
# (default: RedisQueuedLocks::Logging::VoidLogger)
|
186
200
|
# - the logger object;
|
187
201
|
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
202
|
+
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
188
203
|
# - at this moment the only debug logs are realised in following cases:
|
189
204
|
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id");
|
190
205
|
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id");
|
191
206
|
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id");
|
192
|
-
# - "[redis_queued_locks.lock_obtained]" (logs "
|
193
|
-
# - "[redis_queued_locks.
|
194
|
-
# - "[redis_queued_locks.
|
207
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
208
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
209
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
210
|
+
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id");
|
211
|
+
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id");
|
212
|
+
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id");
|
195
213
|
# - by default uses VoidLogger that does nothing;
|
196
214
|
config.logger = RedisQueuedLocks::Logging::VoidLogger
|
197
215
|
|
@@ -201,6 +219,10 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
201
219
|
# - it adds following logs in addition to the existing:
|
202
220
|
# - "[redis_queued_locks.try_lock.start]" (logs "lock_key", "queue_ttl", "acq_id");
|
203
221
|
# - "[redis_queued_locks.try_lock.rconn_fetched]" (logs "lock_key", "queue_ttl", "acq_id");
|
222
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_detected]" (logs "lock_key", "queue_ttl", "acq_id");
|
223
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_analyzed]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status");
|
224
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_ext_ttl", "last_ext_ts");
|
225
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "spc_status", last_spc_ts);
|
204
226
|
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id)";
|
205
227
|
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id");
|
206
228
|
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
@@ -257,6 +279,7 @@ def lock(
|
|
257
279
|
retry_jitter: config[:retry_jitter],
|
258
280
|
raise_errors: false,
|
259
281
|
fail_fast: false,
|
282
|
+
conflict_strategy: config[:default_conflict_strategy],
|
260
283
|
identity: uniq_identity, # (attr_accessor) calculated during client instantiation via config[:uniq_identifier] proc;
|
261
284
|
meta: nil,
|
262
285
|
instrument: nil,
|
@@ -295,7 +318,7 @@ def lock(
|
|
295
318
|
- See [Instrumentation](#instrumentation) section of docs;
|
296
319
|
- pre-configured in `config[:isntrumenter]` with void notifier (`RedisQueuedLocks::Instrumenter::VoidNotifier`);
|
297
320
|
- `raise_errors` - (optional) `[Boolean]`
|
298
|
-
- Raise errors on library-related limits such as timeout or retry count limit;
|
321
|
+
- Raise errors on library-related limits (such as timeout or retry count limit) and on lock conflicts (such as same-process dead locks);
|
299
322
|
- `false` by default;
|
300
323
|
- `fail_fast` - (optional) `[Boolean]`
|
301
324
|
- Should the required lock to be checked before the try and exit immidietly if lock is
|
@@ -303,6 +326,17 @@ def lock(
|
|
303
326
|
- Should the logic exit immidietly after the first try if the lock was obtained
|
304
327
|
by another process while the lock request queue was initially empty;
|
305
328
|
- `false` by default;
|
329
|
+
- `conflict_strategy` - (optional) - `[Symbol]``
|
330
|
+
- The conflict strategy mode for cases when the process that obtained the lock
|
331
|
+
want to acquire this lock again;
|
332
|
+
- By default uses `:wait_for_lock` strategy;
|
333
|
+
- pre-confured in `config[:default_conflict_strategy]`;
|
334
|
+
- Strategies:
|
335
|
+
- `:work_through` - continue working under the lock **without** lock's TTL extension;
|
336
|
+
- `:extendable_work_through` - continue working under the lock **with** lock's TTL extension;
|
337
|
+
- `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
338
|
+
- `:dead_locking` - fail with deadlock exception;
|
339
|
+
- See [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) readme section for details;
|
306
340
|
- `identity` - (optional) `[String]`
|
307
341
|
- An unique string that is unique per `RedisQueuedLock::Client` instance. Resolves the
|
308
342
|
collisions between the same process_id/thread_id/fiber_id/ractor_id identifiers on different
|
@@ -324,10 +358,12 @@ def lock(
|
|
324
358
|
- should be logged the each try of lock acquiring (a lot of logs can be generated depending on your retry configurations);
|
325
359
|
- pre-configured in `config[:log_lock_try]`;
|
326
360
|
- `false` by default;
|
327
|
-
- `block` - `[Block]`
|
361
|
+
- `block` - (optional) `[Block]`
|
328
362
|
- A block of code that should be executed after the successfully acquired lock.
|
329
363
|
- If block is **passed** the obtained lock will be released after the block execution or it's ttl (what will happen first);
|
330
364
|
- If block is **not passed** the obtained lock will be released after it's ttl;
|
365
|
+
- If you want the block to have a TTL too and this TTL to be the same as TTL of the lock
|
366
|
+
use `timed: true` option (`rql.lock("my_lock", timed: true, ttl: 5_000) { ... }`)
|
331
367
|
|
332
368
|
Return value:
|
333
369
|
|
@@ -346,14 +382,21 @@ Return value:
|
|
346
382
|
lock_key: "rql:lock:my_lock",
|
347
383
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
348
384
|
ts: 1711909612.653696,
|
349
|
-
ttl: 10000
|
385
|
+
ttl: 10000,
|
386
|
+
process: :lock_obtaining
|
350
387
|
}
|
351
388
|
}
|
352
389
|
```
|
353
390
|
- Lock information result:
|
354
391
|
- Signature: `[yield, Hash<Symbol,Boolean|Hash<Symbol,Numeric|String>>]`
|
355
392
|
- Format: `{ ok: true/false, result: <Symbol|Hash<Symbol,Hash>> }`;
|
356
|
-
-
|
393
|
+
- Includes the `:process` key that describes a logical type of the lock obtaining process. Possible values:
|
394
|
+
- `:lock_obtaining` - classic lock obtaining proces. Default behavior (`conflict_strategy: :wait_for_lock`);
|
395
|
+
- `:extendable_conflict_work_through` - reentrant lock acquiring process with lock's TTL extension. Suitable for `conflict_strategy: :extendable_work_through`;
|
396
|
+
- `:conflict_work_through` - reentrant lock acquiring process without lock's TTL extension. Suitable for `conflict_strategy: :work_through`;
|
397
|
+
- `:dead_locking` - current process tries to acquire a lock that is already acquired by them. Suitalbe for `conflict_startegy: :dead_locking`;
|
398
|
+
- For more details see [Dead locks and Reentrant locks](#dead-locks-and-reentrant-locks) readme section;
|
399
|
+
- For successful lock obtaining:
|
357
400
|
```ruby
|
358
401
|
{
|
359
402
|
ok: true,
|
@@ -361,10 +404,12 @@ Return value:
|
|
361
404
|
lock_key: String, # acquierd lock key ("rql:lock:your_lock_name")
|
362
405
|
acq_id: String, # acquier identifier ("process_id/thread_id/fiber_id/ractor_id/identity")
|
363
406
|
ts: Float, # time (epoch) when lock was obtained (float, Time#to_f)
|
364
|
-
ttl: Integer # lock's time to live in milliseconds (integer)
|
407
|
+
ttl: Integer, # lock's time to live in milliseconds (integer)
|
408
|
+
process: Symbol # which logical process has acquired the lock (:lock_obtaining, :extendable_conflict_work_through, :conflict_work_through, :conflict_dead_lock)
|
365
409
|
}
|
366
410
|
}
|
367
411
|
```
|
412
|
+
|
368
413
|
```ruby
|
369
414
|
# example:
|
370
415
|
{
|
@@ -373,14 +418,16 @@ Return value:
|
|
373
418
|
lock_key: "rql:lock:my_lock",
|
374
419
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
375
420
|
ts: 1711909612.653696,
|
376
|
-
ttl: 10000
|
421
|
+
ttl: 10000,
|
422
|
+
process: :lock_obtaining # for custom conflict strategies may be: :conflict_dead_lock, :conflict_work_through, :extendable_conflict_work_through
|
377
423
|
}
|
378
424
|
}
|
379
425
|
```
|
380
|
-
-
|
426
|
+
- For failed lock obtaining:
|
381
427
|
```ruby
|
382
428
|
{ ok: false, result: :timeout_reached }
|
383
429
|
{ ok: false, result: :retry_count_reached }
|
430
|
+
{ ok: false, result: :conflict_dead_lock } # see <conflict_strategy> option for details (:dead_locking strategy)
|
384
431
|
{ ok: false, result: :fail_fast_no_try } # see <fail_fast> option
|
385
432
|
{ ok: false, result: :fail_fast_after_try } # see <fail_fast> option
|
386
433
|
{ ok: false, result: :unknown }
|
@@ -453,6 +500,43 @@ rql.lock_info("my_lock")
|
|
453
500
|
}
|
454
501
|
```
|
455
502
|
|
503
|
+
- (`:queue_ttl`) setting a short limit of time to the lock request queue position (if a process fails to acquire
|
504
|
+
the lock within this period of time (and before timeout/retry_count limits occurs of course) -
|
505
|
+
it's lock request will be moved to the end of queue):
|
506
|
+
|
507
|
+
```ruby
|
508
|
+
rql.lock("my_lock", queue_ttl: 5, timeout: 10_000, retry_count: nil)
|
509
|
+
# "queue_ttl: 5": 5 seconds time slot before the lock request moves to the end of queue;
|
510
|
+
# "timeout" and "retry_count" is used as "endless lock try attempts" example to show the lock queue behavior;
|
511
|
+
|
512
|
+
# lock queue: =>
|
513
|
+
[
|
514
|
+
"rql:acq:123/456/567/676/374dd74324",
|
515
|
+
"rql:acq:123/456/567/677/374dd74322", # <- long living lock
|
516
|
+
"rql:acq:123/456/567/679/374dd74321",
|
517
|
+
"rql:acq:123/456/567/683/374dd74322", # <== we are here
|
518
|
+
"rql:acq:123/456/567/685/374dd74329", # some other waiting process
|
519
|
+
]
|
520
|
+
|
521
|
+
# ... some period of time (2 seconds later)
|
522
|
+
# lock queue: =>
|
523
|
+
[
|
524
|
+
"rql:acq:123/456/567/677/374dd74322", # <- long living lock
|
525
|
+
"rql:acq:123/456/567/679/374dd74321",
|
526
|
+
"rql:acq:123/456/567/683/374dd74322", # <== we are here
|
527
|
+
"rql:acq:123/456/567/685/374dd74329", # some other waiting process
|
528
|
+
]
|
529
|
+
|
530
|
+
# ... some period of time (3 seconds later)
|
531
|
+
# ... queue_ttl time limit is reached
|
532
|
+
# lock queue: =>
|
533
|
+
[
|
534
|
+
"rql:acq:123/456/567/685/374dd74329", # some other waiting process
|
535
|
+
"rql:acq:123/456/567/683/374dd74322", # <== we are here (moved to the end of the queue)
|
536
|
+
]
|
537
|
+
|
538
|
+
```
|
539
|
+
|
456
540
|
---
|
457
541
|
|
458
542
|
#### #lock! - exceptional lock obtaining
|
@@ -469,16 +553,18 @@ def lock!(
|
|
469
553
|
lock_name,
|
470
554
|
ttl: config[:default_lock_ttl],
|
471
555
|
queue_ttl: config[:default_queue_ttl],
|
472
|
-
timeout: config[:
|
556
|
+
timeout: config[:try_to_lock_timeout],
|
557
|
+
timed: config[:is_timed_by_default],
|
473
558
|
retry_count: config[:retry_count],
|
474
559
|
retry_delay: config[:retry_delay],
|
475
560
|
retry_jitter: config[:retry_jitter],
|
476
|
-
identity: uniq_identity,
|
477
561
|
fail_fast: false,
|
562
|
+
identity: uniq_identity,
|
478
563
|
meta: nil,
|
479
|
-
instrument: nil,
|
480
564
|
logger: config[:logger],
|
481
565
|
log_lock_try: config[:log_lock_try],
|
566
|
+
instrument: nil,
|
567
|
+
conflict_strategy: config[:default_conflict_strategy],
|
482
568
|
&block
|
483
569
|
)
|
484
570
|
```
|
@@ -496,27 +582,36 @@ See `#lock` method [documentation](#lock---obtain-a-lock).
|
|
496
582
|
- lock data (`Hash<String,String|Integer>`):
|
497
583
|
- `"lock_key"` - `string` - lock key in redis;
|
498
584
|
- `"acq_id"` - `string` - acquier identifier (process_id/thread_id/fiber_id/ractor_id/identity);
|
499
|
-
- `"ts"` - `
|
585
|
+
- `"ts"` - `numeric`/`epoch` - the time when lock was obtained;
|
500
586
|
- `"init_ttl"` - `integer` - (milliseconds) initial lock key ttl;
|
501
587
|
- `"rem_ttl"` - `integer` - (milliseconds) remaining lock key ttl;
|
502
|
-
-
|
588
|
+
- `<custom metadata>`- `string`/`integer` - custom metadata passed to the `lock`/`lock!` methods via `meta:` keyword argument (see [lock]((#lock---obtain-a-lock)) method documentation);
|
589
|
+
- additional keys for **reentrant locks** and **extendable reentrant locks**:
|
590
|
+
- for any type of reentrant locks:
|
591
|
+
- `"spc_cnt"` - `integer` - how many times the lock was obtained as reentrant lock;
|
592
|
+
- for non-extendable reentrant locks:
|
593
|
+
- `"l_spc_ts"` - `numeric`/`epoch` - timestamp of the last **non-extendable** reentrant lock obtaining;
|
594
|
+
- for extendalbe reentrant locks:
|
595
|
+
- `"spc_ext_ttl"` - `integer` - (milliseconds) sum of TTL of the each **extendable** reentrant lock (the total TTL extension time);
|
596
|
+
- `"l_spc_ext_ini_ttl"` - `integer` - (milliseconds) TTL of the last reentrant lock;
|
597
|
+
- `"l_spc_ext_ts"` - `numeric`/`epoch` - timestamp of the last extendable reentrant lock obtaining;
|
503
598
|
|
504
599
|
```ruby
|
505
|
-
# without custom metadata
|
600
|
+
# <without custom metadata>
|
506
601
|
rql.lock_info("your_lock_name")
|
507
602
|
|
508
603
|
# =>
|
509
604
|
{
|
510
605
|
"lock_key" => "rql:lock:your_lock_name",
|
511
606
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
512
|
-
"ts" => 123456789,
|
513
|
-
"ini_ttl" =>
|
514
|
-
"rem_ttl" =>
|
607
|
+
"ts" => 123456789.12345,
|
608
|
+
"ini_ttl" => 5_000,
|
609
|
+
"rem_ttl" => 4_999
|
515
610
|
}
|
516
611
|
```
|
517
612
|
|
518
613
|
```ruby
|
519
|
-
# with custom metadata
|
614
|
+
# <with custom metadata>
|
520
615
|
rql.lock("your_lock_name", meta: { "kek" => "pek", "bum" => 123 })
|
521
616
|
rql.lock_info("your_lock_name")
|
522
617
|
|
@@ -524,14 +619,41 @@ rql.lock_info("your_lock_name")
|
|
524
619
|
{
|
525
620
|
"lock_key" => "rql:lock:your_lock_name",
|
526
621
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
527
|
-
"ts" => 123456789,
|
528
|
-
"ini_ttl" =>
|
529
|
-
"rem_ttl" =>
|
622
|
+
"ts" => 123456789.12345,
|
623
|
+
"ini_ttl" => 5_000,
|
624
|
+
"rem_ttl" => 4_999,
|
530
625
|
"kek" => "pek",
|
531
626
|
"bum" => "123" # NOTE: returned as a raw string directly from Redis
|
532
627
|
}
|
533
628
|
```
|
534
629
|
|
630
|
+
```ruby
|
631
|
+
# <for reentrant locks>
|
632
|
+
# (see `conflict_strategy:` kwarg attribute of #lock/#lock! methods and `config.default_conflict_strategy` config)
|
633
|
+
|
634
|
+
rql.lock("your_lock_name", ttl: 5_000)
|
635
|
+
rql.lock("your_lock_name", ttl: 3_000)
|
636
|
+
rql.lock("your_lock_name", ttl: 2_000)
|
637
|
+
rql.lock_info("your_lock_name")
|
638
|
+
|
639
|
+
# =>
|
640
|
+
{
|
641
|
+
"lock_key" => "rql:lock:your_lock_name",
|
642
|
+
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
643
|
+
"ts" => 123456789.12345,
|
644
|
+
"ini_ttl" => 5_000,
|
645
|
+
"rem_ttl" => 9_444,
|
646
|
+
# ==> keys for any type of reentrant lock:
|
647
|
+
"spc_count" => 2, # how many times the lock was obtained as reentrant lock
|
648
|
+
# ==> keys for extendable reentarnt locks with `:extendable_work_through` strategy:
|
649
|
+
"spc_ext_ttl" => 5_000, # sum of TTL of the each <extendable> reentrant lock (3_000 + 2_000)
|
650
|
+
"l_spc_ext_ini_ttl" => 2_000, # TTL of the last <extendable> reentrant lock
|
651
|
+
"l_spc_ext_ts" => 123456792.12345 # timestamp of the last <extendable> reentrant lock obtaining
|
652
|
+
# ==> keys for non-extendable locks with `:work_through` strategy:
|
653
|
+
"l_spc_ts" => 123456.789 # timestamp of the last <non-extendable> reentrant lock obtaining
|
654
|
+
}
|
655
|
+
```
|
656
|
+
|
535
657
|
---
|
536
658
|
|
537
659
|
#### #queue_info
|
@@ -563,9 +685,9 @@ rql.queue_info("your_lock_name")
|
|
563
685
|
{
|
564
686
|
"lock_queue" => "rql:lock_queue:your_lock_name",
|
565
687
|
"queue" => [
|
566
|
-
{ "acq_id" => "rql:acq:123/456/567/678/fa76df9cc2", "score" =>
|
567
|
-
{ "acq_id" => "rql:acq:123/567/456/679/c7bfcaf4f9", "score" =>
|
568
|
-
{ "acq_id" => "rql:acq:555/329/523/127/7329553b11", "score" =>
|
688
|
+
{ "acq_id" => "rql:acq:123/456/567/678/fa76df9cc2", "score" => 1711606640.540842},
|
689
|
+
{ "acq_id" => "rql:acq:123/567/456/679/c7bfcaf4f9", "score" => 1711606640.540906},
|
690
|
+
{ "acq_id" => "rql:acq:555/329/523/127/7329553b11", "score" => 1711606640.540963},
|
569
691
|
# ...etc
|
570
692
|
]
|
571
693
|
}
|
@@ -587,6 +709,8 @@ rql.locked?("your_lock_name") # => true/false
|
|
587
709
|
|
588
710
|
#### #queued?
|
589
711
|
|
712
|
+
<sup>\[[back to top](#usage)\]</sup>
|
713
|
+
|
590
714
|
- is the lock queued for obtain / has requests for obtain?
|
591
715
|
|
592
716
|
```ruby
|
@@ -635,7 +759,7 @@ rql.unlock("your_lock_name")
|
|
635
759
|
result: {
|
636
760
|
rel_time: 0.02, # time spent to lock release (in seconds)
|
637
761
|
rel_key: "rql:lock:your_lock_name", # released lock key
|
638
|
-
rel_queue: "rql:lock_queue:your_lock_name" # released lock key queue
|
762
|
+
rel_queue: "rql:lock_queue:your_lock_name", # released lock key queue
|
639
763
|
queue_res: :released, # or :nothing_to_release
|
640
764
|
lock_res: :released # or :nothing_to_release
|
641
765
|
}
|
@@ -662,12 +786,11 @@ rql.unlock("your_lock_name")
|
|
662
786
|
- pre-configured value in `config[:isntrumenter]`;
|
663
787
|
- `:instrument` - (optional) `[NilClass,Any]`
|
664
788
|
- custom instrumentation data wich will be passed to the instrumenter's payload with `:instrument` key;
|
665
|
-
|
666
789
|
- returns:
|
667
|
-
|
668
|
-
|
669
|
-
|
670
|
-
|
790
|
+
- `[Hash<Symbol,Numeric>]` - Format: `{ ok: true, result: Hash<Symbol,Numeric> }`;
|
791
|
+
- result data:
|
792
|
+
- `:rel_time` - `Numeric` - time spent to release all locks and related queus;
|
793
|
+
- `:rel_key_cnt` - `Integer` - the number of released Redis keys (queues+locks);
|
671
794
|
|
672
795
|
```ruby
|
673
796
|
rql.clear_locks
|
@@ -699,13 +822,14 @@ rql.clear_locks
|
|
699
822
|
- custom logger object;
|
700
823
|
- pre-configured in `config[:logger]`;
|
701
824
|
- returns `{ ok: true, result: :ttl_extended }` when ttl is extended;
|
702
|
-
- returns `{ ok: false, result: :async_expire_or_no_lock }` when lock not found or lock is expired during
|
825
|
+
- returns `{ ok: false, result: :async_expire_or_no_lock }` when a lock not found or a lock is already expired during
|
703
826
|
some steps of invocation (see **Important** section below);
|
704
827
|
- **Important**:
|
705
828
|
- the method is non-atomic cuz redis does not provide an atomic function for TTL/PTTL extension;
|
706
829
|
- the method consists of two commands:
|
707
830
|
- (1) read current pttl;
|
708
831
|
- (2) set new ttl that is calculated as "current pttl + additional milliseconds";
|
832
|
+
- the method uses Redis'es **CAS** (check-and-set) behavior;
|
709
833
|
- what can happen during these steps:
|
710
834
|
- lock is expired between commands or before the first command;
|
711
835
|
- lock is expired before the second command;
|
@@ -729,6 +853,7 @@ rql.extend_lock_ttl("my_lock", 5_000) # NOTE: add 5_000 milliseconds
|
|
729
853
|
|
730
854
|
<sup>\[[back to top](#usage)\]</sup>
|
731
855
|
|
856
|
+
- get list of obtained locks;
|
732
857
|
- uses redis `SCAN` under the hood;
|
733
858
|
- accepts:
|
734
859
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -761,6 +886,7 @@ rql.locks # or rql.locks(scan_size: 123)
|
|
761
886
|
|
762
887
|
<sup>\[[back to top](#usage)\]</sup>
|
763
888
|
|
889
|
+
- get list of lock request queues;
|
764
890
|
- uses redis `SCAN` under the hood;
|
765
891
|
- accepts
|
766
892
|
- `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -793,6 +919,7 @@ rql.queues # or rql.queues(scan_size: 123)
|
|
793
919
|
|
794
920
|
<sup>\[[back to top](#usage)\]</sup>
|
795
921
|
|
922
|
+
- get list of taken locks and queues;
|
796
923
|
- uses redis `SCAN` under the hood;
|
797
924
|
- accepts:
|
798
925
|
`:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
|
@@ -826,6 +953,7 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
826
953
|
|
827
954
|
<sup>\[[back to top](#usage)\]</sup>
|
828
955
|
|
956
|
+
- get list of locks with their info;
|
829
957
|
- uses redis `SCAN` under the hod;
|
830
958
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
831
959
|
- returns `Set<Hash<Symbol,Any>>` (see [#lock_info](#lock_info) and examples below for details).
|
@@ -834,7 +962,9 @@ rql.keys # or rql.keys(scan_size: 123)
|
|
834
962
|
- `:status` - `Symbol`- `:released` or `:alive`
|
835
963
|
- the lock may become relased durign the lock info extraction process;
|
836
964
|
- `:info` for `:released` keys is empty (`{}`);
|
837
|
-
- `:info` - `Hash<String,Any>`
|
965
|
+
- `:info` - `Hash<String,Any>`
|
966
|
+
- lock data stored in the lock key in Redis;
|
967
|
+
- See [#lock_info](#lock_info) for details;
|
838
968
|
|
839
969
|
```ruby
|
840
970
|
rql.locks_info # or rql.locks_info(scan_size: 123)
|
@@ -860,6 +990,7 @@ rql.locks_info # or rql.locks_info(scan_size: 123)
|
|
860
990
|
|
861
991
|
<sup>\[[back to top](#usage)\]</sup>
|
862
992
|
|
993
|
+
- get list of queues with their info;
|
863
994
|
- uses redis `SCAN` under the hod;
|
864
995
|
- accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
|
865
996
|
- returns `Set<Hash<Symbol,Any>>` (see [#queue_info](#queue_info) and examples below for details).
|
@@ -908,7 +1039,7 @@ Accepts:
|
|
908
1039
|
- has a preconfigured value in `config[:dead_request_ttl]` (1 day by default);
|
909
1040
|
- `:sacn_size` - (optional) `[Integer]`
|
910
1041
|
- the batch of scanned keys for Redis'es SCAN command;
|
911
|
-
- has a preconfigured valie in `config[:
|
1042
|
+
- has a preconfigured valie in `config[:lock_release_batch_size]`;
|
912
1043
|
- `:logger` - (optional) `[::Logger,#debug]`
|
913
1044
|
- custom logger object;
|
914
1045
|
- pre-configured in `config[:logger]`;
|
@@ -938,6 +1069,20 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
938
1069
|
|
939
1070
|
---
|
940
1071
|
|
1072
|
+
## Dead locks and Reentrant locks
|
1073
|
+
|
1074
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1075
|
+
|
1076
|
+
- **documentation is in progress**
|
1077
|
+
- (little details for a context of current implementation and feautres):
|
1078
|
+
- at this moment we support only **reentrant locks**: they works via customizable conflict strategy behavior;
|
1079
|
+
- in non-reentrant conflict cases your lock obtaining process will work in a classic way (some dead lock conflict => work in "wait for lock" style);
|
1080
|
+
- for current implementation details check:
|
1081
|
+
- (Configuration)[#configuration] documentation: see `config.default_conflict_strategy` config docs;
|
1082
|
+
- (#lock)[#lock] method documentation: see `conflict_strategy` attribute docs;
|
1083
|
+
|
1084
|
+
---
|
1085
|
+
|
941
1086
|
## Logging
|
942
1087
|
|
943
1088
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
@@ -948,9 +1093,12 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
948
1093
|
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id");
|
949
1094
|
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id");
|
950
1095
|
"[redis_queued_locks.dead_score_reached__reset_acquier_position]" # (logs "lock_key", "queue_ttl", "acq_id");
|
951
|
-
"[redis_queued_locks.lock_obtained]" # (logs "
|
952
|
-
"[redis_queued_locks.
|
1096
|
+
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1097
|
+
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1098
|
+
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1099
|
+
"[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id");
|
953
1100
|
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1101
|
+
"[redis_queued_locks.decrease_lock]" # (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id");
|
954
1102
|
```
|
955
1103
|
|
956
1104
|
- additional logs (raised from `#lock`/`#lock!` with `confg[:log_lock_try] == true`):
|
@@ -958,6 +1106,11 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
958
1106
|
```ruby
|
959
1107
|
"[redis_queued_locks.try_lock.start]" # (logs "lock_key", "queue_ttl", "acq_id");
|
960
1108
|
"[redis_queued_locks.try_lock.rconn_fetched]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1109
|
+
"[redis_queued_locks.try_lock.same_process_conflict_detected]" # (logs "lock_key", "queue_ttl", "acq_id");
|
1110
|
+
"[redis_queued_locks.try_lock.same_process_conflict_analyzed]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status");
|
1111
|
+
"[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_ext_ttl", "last_ext_ts");
|
1112
|
+
"[redis_queued_locks.try_lock.reentrant_lock__work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", last_spc_ts);
|
1113
|
+
"[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "spc_status", "last_spc_ts");
|
961
1114
|
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id)";
|
962
1115
|
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id");
|
963
1116
|
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "first_acq_id_in_queue");
|
@@ -994,12 +1147,15 @@ By default `RedisQueuedLocks::Client` is configured with the void notifier (whic
|
|
994
1147
|
|
995
1148
|
### Instrumentation Events
|
996
1149
|
|
997
|
-
<sup>\[[back to top](#instrumentation
|
1150
|
+
<sup>\[[back to top](#instrumentation)\]</sup>
|
998
1151
|
|
999
1152
|
List of instrumentation events
|
1000
1153
|
|
1001
1154
|
- `redis_queued_locks.lock_obtained`;
|
1155
|
+
- `redis_queued_locks.extendable_reentrant_lock_obtained`;
|
1156
|
+
- `redis_queued_locks.reentrant_lock_obtained`;
|
1002
1157
|
- `redis_queued_locks.lock_hold_and_release`;
|
1158
|
+
- `redis_queued_locks.reentrant_lock_hold_completes`;
|
1003
1159
|
- `redis_queued_locks.explicit_lock_release`;
|
1004
1160
|
- `redis_queued_locks.explicit_all_locks_release`;
|
1005
1161
|
|
@@ -1012,9 +1168,31 @@ Detalized event semantics and payload structure:
|
|
1012
1168
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1013
1169
|
- `:acq_id` - `string` - lock acquier identifier;
|
1014
1170
|
- `:lock_key` - `string` - lock name;
|
1015
|
-
- `:ts` - `
|
1171
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend;
|
1172
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1173
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1174
|
+
|
1175
|
+
- `"redis_queued_locks.extendable_reentrant_lock_obtained"`
|
1176
|
+
- an event signalizes about the "extendable reentrant lock" obtaining is happened;
|
1177
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1178
|
+
- payload:
|
1179
|
+
- `:lock_key` - `string` - lock name;
|
1180
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1181
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1182
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as extendable reentrant lock;
|
1016
1183
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1017
|
-
- `:instrument` - `nil`/`Any` - custom data passed to the
|
1184
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1185
|
+
|
1186
|
+
- `"redis_queued_locks.reentrant_lock_obtained"`
|
1187
|
+
- an event signalizes about the "reentrant lock" obtaining is happened (without TTL extension);
|
1188
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1189
|
+
- payload:
|
1190
|
+
- `:lock_key` - `string` - lock name;
|
1191
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1192
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1193
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1194
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1195
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1018
1196
|
|
1019
1197
|
- `"redis_queued_locks.lock_hold_and_release"`
|
1020
1198
|
- an event signalizes about the "hold+and+release" process is finished;
|
@@ -1024,9 +1202,22 @@ Detalized event semantics and payload structure:
|
|
1024
1202
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1025
1203
|
- `:acq_id` - `string` - lock acquier identifier;
|
1026
1204
|
- `:lock_key` - `string` - lock name;
|
1027
|
-
- `:ts` - `
|
1205
|
+
- `:ts` - `numeric`/`epoch` - the time when lock was obtained;
|
1028
1206
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1029
|
-
- `:instrument` - `nil`/`Any` - custom data passed to the
|
1207
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1208
|
+
|
1209
|
+
- `"redis_queued_locks.reentrant_lock_hold_completes"`
|
1210
|
+
- an event signalizes about the "reentrant lock hold" is complete (both extendable and non-extendable);
|
1211
|
+
- lock re-entering can happen many times and this event happens for each of them separately;
|
1212
|
+
- raised from `#lock`/`#lock!` when the lock was obtained as reentrant lock;
|
1213
|
+
- payload:
|
1214
|
+
- `:hold_time` - `float`/`milliseconds` - lock hold time;
|
1215
|
+
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1216
|
+
- `:acq_id` - `string` - lock acquier identifier;
|
1217
|
+
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1218
|
+
- `:lock_key` - `string` - lock name;
|
1219
|
+
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1220
|
+
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
1030
1221
|
|
1031
1222
|
- `"redis_queued_locks.explicit_lock_release"`
|
1032
1223
|
- an event signalizes about the explicit lock release (invoked via `RedisQueuedLock#unlock`);
|
@@ -1051,18 +1242,29 @@ Detalized event semantics and payload structure:
|
|
1051
1242
|
|
1052
1243
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1053
1244
|
|
1054
|
-
- **
|
1055
|
-
-
|
1056
|
-
-
|
1057
|
-
-
|
1058
|
-
|
1059
|
-
|
1060
|
-
|
1061
|
-
|
1062
|
-
-
|
1063
|
-
-
|
1064
|
-
-
|
1065
|
-
-
|
1245
|
+
- **Major**:
|
1246
|
+
- support for Random Access strategy (non-queued behavior);
|
1247
|
+
- lock request prioritization;
|
1248
|
+
- **strict redlock algorithm support** (support for many `RedisClient` instances);
|
1249
|
+
- `#lock_series` - acquire a series of locks:
|
1250
|
+
```ruby
|
1251
|
+
rql.lock_series('lock_a', 'lock_b', 'lock_c') { puts 'locked' }
|
1252
|
+
```
|
1253
|
+
- support for `Dragonfly` database backend (https://github.com/dragonflydb/dragonfly) (https://www.dragonflydb.io/);
|
1254
|
+
- **Minor**:
|
1255
|
+
- Semantic error objects for unexpected Redis errors;
|
1256
|
+
- change all `::Process.clock_gettime(::Process::CLOCK_MONOTONIC)` milliseconds-related invocations to
|
1257
|
+
`::Process.clock_gettime(::Process::CLOCK_MONOTONIC, :millisecond)` in order to reduce time-related math operations count;
|
1258
|
+
- **Experimental feature**: (non-`timed` locks): per-ruby-block-holding-the-lock sidecar `Ractor` and `in progress queue` in RedisDB that will extend
|
1259
|
+
the acquired lock for long-running blocks of code (that invoked "under" the lock
|
1260
|
+
whose ttl may expire before the block execution completes). It makes sense for non-`timed` locks *only*;
|
1261
|
+
- better code stylization (+ some refactorings);
|
1262
|
+
- `RedisQueuedLocks::Acquier::Try.try_to_lock` - detailed successful result analization;
|
1263
|
+
- Support for LIFO strategy;
|
1264
|
+
- better specs with 100% test coverage (total specs rework);
|
1265
|
+
- statistics with UI;
|
1266
|
+
- JSON log formatter;
|
1267
|
+
- `go`-lang implementation;
|
1066
1268
|
|
1067
1269
|
---
|
1068
1270
|
|
@@ -1,7 +1,7 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
# @api private
|
4
|
-
# @since
|
4
|
+
# @since 1.0.0
|
5
5
|
module RedisQueuedLocks::Acquier::AcquireLock::DelayExecution
|
6
6
|
# Sleep with random time-shifting (it is necessary for empty lock-acquirement time slots).
|
7
7
|
#
|
@@ -10,7 +10,7 @@ module RedisQueuedLocks::Acquier::AcquireLock::DelayExecution
|
|
10
10
|
# @return [void]
|
11
11
|
#
|
12
12
|
# @api private
|
13
|
-
# @since
|
13
|
+
# @since 1.0.0
|
14
14
|
def delay_execution(retry_delay, retry_jitter)
|
15
15
|
delay = (retry_delay + rand(retry_jitter)).to_f / 1_000
|
16
16
|
sleep(delay)
|