redis_queued_locks 1.8.0 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.ruby-version +1 -1
- data/CHANGELOG.md +66 -0
- data/README.md +528 -51
- data/lib/redis_queued_locks/acquier/acquire_lock/dequeue_from_lock_queue/log_visitor.rb +4 -0
- data/lib/redis_queued_locks/acquier/acquire_lock/dequeue_from_lock_queue.rb +4 -1
- data/lib/redis_queued_locks/acquier/acquire_lock/instr_visitor.rb +20 -5
- data/lib/redis_queued_locks/acquier/acquire_lock/log_visitor.rb +24 -0
- data/lib/redis_queued_locks/acquier/acquire_lock/try_to_lock/log_visitor.rb +56 -0
- data/lib/redis_queued_locks/acquier/acquire_lock/try_to_lock.rb +37 -30
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_expire/log_visitor.rb +8 -0
- data/lib/redis_queued_locks/acquier/acquire_lock/yield_expire.rb +24 -12
- data/lib/redis_queued_locks/acquier/acquire_lock.rb +44 -20
- data/lib/redis_queued_locks/acquier/clear_dead_requests.rb +5 -1
- data/lib/redis_queued_locks/acquier/extend_lock_ttl.rb +5 -1
- data/lib/redis_queued_locks/acquier/lock_info.rb +4 -3
- data/lib/redis_queued_locks/acquier/locks.rb +2 -2
- data/lib/redis_queued_locks/acquier/queue_info.rb +2 -2
- data/lib/redis_queued_locks/acquier/release_all_locks.rb +12 -2
- data/lib/redis_queued_locks/acquier/release_lock.rb +12 -2
- data/lib/redis_queued_locks/client.rb +284 -34
- data/lib/redis_queued_locks/errors.rb +8 -0
- data/lib/redis_queued_locks/instrument.rb +8 -1
- data/lib/redis_queued_locks/logging.rb +8 -1
- data/lib/redis_queued_locks/resource.rb +59 -1
- data/lib/redis_queued_locks/swarm/acquirers.rb +44 -0
- data/lib/redis_queued_locks/swarm/flush_zombies.rb +133 -0
- data/lib/redis_queued_locks/swarm/probe_hosts.rb +69 -0
- data/lib/redis_queued_locks/swarm/redis_client_builder.rb +67 -0
- data/lib/redis_queued_locks/swarm/supervisor.rb +83 -0
- data/lib/redis_queued_locks/swarm/swarm_element/isolated.rb +287 -0
- data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +351 -0
- data/lib/redis_queued_locks/swarm/swarm_element.rb +8 -0
- data/lib/redis_queued_locks/swarm/zombie_info.rb +145 -0
- data/lib/redis_queued_locks/swarm.rb +246 -0
- data/lib/redis_queued_locks/utilities/lock.rb +22 -0
- data/lib/redis_queued_locks/utilities.rb +75 -0
- data/lib/redis_queued_locks/version.rb +2 -2
- data/lib/redis_queued_locks.rb +2 -0
- metadata +14 -4
- data/lib/redis_queued_locks/watcher.rb +0 -1
data/README.md
CHANGED
@@ -35,12 +35,30 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
|
|
35
35
|
- [queues_info](#queues_info---get-list-of-queues-with-their-info)
|
36
36
|
- [clear_dead_requests](#clear_dead_requests)
|
37
37
|
- [current_acquirer_id](#current_acquirer_id)
|
38
|
+
- [current_host_id](#current_host_id)
|
39
|
+
- [possible_host_ids](#possible_host_ids)
|
40
|
+
- [Swarm Mode and Zombie Locks](#swarm-mode-and-zombie-locks)
|
41
|
+
- [work and usage preview (temporary example-based docs)](#work-and-usage-preview-temporary-example-based-docs)
|
42
|
+
- [How to Swarm](#how-to-swarm)
|
43
|
+
- [configuration](#)
|
44
|
+
- [swarm_status](#swarm_status)
|
45
|
+
- [swarm_info](#swarm_info)
|
46
|
+
- [swarmize!](#swarmize!)
|
47
|
+
- [deswarmize!](#deswarmize!)
|
48
|
+
- [probe_hosts](#probe_hosts)
|
49
|
+
- [flush_zobmies](#flush_zombies)
|
50
|
+
- [zombies_info](#zombies_info)
|
51
|
+
- [zombie_locks](#zombie_locks)
|
52
|
+
- [zombie_hosts](#zombie_hosts)
|
53
|
+
- [zombie_acquiers](#zombie_acquiers)
|
38
54
|
- [Lock Access Strategies](#lock-access-strategies)
|
39
55
|
- [queued](#lock-access-strategies)
|
40
56
|
- [random](#lock-access-strategies)
|
41
|
-
- [
|
57
|
+
- [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks)
|
42
58
|
- [Logging](#logging)
|
59
|
+
- [Logging Configuration](#logging-configuration)
|
43
60
|
- [Instrumentation](#instrumentation)
|
61
|
+
- [Instrumentation Configuration](#instrumentation-configuration)
|
44
62
|
- [Instrumentation Events](#instrumentation-events)
|
45
63
|
- [Roadmap](#roadmap)
|
46
64
|
- [Contributing](#contributing)
|
@@ -244,35 +262,35 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
244
262
|
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
245
263
|
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
246
264
|
# - at this moment the only debug logs are realised in following cases:
|
247
|
-
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
248
|
-
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
249
|
-
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
250
|
-
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time", "acs_strat");
|
251
|
-
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time", "acs_strat");
|
252
|
-
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acq_time", "acs_strat");
|
253
|
-
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
254
|
-
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
255
|
-
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "acs_strat");
|
265
|
+
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
266
|
+
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
267
|
+
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
268
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time", "acs_strat");
|
269
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time", "acs_strat");
|
270
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time", "acs_strat");
|
271
|
+
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
272
|
+
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
273
|
+
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
256
274
|
# - by default uses VoidLogger that does nothing;
|
257
275
|
config.logger = RedisQueuedLocks::Logging::VoidLogger
|
258
276
|
|
259
277
|
# (default: false)
|
260
278
|
# - adds additional debug logs;
|
261
279
|
# - enables additional logs for each internal try-retry lock acquiring (a lot of logs can be generated depending on your retry configurations);
|
262
|
-
# - it adds following logs in addition to the existing:
|
263
|
-
# - "[redis_queued_locks.try_lock.start]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
264
|
-
# - "[redis_queued_locks.try_lock.rconn_fetched]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
265
|
-
# - "[redis_queued_locks.try_lock.same_process_conflict_detected]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
266
|
-
# - "[redis_queued_locks.try_lock.same_process_conflict_analyzed]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status");
|
267
|
-
# - "[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status", "last_ext_ttl", "last_ext_ts");
|
268
|
-
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status", last_spc_ts);
|
269
|
-
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id, "acs_strat")";
|
270
|
-
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
271
|
-
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "first_acq_id_in_queue");
|
272
|
-
# - "[redis_queued_locks.try_lock.exit__queue_ttl_reached]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
273
|
-
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "first_acq_id_in_queue", "<current_lock_data>");
|
274
|
-
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
275
|
-
# - "[redis_queued_locks.try_lock.obtain__free_to_acquire]" (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
280
|
+
# - it adds following debug logs in addition to the existing:
|
281
|
+
# - "[redis_queued_locks.try_lock.start]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
282
|
+
# - "[redis_queued_locks.try_lock.rconn_fetched]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
283
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_detected]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
284
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_analyzed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status");
|
285
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", "last_ext_ttl", "last_ext_ts");
|
286
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", last_spc_ts);
|
287
|
+
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat")";
|
288
|
+
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
289
|
+
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue");
|
290
|
+
# - "[redis_queued_locks.try_lock.exit__queue_ttl_reached]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
291
|
+
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue", "<current_lock_data>");
|
292
|
+
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
293
|
+
# - "[redis_queued_locks.try_lock.obtain__free_to_acquire]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
276
294
|
config.log_lock_try = false
|
277
295
|
|
278
296
|
# (default: false)
|
@@ -341,6 +359,8 @@ end
|
|
341
359
|
- [queues_info](#queues_info---get-list-of-queues-with-their-info)
|
342
360
|
- [clear_dead_requests](#clear_dead_requests)
|
343
361
|
- [current_acquirer_id](#current_acquirer_id)
|
362
|
+
- [current_host_id](#current_host_id)
|
363
|
+
- [possible_host_ids](#possible_host_ids)
|
344
364
|
|
345
365
|
---
|
346
366
|
|
@@ -381,9 +401,11 @@ def lock(
|
|
381
401
|
log_sampling_enabled: config[:log_sampling_enabled],
|
382
402
|
log_sampling_percent: config[:log_sampling_percent],
|
383
403
|
log_sampler: config[:log_sampler],
|
404
|
+
log_sample_this: false,
|
384
405
|
instr_sampling_enabled: config[:instr_sampling_enabled],
|
385
406
|
instr_sampling_percent: config[:instr_sampling_percent],
|
386
407
|
instr_sampler: config[:instr_sampler],
|
408
|
+
instr_sample_this: false,
|
387
409
|
&block
|
388
410
|
)
|
389
411
|
```
|
@@ -447,7 +469,7 @@ def lock(
|
|
447
469
|
- `:extendable_work_through` - continue working under the lock **with** lock's TTL extension;
|
448
470
|
- `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
|
449
471
|
- `:dead_locking` - fail with deadlock exception;
|
450
|
-
- See [
|
472
|
+
- See [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks) documentation section for details;
|
451
473
|
- `identity` - (optional) `[String]`
|
452
474
|
- An unique string that is unique per `RedisQueuedLock::Client` instance. Resolves the
|
453
475
|
collisions between the same process_id/thread_id/fiber_id/ractor_id identifiers on different
|
@@ -503,6 +525,10 @@ def lock(
|
|
503
525
|
- you can provide your own log sampler with bettter algorithm that should realize
|
504
526
|
`sampling_happened?(percent) => boolean` interface (see `RedisQueuedLocks::Logging::Sampler` for example);
|
505
527
|
- pre-configured in `config[:log_sampler]`;
|
528
|
+
- `log_sample_this` - (optional) `[Boolean]`
|
529
|
+
- marks the method that everything should be logged despite the enabled log sampling;
|
530
|
+
- makes sense when log sampling is enabled;
|
531
|
+
- `false` by default;
|
506
532
|
- `instr_sampling_enabled` - (optional) `[Boolean]`
|
507
533
|
- enables **instrumentaion sampling**: only the configured percent of RQL cases will be instrumented;
|
508
534
|
- disabled by default;
|
@@ -521,6 +547,10 @@ def lock(
|
|
521
547
|
- you can provide your own log sampler with bettter algorithm that should realize
|
522
548
|
`sampling_happened?(percent) => boolean` interface (see `RedisQueuedLocks::Instrument::Sampler` for example);
|
523
549
|
- pre-configured in `config[:instr_sampler]`;
|
550
|
+
- `instr_sample_this` - (optional) `[Boolean]`
|
551
|
+
- marks the method that everything should be instrumneted despite the enabled instrumentation sampling;
|
552
|
+
- makes sense when instrumentation sampling is enabled;
|
553
|
+
- `false` by default;
|
524
554
|
- `block` - (optional) `[Block]`
|
525
555
|
- A block of code that should be executed after the successfully acquired lock.
|
526
556
|
- If block is **passed** the obtained lock will be released after the block execution or it's ttl (what will happen first);
|
@@ -558,7 +588,7 @@ Return value:
|
|
558
588
|
- `:extendable_conflict_work_through` - reentrant lock acquiring process with lock's TTL extension. Suitable for `conflict_strategy: :extendable_work_through`;
|
559
589
|
- `:conflict_work_through` - reentrant lock acquiring process without lock's TTL extension. Suitable for `conflict_strategy: :work_through`;
|
560
590
|
- `:dead_locking` - current process tries to acquire a lock that is already acquired by them. Suitalbe for `conflict_startegy: :dead_locking`;
|
561
|
-
- For more details see [
|
591
|
+
- For more details see [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks) readme section;
|
562
592
|
- For successful lock obtaining:
|
563
593
|
```ruby
|
564
594
|
{
|
@@ -566,6 +596,7 @@ Return value:
|
|
566
596
|
result: {
|
567
597
|
lock_key: String, # acquierd lock key ("rql:lock:your_lock_name")
|
568
598
|
acq_id: String, # acquier identifier ("process_id/thread_id/fiber_id/ractor_id/identity")
|
599
|
+
hst_id: String, # host identifier ("process_id/thread_id/ractor_id/identity")
|
569
600
|
ts: Float, # time (epoch) when lock was obtained (float, Time#to_f)
|
570
601
|
ttl: Integer, # lock's time to live in milliseconds (integer)
|
571
602
|
process: Symbol # which logical process has acquired the lock (:lock_obtaining, :extendable_conflict_work_through, :conflict_work_through, :conflict_dead_lock)
|
@@ -580,6 +611,7 @@ Return value:
|
|
580
611
|
result: {
|
581
612
|
lock_key: "rql:lock:my_lock",
|
582
613
|
acq_id: "rql:acq:26672/2280/2300/2320/70ea5dbf10ea1056",
|
614
|
+
acq_id: "rql:acq:26672/2280/2320/70ea5dbf10ea1056",
|
583
615
|
ts: 1711909612.653696,
|
584
616
|
ttl: 10000,
|
585
617
|
process: :lock_obtaining # for custom conflict strategies may be: :conflict_dead_lock, :conflict_work_through, :extendable_conflict_work_through
|
@@ -655,6 +687,7 @@ rql.lock_info("my_lock")
|
|
655
687
|
{
|
656
688
|
"lock_key" => "rql:lock:my_lock",
|
657
689
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
690
|
+
"hst_id" => "rql:acq:123/456/678/374dd74324",
|
658
691
|
"ts" => 123456789,
|
659
692
|
"ini_ttl" => 123456,
|
660
693
|
"rem_ttl" => 123440,
|
@@ -755,7 +788,7 @@ rql.lock('my_lock', retry_delay: 3000, ttl: 3000, access_strategy: :random)
|
|
755
788
|
- (`RedisQueuedLocks::LockAlreadyObtainedError`) when `fail_fast` is `true` and lock is already obtained;
|
756
789
|
- (`RedisQueuedLocks::LockAcquiermentTimeoutError`) `timeout` limit reached before lock is obtained;
|
757
790
|
- (`RedisQueuedLocks::LockAcquiermentRetryLimitError`) `retry_count` limit reached before lock is obtained;
|
758
|
-
- (`RedisQueuedLocks::ConflictLockObtainError`) when `conflict_strategy: :dead_locking` is used and the "same-process-dead-lock" is happened (see [
|
791
|
+
- (`RedisQueuedLocks::ConflictLockObtainError`) when `conflict_strategy: :dead_locking` is used and the "same-process-dead-lock" is happened (see [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks) for details);
|
759
792
|
|
760
793
|
```ruby
|
761
794
|
def lock!(
|
@@ -780,9 +813,11 @@ def lock!(
|
|
780
813
|
log_sampling_enabled: config[:log_sampling_enabled],
|
781
814
|
log_sampling_percent: config[:log_sampling_percent],
|
782
815
|
log_sampler: config[:log_sampler],
|
816
|
+
log_sample_this: false,
|
783
817
|
instr_sampling_enabled: config[:instr_sampling_enabled],
|
784
818
|
instr_sampling_percent: config[:instr_sampling_percent],
|
785
819
|
instr_sampler: config[:instr_sampler],
|
820
|
+
instr_sample_this: false,
|
786
821
|
&block
|
787
822
|
)
|
788
823
|
```
|
@@ -800,6 +835,7 @@ See `#lock` method [documentation](#lock---obtain-a-lock).
|
|
800
835
|
- lock data (`Hash<String,String|Integer>`):
|
801
836
|
- `"lock_key"` - `string` - lock key in redis;
|
802
837
|
- `"acq_id"` - `string` - acquier identifier (process_id/thread_id/fiber_id/ractor_id/identity);
|
838
|
+
- `"hst_id"` - `string` - host identifier (process_id/thread_id/ractor_id/identity);
|
803
839
|
- `"ts"` - `numeric`/`epoch` - the time when lock was obtained;
|
804
840
|
- `"init_ttl"` - `integer` - (milliseconds) initial lock key ttl;
|
805
841
|
- `"rem_ttl"` - `integer` - (milliseconds) remaining lock key ttl;
|
@@ -822,6 +858,7 @@ rql.lock_info("your_lock_name")
|
|
822
858
|
{
|
823
859
|
"lock_key" => "rql:lock:your_lock_name",
|
824
860
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
861
|
+
"hst_id" => "rql:acq:123/456/678/374dd74324",
|
825
862
|
"ts" => 123456789.12345,
|
826
863
|
"ini_ttl" => 5_000,
|
827
864
|
"rem_ttl" => 4_999
|
@@ -837,6 +874,7 @@ rql.lock_info("your_lock_name")
|
|
837
874
|
{
|
838
875
|
"lock_key" => "rql:lock:your_lock_name",
|
839
876
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
877
|
+
"hst_id" => "rql:acq:123/456/678/374dd74324",
|
840
878
|
"ts" => 123456789.12345,
|
841
879
|
"ini_ttl" => 5_000,
|
842
880
|
"rem_ttl" => 4_999,
|
@@ -858,6 +896,7 @@ rql.lock_info("your_lock_name")
|
|
858
896
|
{
|
859
897
|
"lock_key" => "rql:lock:your_lock_name",
|
860
898
|
"acq_id" => "rql:acq:123/456/567/678/374dd74324",
|
899
|
+
"hst_id" => "rql:acq:123/456/678/374dd74324",
|
861
900
|
"ts" => 123456789.12345,
|
862
901
|
"ini_ttl" => 5_000,
|
863
902
|
"rem_ttl" => 9_444,
|
@@ -943,6 +982,7 @@ rql.queued?("your_lock_name") # => true/false
|
|
943
982
|
|
944
983
|
- release the concrete lock with lock request queue;
|
945
984
|
- queue will be relased first;
|
985
|
+
- has an alias: `#release_lock`;
|
946
986
|
- accepts:
|
947
987
|
- `lock_name` - (required) `[String]` - the lock name that should be released.
|
948
988
|
- `:logger` - (optional) `[::Logger,#debug]`
|
@@ -963,6 +1003,10 @@ rql.queued?("your_lock_name") # => true/false
|
|
963
1003
|
- `:log_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Logging::Sampler>]`
|
964
1004
|
- **log sampling**: percent-based log sampler that decides should be RQL case logged or not;
|
965
1005
|
- pre-configured in `config[:log_sampler]`;
|
1006
|
+
- `log_sample_this` - (optional) `[Boolean]`
|
1007
|
+
- marks the method that everything should be logged despite the enabled log sampling;
|
1008
|
+
- makes sense when log sampling is enabled;
|
1009
|
+
- `false` by default;
|
966
1010
|
- `:instr_sampling_enabled` - (optional) `[Boolean]`
|
967
1011
|
- enables **instrumentaion sampling**;
|
968
1012
|
- pre-configured in `config[:instr_sampling_enabled]`;
|
@@ -972,6 +1016,10 @@ rql.queued?("your_lock_name") # => true/false
|
|
972
1016
|
- `instr_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Instrument::Sampler>]`
|
973
1017
|
- percent-based log sampler that decides should be RQL case instrumented or not;
|
974
1018
|
- pre-configured in `config[:instr_sampler]`;
|
1019
|
+
- `instr_sample_this` - (optional) `[Boolean]`
|
1020
|
+
- marks the method that everything should be instrumneted despite the enabled instrumentation sampling;
|
1021
|
+
- makes sense when instrumentation sampling is enabled;
|
1022
|
+
- `false` by default;
|
975
1023
|
- if you try to unlock non-existent lock you will receive `ok: true` result with operation timings
|
976
1024
|
and `:nothing_to_release` result factor inside;
|
977
1025
|
|
@@ -1010,6 +1058,7 @@ rql.unlock("your_lock_name")
|
|
1010
1058
|
|
1011
1059
|
- release all obtained locks and related lock request queues;
|
1012
1060
|
- queues will be released first;
|
1061
|
+
- has an alias: `#release_locks`;
|
1013
1062
|
- accepts:
|
1014
1063
|
- `:batch_size` - (optional) `[Integer]`
|
1015
1064
|
- the size of batch of locks and lock queus that should be cleared under the one pipelined redis command at once;
|
@@ -1031,6 +1080,10 @@ rql.unlock("your_lock_name")
|
|
1031
1080
|
- `:log_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Logging::Sampler>]`
|
1032
1081
|
- **log sampling**: percent-based log sampler that decides should be RQL case logged or not;
|
1033
1082
|
- pre-configured in `config[:log_sampler]`;
|
1083
|
+
- `log_sample_this` - (optional) `[Boolean]`
|
1084
|
+
- marks the method that everything should be logged despite the enabled log sampling;
|
1085
|
+
- makes sense when log sampling is enabled;
|
1086
|
+
- `false` by default;
|
1034
1087
|
- `:instr_sampling_enabled` - (optional) `[Boolean]`
|
1035
1088
|
- enables **instrumentaion sampling**;
|
1036
1089
|
- pre-configured in `config[:instr_sampling_enabled]`;
|
@@ -1040,6 +1093,10 @@ rql.unlock("your_lock_name")
|
|
1040
1093
|
- `instr_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Instrument::Sampler>]`
|
1041
1094
|
- percent-based log sampler that decides should be RQL case instrumented or not;
|
1042
1095
|
- pre-configured in `config[:instr_sampler]`;
|
1096
|
+
- `instr_sample_this` - (optional) `[Boolean]`
|
1097
|
+
- marks the method that everything should be instrumneted despite the enabled instrumentation sampling;
|
1098
|
+
- makes sense when instrumentation sampling is enabled;
|
1099
|
+
- `false` by default;
|
1043
1100
|
- returns:
|
1044
1101
|
- `[Hash<Symbol,Numeric>]` - Format: `{ ok: true, result: Hash<Symbol,Numeric> }`;
|
1045
1102
|
- result data:
|
@@ -1090,6 +1147,10 @@ rql.clear_locks
|
|
1090
1147
|
- `:log_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Logging::Sampler>]`
|
1091
1148
|
- **log sampling**: percent-based log sampler that decides should be RQL case logged or not;
|
1092
1149
|
- pre-configured in `config[:log_sampler]`;
|
1150
|
+
- `log_sample_this` - (optional) `[Boolean]`
|
1151
|
+
- marks the method that everything should be logged despite the enabled log sampling;
|
1152
|
+
- makes sense when log sampling is enabled;
|
1153
|
+
- `false` by default;
|
1093
1154
|
- `:instr_sampling_enabled` - (optional) `[Boolean]`
|
1094
1155
|
- enables **instrumentaion sampling**;
|
1095
1156
|
- pre-configured in `config[:instr_sampling_enabled]`;
|
@@ -1099,6 +1160,10 @@ rql.clear_locks
|
|
1099
1160
|
- `instr_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Instrument::Sampler>]`
|
1100
1161
|
- percent-based log sampler that decides should be RQL case instrumented or not;
|
1101
1162
|
- pre-configured in `config[:instr_sampler]`;
|
1163
|
+
- `instr_sample_this` - (optional) `[Boolean]`
|
1164
|
+
- marks the method that everything should be instrumneted despite the enabled instrumentation sampling;
|
1165
|
+
- makes sense when instrumentation sampling is enabled;
|
1166
|
+
- `false` by default;
|
1102
1167
|
- returns `{ ok: true, result: :ttl_extended }` when ttl is extended;
|
1103
1168
|
- returns `{ ok: false, result: :async_expire_or_no_lock }` when a lock not found or a lock is already expired during
|
1104
1169
|
some steps of invocation (see **Important** section below);
|
@@ -1253,6 +1318,7 @@ rql.locks_info # or rql.locks_info(scan_size: 123)
|
|
1253
1318
|
:status=>:alive,
|
1254
1319
|
:info=>{
|
1255
1320
|
"acq_id"=>"rql:acq:41478/4320/4340/4360/848818f09d8c3420",
|
1321
|
+
"hst_id"=>"rql:hst:41478/4320/4360/848818f09d8c3420"
|
1256
1322
|
"ts"=>1711607112.670343,
|
1257
1323
|
"ini_ttl"=>15000,
|
1258
1324
|
"rem_ttl"=>13998}},
|
@@ -1336,6 +1402,10 @@ Accepts:
|
|
1336
1402
|
- `:log_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Logging::Sampler>]`
|
1337
1403
|
- **log sampling**: percent-based log sampler that decides should be RQL case logged or not;
|
1338
1404
|
- pre-configured in `config[:log_sampler]`;
|
1405
|
+
- `log_sample_this` - (optional) `[Boolean]`
|
1406
|
+
- marks the method that everything should be logged despite the enabled log sampling;
|
1407
|
+
- makes sense when log sampling is enabled;
|
1408
|
+
- `false` by default;
|
1339
1409
|
- `:instr_sampling_enabled` - (optional) `[Boolean]`
|
1340
1410
|
- enables **instrumentaion sampling**;
|
1341
1411
|
- pre-configured in `config[:instr_sampling_enabled]`;
|
@@ -1345,6 +1415,10 @@ Accepts:
|
|
1345
1415
|
- `instr_sampler` - (optional) `[#sampling_happened?,Module<RedisQueuedLocks::Instrument::Sampler>]`
|
1346
1416
|
- percent-based log sampler that decides should be RQL case instrumented or not;
|
1347
1417
|
- pre-configured in `config[:instr_sampler]`;
|
1418
|
+
- `instr_sample_this` - (optional) `[Boolean]`
|
1419
|
+
- marks the method that everything should be instrumneted despite the enabled instrumentation sampling;
|
1420
|
+
- makes sense when instrumentation sampling is enabled;
|
1421
|
+
- `false` by default;
|
1348
1422
|
|
1349
1423
|
Returns: `{ ok: true, processed_queues: Set<String> }` returns the list of processed lock queues;
|
1350
1424
|
|
@@ -1367,6 +1441,8 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
1367
1441
|
|
1368
1442
|
#### #current_acquirer_id
|
1369
1443
|
|
1444
|
+
<sup>\[[back to top](#usage)\]</sup>
|
1445
|
+
|
1370
1446
|
- get the current acquirer identifier in RQL notation that you can use for debugging purposes during the lock analyzation;
|
1371
1447
|
- acquirer identifier format:
|
1372
1448
|
```ruby
|
@@ -1400,6 +1476,278 @@ rql.current_acquirer_id
|
|
1400
1476
|
|
1401
1477
|
---
|
1402
1478
|
|
1479
|
+
#### #current_host_id
|
1480
|
+
|
1481
|
+
<sup>\[[back to top](#usage)\]</sup>
|
1482
|
+
|
1483
|
+
- get a current host identifier in RQL notation that you can use for debugging purposes during the lock analyzis;
|
1484
|
+
- the host is a ruby worker (a combination of process/thread/ractor/identity) that is alive and can obtain locks;
|
1485
|
+
- the host is limited to `process`/`thread`/`ractor` (without `fiber`) combination cuz we have no abilities to extract
|
1486
|
+
all fiber objects from the current ruby process when at least one ractor object is defined (**ObjectSpace** loses
|
1487
|
+
abilities to extract `Fiber` and `Thread` objects after the any ractor is created) (`Thread` objects are analyzed
|
1488
|
+
via `Thread.list` API which does not lose their abilites);
|
1489
|
+
- host identifier format:
|
1490
|
+
```ruby
|
1491
|
+
"rql:hst:#{process_id}/#{thread_id}/#{ractor_id}/#{uniq_identity}"
|
1492
|
+
```
|
1493
|
+
- because of the moment that `#lock`/`#lock!` gives you a possibility to customize `process_id`,
|
1494
|
+
`fiber_id`, `thread_id`, `ractor_id` and `unique identity` identifiers the `#current_host_id` method provides this possibility too
|
1495
|
+
(except the `fiber_id` correspondingly);
|
1496
|
+
|
1497
|
+
Accepts:
|
1498
|
+
|
1499
|
+
- `process_id:` - (optional) `[Integer,Any]`
|
1500
|
+
- `::Process.pid` by default;
|
1501
|
+
- `thread_id:` - (optional) `[Integer,Any]`;
|
1502
|
+
- `::Thread.current.object_id` by default;
|
1503
|
+
- `ractor_id:` - (optional) `[Integer,Any]`;
|
1504
|
+
- `::Ractor.current.object_id` by default;
|
1505
|
+
- `identity:` - (optional) `[String]`;
|
1506
|
+
- this value is calculated once during `RedisQueuedLock::Client` instantiation and stored in `@uniq_identity`;
|
1507
|
+
- this value can be accessed from `RedisQueuedLock::Client#uniq_identity`;
|
1508
|
+
- [Configuration](#configuration) documentation: see `config[:uniq_identifier]`;
|
1509
|
+
- [#lock](#lock---obtain-a-lock) method documentation: see `uniq_identifier`;
|
1510
|
+
|
1511
|
+
```ruby
|
1512
|
+
rql.current_host_id
|
1513
|
+
|
1514
|
+
# =>
|
1515
|
+
"rql:acq:38529/4500/4360/66093702f24a3129"
|
1516
|
+
```
|
1517
|
+
|
1518
|
+
---
|
1519
|
+
|
1520
|
+
#### #possible_host_ids
|
1521
|
+
|
1522
|
+
<sup>\[[back to top](#usage)\]</sup>
|
1523
|
+
|
1524
|
+
- return the list (`Array<String>`) of possible host identifiers that can be reached from the current ractor;
|
1525
|
+
- the host is a ruby worker (a combination of process/thread/ractor/identity) that is alive and can obtain locks;
|
1526
|
+
- the host is limited to `process`/`thread`/`ractor` (without `fiber`) combination cuz we have no abilities to extract
|
1527
|
+
all fiber objects from the current ruby process when at least one ractor object is defined (**ObjectSpace** loses
|
1528
|
+
abilities to extract `Fiber` and `Thread` objects after the any ractor is created) (`Thread` objects are analyzed
|
1529
|
+
via `Thread.list` API which does not lose their abilites);
|
1530
|
+
- host identifier format:
|
1531
|
+
```ruby
|
1532
|
+
"rql:hst:#{process_id}/#{thread_id}/#{ractor_id}/#{uniq_identity}"
|
1533
|
+
```
|
1534
|
+
|
1535
|
+
Accepts:
|
1536
|
+
|
1537
|
+
- `identity` - (optional) `[String]`;
|
1538
|
+
- this value is calculated once during `RedisQueuedLock::Client` instantiation and stored in `@uniq_identity`;
|
1539
|
+
- this value can be accessed from `RedisQueuedLock::Client#uniq_identity`;
|
1540
|
+
- [Configuration](#configuration) documentation: see `config[:uniq_identifier]`;
|
1541
|
+
- [#lock](#lock---obtain-a-lock) method documentation: see `uniq_identifier`;
|
1542
|
+
|
1543
|
+
```ruby
|
1544
|
+
rql.possible_host_ids
|
1545
|
+
|
1546
|
+
# =>
|
1547
|
+
[
|
1548
|
+
"rql:hst:18814/2300/2280/5ce0c4582fc59c06", # process id / thread id / ractor id / uniq identity
|
1549
|
+
"rql:hst:18814/2320/2280/5ce0c4582fc59c06", # ...
|
1550
|
+
"rql:hst:18814/2340/2280/5ce0c4582fc59c06", # ...
|
1551
|
+
"rql:hst:18814/2360/2280/5ce0c4582fc59c06", # ...
|
1552
|
+
"rql:hst:18814/2380/2280/5ce0c4582fc59c06", # ...
|
1553
|
+
"rql:hst:18814/2400/2280/5ce0c4582fc59c06"
|
1554
|
+
]
|
1555
|
+
```
|
1556
|
+
---
|
1557
|
+
|
1558
|
+
## Swarm Mode and Zombie Locks
|
1559
|
+
|
1560
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1561
|
+
|
1562
|
+
> Eliminate zombie locks with a swarm.
|
1563
|
+
|
1564
|
+
**This documentation section is in progress!** (see the changelog and the usage preview for details at this moment)
|
1565
|
+
|
1566
|
+
[(work and usage preview (temporary example-based docs))](#work-and-usage-preview-temporary-example-based-docs)
|
1567
|
+
|
1568
|
+
- [How to Swarm](#how-to-swarm)
|
1569
|
+
- [configuration](#)
|
1570
|
+
- [swarm_status](#swarm_status)
|
1571
|
+
- [swarm_info](#swarm_info)
|
1572
|
+
- [swarmize!](#swarmize!)
|
1573
|
+
- [deswarmize!](#deswarmize!)
|
1574
|
+
- [probe_hosts](#probe_hosts)
|
1575
|
+
- [flush_zobmies](#flush_zombies)
|
1576
|
+
- [zombies_info](#zombies_info)
|
1577
|
+
- [zombie_locks](#zombie_locks)
|
1578
|
+
- [zombie_hosts](#zombie_hosts)
|
1579
|
+
- [zombie_acquiers](#zombie_acquiers)
|
1580
|
+
|
1581
|
+
<hr>
|
1582
|
+
|
1583
|
+
#### Work and Usage Preview (temporary example-based docs)
|
1584
|
+
|
1585
|
+
<sup>\[[back to top](#swarm-mode-and-zombie-locks)\]</sup>
|
1586
|
+
|
1587
|
+
<details>
|
1588
|
+
<summary>configuration</summary>
|
1589
|
+
|
1590
|
+
```ruby
|
1591
|
+
redis_client = RedisClient.config.new_pool # NOTE: provide your own RedisClient instance
|
1592
|
+
|
1593
|
+
clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
|
1594
|
+
# NOTE: auto-swarm your RQL client after initalization (run swarm elements and their supervisor)
|
1595
|
+
config.swarm.auto_swarm = false
|
1596
|
+
|
1597
|
+
# supervisor configs
|
1598
|
+
config.swarm.supervisor.liveness_probing_period = 2 # NOTE: in seconds
|
1599
|
+
|
1600
|
+
# (probe_hosts) host probing configuration
|
1601
|
+
config.swarm.probe_hosts.enabled_for_swarm = true # NOTE: run host-probing from or not
|
1602
|
+
config.swarm.probe_hosts.probe_period = 2 # NOTE: (in seconds) the period of time when the probing process is triggered
|
1603
|
+
# (probe_hosts) individual redis config
|
1604
|
+
config.swarm.probe_hosts.redis_config.sentinel = false # NOTE: individual redis config
|
1605
|
+
config.swarm.probe_hosts.redis_config.pooled = false # NOTE: individual redis config
|
1606
|
+
config.swarm.probe_hosts.redis_config.config = {} # NOTE: individual redis config
|
1607
|
+
config.swarm.probe_hosts.redis_config.pool_config = {} # NOTE: individual redis config
|
1608
|
+
|
1609
|
+
# (flush_zombies) zombie flushing configuration
|
1610
|
+
config.swarm.flush_zombies.enabled_for_swarm = true # NOTE: run zombie flushing or not
|
1611
|
+
config.swarm.flush_zombies.zombie_flush_period = 10 # NOTE: (in seconds) period of time when the zombie flusher is triggered
|
1612
|
+
config.swarm.flush_zombies.zombie_ttl = 15_000 # NOTE: (in milliseconds) when the lock/host/acquier is considered a zombie
|
1613
|
+
config.swarm.flush_zombies.zombie_lock_scan_size = 500 # NOTE: scan sizec during zombie flushing
|
1614
|
+
config.swarm.flush_zombies.zombie_queue_scan_size = 500 # NOTE: scan sizec during zombie flushing
|
1615
|
+
# (flush_zombies) individual redis config
|
1616
|
+
config.swarm.flush_zombies.redis_config.sentinel = false # NOTE: individual redis config
|
1617
|
+
config.swarm.flush_zombies.redis_config.pooled = false # NOTE: individual redis config
|
1618
|
+
config.swarm.flush_zombies.redis_config.config = {} # NOTE: individual redis config
|
1619
|
+
config.swarm.flush_zombies.redis_config.pool_config = {} # NOTE: individual redis config
|
1620
|
+
end
|
1621
|
+
```
|
1622
|
+
</details>
|
1623
|
+
|
1624
|
+
<details>
|
1625
|
+
<summary>seed a zombie</summary>
|
1626
|
+
|
1627
|
+
- obtain some long living lock and kill the host process which will lead the lock becoming a zombie:
|
1628
|
+
|
1629
|
+
```ruby
|
1630
|
+
daiver => ~/Projects/redis_queued_locks master [$]
|
1631
|
+
➜ bin/console
|
1632
|
+
[1] pry(main)> rql = RedisQueuedLocks::Client.new(RedisClient.new);
|
1633
|
+
[2] pry(main)> rql.swarmize!
|
1634
|
+
/Users/daiver/Projects/redis_queued_locks/lib/redis_queued_locks/swarm/flush_zombies.rb:107: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.
|
1635
|
+
=> {:ok=>true, :result=>:swarming}
|
1636
|
+
[3] pry(main)> rql.lock('kekpek', ttl: 1111111111)
|
1637
|
+
=> {:ok=>true,
|
1638
|
+
:result=>
|
1639
|
+
{:lock_key=>"rql:lock:kekpek",
|
1640
|
+
:acq_id=>"rql:acq:17580/2260/2380/2280/3f16b93973612580",
|
1641
|
+
:hst_id=>"rql:hst:17580/2260/2280/3f16b93973612580",
|
1642
|
+
:ts=>1720305351.069259,
|
1643
|
+
:ttl=>1111111111,
|
1644
|
+
:process=>:lock_obtaining}}
|
1645
|
+
[4] pry(main)> exit
|
1646
|
+
```
|
1647
|
+
</details>
|
1648
|
+
|
1649
|
+
<details>
|
1650
|
+
<summary>find zombies</summary>
|
1651
|
+
|
1652
|
+
- start another process, fetch the swarm info, see that our last process is a zombie now and their hosted lock is a zombie too:
|
1653
|
+
|
1654
|
+
```ruby
|
1655
|
+
daiver => ~/Projects/redis_queued_locks master [$] took 27.2s
|
1656
|
+
➜ bin/console
|
1657
|
+
[1] pry(main)> rql = RedisQueuedLocks::Client.new(RedisClient.new);
|
1658
|
+
[2] pry(main)> rql.swarm_info
|
1659
|
+
=> {"rql:hst:17580/2260/2280/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 12897/262144 +0300, :last_probe_score=>1720305353.0491982},
|
1660
|
+
"rql:hst:17580/2300/2280/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 211107/4194304 +0300, :last_probe_score=>1720305353.0503318},
|
1661
|
+
"rql:hst:17580/2320/2280/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 106615/2097152 +0300, :last_probe_score=>1720305353.050838},
|
1662
|
+
"rql:hst:17580/2260/2340/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 26239/524288 +0300, :last_probe_score=>1720305353.050047},
|
1663
|
+
"rql:hst:17580/2300/2340/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 106359/2097152 +0300, :last_probe_score=>1720305353.050716},
|
1664
|
+
"rql:hst:17580/2320/2340/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 213633/4194304 +0300, :last_probe_score=>1720305353.050934},
|
1665
|
+
"rql:hst:17580/2360/2280/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 214077/4194304 +0300, :last_probe_score=>1720305353.05104},
|
1666
|
+
"rql:hst:17580/2360/2340/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 214505/4194304 +0300, :last_probe_score=>1720305353.051142},
|
1667
|
+
"rql:hst:17580/2400/2280/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 53729/1048576 +0300, :last_probe_score=>1720305353.05124},
|
1668
|
+
"rql:hst:17580/2400/2340/3f16b93973612580"=>{:zombie=>true, :last_probe_time=>2024-07-07 01:35:53 3365/65536 +0300, :last_probe_score=>1720305353.0513458}}
|
1669
|
+
[3] pry(main)> rql.swarm_status
|
1670
|
+
=> {:auto_swarm=>false,
|
1671
|
+
:supervisor=>{:running=>false, :state=>"non_initialized", :observable=>"non_initialized"},
|
1672
|
+
:probe_hosts=>{:enabled=>true, :thread=>{:running=>false, :state=>"non_initialized"}, :main_loop=>{:running=>false, :state=>"non_initialized"}},
|
1673
|
+
:flush_zombies=>{:enabled=>true, :ractor=>{:running=>false, :state=>"non_initialized"}, :main_loop=>{:running=>false, :state=>"non_initialized"}}}
|
1674
|
+
[4] pry(main)> rql.zombies_info
|
1675
|
+
=> {:zombie_hosts=>
|
1676
|
+
#<Set:
|
1677
|
+
{"rql:hst:17580/2260/2280/3f16b93973612580",
|
1678
|
+
"rql:hst:17580/2300/2280/3f16b93973612580",
|
1679
|
+
"rql:hst:17580/2320/2280/3f16b93973612580",
|
1680
|
+
"rql:hst:17580/2260/2340/3f16b93973612580",
|
1681
|
+
"rql:hst:17580/2300/2340/3f16b93973612580",
|
1682
|
+
"rql:hst:17580/2320/2340/3f16b93973612580",
|
1683
|
+
"rql:hst:17580/2360/2280/3f16b93973612580",
|
1684
|
+
"rql:hst:17580/2360/2340/3f16b93973612580",
|
1685
|
+
"rql:hst:17580/2400/2280/3f16b93973612580",
|
1686
|
+
"rql:hst:17580/2400/2340/3f16b93973612580"}>,
|
1687
|
+
:zombie_acquirers=>#<Set: {"rql:acq:17580/2260/2380/2280/3f16b93973612580"}>,
|
1688
|
+
:zombie_locks=>#<Set: {"rql:lock:kekpek"}>}
|
1689
|
+
[5] pry(main)> rql.zombie_locks
|
1690
|
+
=> #<Set: {"rql:lock:kekpek"}>
|
1691
|
+
[6] pry(main)> rql.zombie_acquiers
|
1692
|
+
=> #<Set: {"rql:acq:17580/2260/2380/2280/3f16b93973612580"}>
|
1693
|
+
[7] pry(main)> rql.zombie_hosts
|
1694
|
+
=> #<Set:
|
1695
|
+
{"rql:hst:17580/2260/2280/3f16b93973612580",
|
1696
|
+
"rql:hst:17580/2300/2280/3f16b93973612580",
|
1697
|
+
"rql:hst:17580/2320/2280/3f16b93973612580",
|
1698
|
+
"rql:hst:17580/2260/2340/3f16b93973612580",
|
1699
|
+
"rql:hst:17580/2300/2340/3f16b93973612580",
|
1700
|
+
"rql:hst:17580/2320/2340/3f16b93973612580",
|
1701
|
+
"rql:hst:17580/2360/2280/3f16b93973612580",
|
1702
|
+
"rql:hst:17580/2360/2340/3f16b93973612580",
|
1703
|
+
"rql:hst:17580/2400/2280/3f16b93973612580",
|
1704
|
+
"rql:hst:17580/2400/2340/3f16b93973612580"}>
|
1705
|
+
```
|
1706
|
+
</details>
|
1707
|
+
|
1708
|
+
<details>
|
1709
|
+
<summary>kill zombies in a background</summary>
|
1710
|
+
|
1711
|
+
- swarmize the new current ruby process that should run the flush zombies element that will drop zombie locks, zombie hosts and their lock requests in a background:
|
1712
|
+
|
1713
|
+
```ruby
|
1714
|
+
[8] pry(main)> rql.swarmize!
|
1715
|
+
/Users/daiver/Projects/redis_queued_locks/lib/redis_queued_locks/swarm/flush_zombies.rb:107: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.
|
1716
|
+
=> {:ok=>true, :result=>:swarming}
|
1717
|
+
[9] pry(main)> rql.swarm_info
|
1718
|
+
=> {"rql:hst:17752/2260/2280/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 4012577/4194304 +0300, :last_probe_score=>1720305399.956673},
|
1719
|
+
"rql:hst:17752/2300/2280/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 4015233/4194304 +0300, :last_probe_score=>1720305399.9573061},
|
1720
|
+
"rql:hst:17752/2320/2280/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 4016755/4194304 +0300, :last_probe_score=>1720305399.957669},
|
1721
|
+
"rql:hst:17752/2260/2340/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 1003611/1048576 +0300, :last_probe_score=>1720305399.957118},
|
1722
|
+
"rql:hst:17752/2300/2340/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 2008027/2097152 +0300, :last_probe_score=>1720305399.957502},
|
1723
|
+
"rql:hst:17752/2320/2340/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 2008715/2097152 +0300, :last_probe_score=>1720305399.95783},
|
1724
|
+
"rql:hst:17752/2360/2280/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 4018063/4194304 +0300, :last_probe_score=>1720305399.9579809},
|
1725
|
+
"rql:hst:17752/2360/2340/89beef198021f16d"=>{:zombie=>false, :last_probe_time=>2024-07-07 01:36:39 1004673/1048576 +0300, :last_probe_score=>1720305399.9581308}}
|
1726
|
+
[10] pry(main)> rql.swarm_status
|
1727
|
+
=> {:auto_swarm=>false,
|
1728
|
+
:supervisor=>{:running=>true, :state=>"sleep", :observable=>"initialized"},
|
1729
|
+
:probe_hosts=>{:enabled=>true, :thread=>{:running=>true, :state=>"sleep"}, :main_loop=>{:running=>true, :state=>"sleep"}},
|
1730
|
+
:flush_zombies=>{:enabled=>true, :ractor=>{:running=>true, :state=>"running"}, :main_loop=>{:running=>true, :state=>"sleep"}}}
|
1731
|
+
[11] pry(main)> rql.zombies_info
|
1732
|
+
=> {:zombie_hosts=>#<Set: {}>, :zombie_acquirers=>#<Set: {}>, :zombie_locks=>#<Set: {}>}
|
1733
|
+
[12] pry(main)> rql.zombie_acquiers
|
1734
|
+
=> #<Set: {}>
|
1735
|
+
[13] pry(main)> rql.zombie_hosts
|
1736
|
+
=> #<Set: {}>
|
1737
|
+
[14] pry(main)>
|
1738
|
+
```
|
1739
|
+
</details>
|
1740
|
+
|
1741
|
+
<details>
|
1742
|
+
<summary>swarm hosts key in Redis</summary>
|
1743
|
+
|
1744
|
+
```ruby
|
1745
|
+
"rql:swarm:hsts"
|
1746
|
+
```
|
1747
|
+
</details>
|
1748
|
+
|
1749
|
+
---
|
1750
|
+
|
1403
1751
|
## Lock Access Strategies
|
1404
1752
|
|
1405
1753
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
@@ -1418,7 +1766,7 @@ rql.current_acquirer_id
|
|
1418
1766
|
|
1419
1767
|
---
|
1420
1768
|
|
1421
|
-
##
|
1769
|
+
## Deadlocks and Reentrant locks
|
1422
1770
|
|
1423
1771
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1424
1772
|
|
@@ -1440,37 +1788,110 @@ rql.current_acquirer_id
|
|
1440
1788
|
|
1441
1789
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1442
1790
|
|
1791
|
+
- [Logging Configuration](#logging-configuration)
|
1792
|
+
|
1793
|
+
|
1443
1794
|
- default logs (raised from `#lock`/`#lock!`):
|
1444
1795
|
|
1445
1796
|
```ruby
|
1446
|
-
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1447
|
-
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1448
|
-
"[redis_queued_locks.dead_score_reached__reset_acquier_position]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1449
|
-
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acq_time");
|
1450
|
-
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "acq_time");
|
1451
|
-
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "acq_time");
|
1452
|
-
"[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1453
|
-
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1454
|
-
"[redis_queued_locks.decrease_lock]" # (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "acs_strat");
|
1797
|
+
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1798
|
+
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1799
|
+
"[redis_queued_locks.dead_score_reached__reset_acquier_position]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1800
|
+
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time");
|
1801
|
+
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "acq_time");
|
1802
|
+
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "acq_time");
|
1803
|
+
"[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1804
|
+
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1805
|
+
"[redis_queued_locks.decrease_lock]" # (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1455
1806
|
```
|
1456
1807
|
|
1457
1808
|
- additional logs (raised from `#lock`/`#lock!` with `confg[:log_lock_try] == true`):
|
1458
1809
|
|
1459
1810
|
```ruby
|
1460
|
-
"[redis_queued_locks.try_lock.start]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1461
|
-
"[redis_queued_locks.try_lock.rconn_fetched]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1462
|
-
"[redis_queued_locks.try_lock.same_process_conflict_detected]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1463
|
-
"[redis_queued_locks.try_lock.same_process_conflict_analyzed]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status");
|
1464
|
-
"[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status", "last_ext_ttl", "last_ext_ts");
|
1465
|
-
"[redis_queued_locks.try_lock.reentrant_lock__work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status", last_spc_ts);
|
1466
|
-
"[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "spc_status", "last_spc_ts");
|
1467
|
-
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1468
|
-
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1469
|
-
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "first_acq_id_in_queue");
|
1470
|
-
"[redis_queued_locks.try_lock.exit__queue_ttl_reached]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1471
|
-
"[redis_queued_locks.try_lock.exit__no_first]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "first_acq_id_in_queue", "<current_lock_data>");
|
1472
|
-
"[redis_queued_locks.try_lock.exit__lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
1473
|
-
"[redis_queued_locks.try_lock.obtain__free_to_acquire]" # (logs "lock_key", "queue_ttl", "acq_id", "acs_strat");
|
1811
|
+
"[redis_queued_locks.try_lock.start]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1812
|
+
"[redis_queued_locks.try_lock.rconn_fetched]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1813
|
+
"[redis_queued_locks.try_lock.same_process_conflict_detected]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1814
|
+
"[redis_queued_locks.try_lock.same_process_conflict_analyzed]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status");
|
1815
|
+
"[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", "last_ext_ttl", "last_ext_ts");
|
1816
|
+
"[redis_queued_locks.try_lock.reentrant_lock__work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", last_spc_ts);
|
1817
|
+
"[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", "last_spc_ts");
|
1818
|
+
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1819
|
+
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1820
|
+
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue");
|
1821
|
+
"[redis_queued_locks.try_lock.exit__queue_ttl_reached]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1822
|
+
"[redis_queued_locks.try_lock.exit__no_first]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue", "<current_lock_data>");
|
1823
|
+
"[redis_queued_locks.try_lock.exit__lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
1824
|
+
"[redis_queued_locks.try_lock.obtain__free_to_acquire]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1825
|
+
```
|
1826
|
+
|
1827
|
+
---
|
1828
|
+
|
1829
|
+
### Logging Configuration
|
1830
|
+
|
1831
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1832
|
+
|
1833
|
+
**NOTICE**: logging can be sampled via:
|
1834
|
+
- `config.log_samplign_enabled = true` (**false** by default);
|
1835
|
+
- `config.log_sampler = RedisQueuedLocks::Logging::Sampler` (used by default);
|
1836
|
+
- see **RedisQueuedLocks::Logging::Sampler** implementation in source code for customization details;
|
1837
|
+
|
1838
|
+
```ruby
|
1839
|
+
# (default: RedisQueuedLocks::Logging::VoidLogger)
|
1840
|
+
# - the logger object;
|
1841
|
+
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
1842
|
+
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
1843
|
+
# - at this moment the only debug logs are realised in following cases:
|
1844
|
+
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1845
|
+
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1846
|
+
# - "[redis_queued_locks.dead_score_reached__reset_acquier_position]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1847
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time", "acs_strat");
|
1848
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time", "acs_strat");
|
1849
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acq_time", "acs_strat");
|
1850
|
+
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1851
|
+
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1852
|
+
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1853
|
+
# - by default uses VoidLogger that does nothing;
|
1854
|
+
config.logger = RedisQueuedLocks::Logging::VoidLogger
|
1855
|
+
|
1856
|
+
# (default: false)
|
1857
|
+
# - adds additional debug logs;
|
1858
|
+
# - enables additional logs for each internal try-retry lock acquiring (a lot of logs can be generated depending on your retry configurations);
|
1859
|
+
# - it adds following debug logs in addition to the existing:
|
1860
|
+
# - "[redis_queued_locks.try_lock.start]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1861
|
+
# - "[redis_queued_locks.try_lock.rconn_fetched]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1862
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_detected]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1863
|
+
# - "[redis_queued_locks.try_lock.same_process_conflict_analyzed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status");
|
1864
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__extend_and_work_through]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", "last_ext_ttl", "last_ext_ts");
|
1865
|
+
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "spc_status", last_spc_ts);
|
1866
|
+
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat")";
|
1867
|
+
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1868
|
+
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue");
|
1869
|
+
# - "[redis_queued_locks.try_lock.exit__queue_ttl_reached]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1870
|
+
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue", "<current_lock_data>");
|
1871
|
+
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
1872
|
+
# - "[redis_queued_locks.try_lock.obtain__free_to_acquire]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat");
|
1873
|
+
config.log_lock_try = false
|
1874
|
+
|
1875
|
+
# (default: false)
|
1876
|
+
# - enables <log sampling>: only the configured percent of RQL cases will be logged;
|
1877
|
+
# - disabled by default;
|
1878
|
+
# - works in tandem with <config.log_sampling_percent> and <log.sampler> configs;
|
1879
|
+
config.log_sampling_enabled = false
|
1880
|
+
|
1881
|
+
# (default: 15)
|
1882
|
+
# - the percent of cases that should be logged;
|
1883
|
+
# - take an effect when <config.log_sampling_enalbed> is true;
|
1884
|
+
# - works in tandem with <config.log_sampling_enabled> and <config.log_sampler> configs;
|
1885
|
+
config.log_sampling_percent = 15
|
1886
|
+
|
1887
|
+
# (default: RedisQueuedLocks::Logging::Sampler)
|
1888
|
+
# - percent-based log sampler that decides should be RQL case logged or not;
|
1889
|
+
# - works in tandem with <config.log_sampling_enabled> and <config.log_sampling_percent> configs;
|
1890
|
+
# - based on the ultra simple percent-based (weight-based) algorithm that uses SecureRandom.rand
|
1891
|
+
# method so the algorithm error is ~(0%..13%);
|
1892
|
+
# - you can provide your own log sampler with bettter algorithm that should realize
|
1893
|
+
# `sampling_happened?(percent) => boolean` interface (see `RedisQueuedLocks::Logging::Sampler` for example);
|
1894
|
+
config.log_sampler = RedisQueuedLocks::Logging::Sampler
|
1474
1895
|
```
|
1475
1896
|
|
1476
1897
|
---
|
@@ -1480,9 +1901,12 @@ rql.current_acquirer_id
|
|
1480
1901
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1481
1902
|
|
1482
1903
|
- [Instrumentation Events](#instrumentation-events)
|
1904
|
+
- [Instrumentation Configuration](#instrumentation-configuration)
|
1483
1905
|
|
1484
1906
|
An instrumentation layer is incapsulated in `instrumenter` object stored in [config](#configuration) (`RedisQueuedLocks::Client#config[:instrumenter]`).
|
1485
1907
|
|
1908
|
+
Instrumentation can be sampled. See [Instrumentation Configuration](#instrumentation-configuration) section for details.
|
1909
|
+
|
1486
1910
|
Instrumenter object should provide `notify(event, payload)` method with the following signarue:
|
1487
1911
|
|
1488
1912
|
- `event` - `string`;
|
@@ -1498,6 +1922,48 @@ By default `RedisQueuedLocks::Client` is configured with the void notifier (whic
|
|
1498
1922
|
|
1499
1923
|
---
|
1500
1924
|
|
1925
|
+
### Instrumentation Configuration
|
1926
|
+
|
1927
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1928
|
+
|
1929
|
+
**NOTICE**: instrumentation can be sampled via:
|
1930
|
+
- `config.instr_sampling_enabled = true` (**false** by default);
|
1931
|
+
- `config.instr_sampler = RedisQueuedLocks::Instrument::Sampler` (used by default);
|
1932
|
+
- see **RedisQueuedLocks::Instrument::Sampler** implementation in source code for customization details;
|
1933
|
+
|
1934
|
+
```ruby
|
1935
|
+
# (default: RedisQueuedLocks::Instrument::VoidNotifier)
|
1936
|
+
# - instrumentation layer;
|
1937
|
+
# - you can provide your own instrumenter that should realize `#notify(event, payload = {})` interface:
|
1938
|
+
# - event: <string> requried;
|
1939
|
+
# - payload: <hash> requried;
|
1940
|
+
# - disabled by default via `VoidNotifier`;
|
1941
|
+
config.instrumenter = RedisQueuedLocks::Instrument::ActiveSupport
|
1942
|
+
|
1943
|
+
# (default: false)
|
1944
|
+
# - enables <instrumentaion sampling>: only the configured percent of RQL cases will be instrumented;
|
1945
|
+
# - disabled by default;
|
1946
|
+
# - works in tandem with <config.instr_sampling_percent and <log.instr_sampler>;
|
1947
|
+
config.instr_sampling_enabled = false
|
1948
|
+
|
1949
|
+
# (default: 15)
|
1950
|
+
# - the percent of cases that should be instrumented;
|
1951
|
+
# - take an effect when <config.instr_sampling_enalbed> is true;
|
1952
|
+
# - works in tandem with <config.instr_sampling_enabled> and <config.instr_sampler> configs;
|
1953
|
+
config.instr_sampling_percent = 15
|
1954
|
+
|
1955
|
+
# (default: RedisQueuedLocks::Instrument::Sampler)
|
1956
|
+
# - percent-based log sampler that decides should be RQL case instrumented or not;
|
1957
|
+
# - works in tandem with <config.instr_sampling_enabled> and <config.instr_sampling_percent> configs;
|
1958
|
+
# - based on the ultra simple percent-based (weight-based) algorithm that uses SecureRandom.rand
|
1959
|
+
# method so the algorithm error is ~(0%..13%);
|
1960
|
+
# - you can provide your own log sampler with bettter algorithm that should realize
|
1961
|
+
# `sampling_happened?(percent) => boolean` interface (see `RedisQueuedLocks::Instrument::Sampler` for example);
|
1962
|
+
config.instr_sampler = RedisQueuedLocks::Instrument::Sampler
|
1963
|
+
```
|
1964
|
+
|
1965
|
+
---
|
1966
|
+
|
1501
1967
|
### Instrumentation Events
|
1502
1968
|
|
1503
1969
|
<sup>\[[back to top](#instrumentation)\]</sup>
|
@@ -1520,6 +1986,7 @@ Detalized event semantics and payload structure:
|
|
1520
1986
|
- payload:
|
1521
1987
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1522
1988
|
- `:acq_id` - `string` - lock acquier identifier;
|
1989
|
+
- `:hst_id` - `string` - lock's host identifier;
|
1523
1990
|
- `:lock_key` - `string` - lock name;
|
1524
1991
|
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend;
|
1525
1992
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
@@ -1532,6 +1999,7 @@ Detalized event semantics and payload structure:
|
|
1532
1999
|
- `:lock_key` - `string` - lock name;
|
1533
2000
|
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1534
2001
|
- `:acq_id` - `string` - lock acquier identifier;
|
2002
|
+
- `:hst_id` - `string` - lock's host identifier;
|
1535
2003
|
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as extendable reentrant lock;
|
1536
2004
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1537
2005
|
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
@@ -1543,6 +2011,7 @@ Detalized event semantics and payload structure:
|
|
1543
2011
|
- `:lock_key` - `string` - lock name;
|
1544
2012
|
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1545
2013
|
- `:acq_id` - `string` - lock acquier identifier;
|
2014
|
+
- `:hst_id` - `string` - lock's host identifier;
|
1546
2015
|
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1547
2016
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
1548
2017
|
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
@@ -1554,6 +2023,7 @@ Detalized event semantics and payload structure:
|
|
1554
2023
|
- `:hold_time` - `float`/`milliseconds` - lock hold time;
|
1555
2024
|
- `:ttl` - `integer`/`milliseconds` - lock ttl;
|
1556
2025
|
- `:acq_id` - `string` - lock acquier identifier;
|
2026
|
+
- `:hst_id` - `string` - lock's host identifier;
|
1557
2027
|
- `:lock_key` - `string` - lock name;
|
1558
2028
|
- `:ts` - `numeric`/`epoch` - the time when lock was obtained;
|
1559
2029
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
@@ -1567,6 +2037,7 @@ Detalized event semantics and payload structure:
|
|
1567
2037
|
- `:hold_time` - `float`/`milliseconds` - lock hold time;
|
1568
2038
|
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
1569
2039
|
- `:acq_id` - `string` - lock acquier identifier;
|
2040
|
+
- `:hst_id` - `string` - lock's host identifier;
|
1570
2041
|
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend as reentrant lock;
|
1571
2042
|
- `:lock_key` - `string` - lock name;
|
1572
2043
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
@@ -1596,6 +2067,12 @@ Detalized event semantics and payload structure:
|
|
1596
2067
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
1597
2068
|
|
1598
2069
|
- **Major**:
|
2070
|
+
- Swarm:
|
2071
|
+
- circuit-breaker for long-living failures of your infrastructure inside the swarm elements and supervisor:
|
2072
|
+
the supervisor will stop (for some period of time or while the some factor will return `true`)
|
2073
|
+
trying to ressurect unexpectedly terminated swarm elements, and will notify about this;
|
2074
|
+
- swarm logs (thread/ractor has some limitations so the initial implementation does not include swarm logging);
|
2075
|
+
- swarm instrumentation (thread/ractor has some limitations so the initial implementation does not include swarm instrumentation);
|
1599
2076
|
- lock request prioritization;
|
1600
2077
|
- **strict redlock algorithm support** (support for many `RedisClient` instances);
|
1601
2078
|
- `#lock_series` - acquire a series of locks:
|