redis_queued_locks 1.21.0 → 1.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/project-overview.md +216 -25
- data/.claude/rules/acquirer.md +6 -5
- data/.claude/rules/arguments.md +2 -2
- data/.claude/rules/changelog.md +27 -0
- data/.claude/rules/git.md +1 -1
- data/.claude/rules/logic.md +10 -12
- data/.claude/rules/swarm.md +3 -4
- data/.claude/rules/tests.md +6 -6
- data/.claude/rules/type-checking.md +13 -14
- data/.claude/rules/visitors.md +5 -5
- data/.claude/skills/rql-code-review/SKILL.md +112 -35
- data/CHANGELOG.md +98 -244
- data/CLAUDE.md +9 -8
- data/README.md +275 -85
- data/Rakefile +11 -0
- data/lib/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rb +28 -12
- data/lib/redis_queued_locks/acquirer/acquire_lock/log_visitor.rb +38 -16
- data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rb +92 -15
- data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +430 -63
- data/lib/redis_queued_locks/acquirer/acquire_lock.rb +72 -22
- data/lib/redis_queued_locks/acquirer/clear_dead_requests.rb +240 -45
- data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +3 -1
- data/lib/redis_queued_locks/acquirer/keys.rb +6 -2
- data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +10 -5
- data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +18 -9
- data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +52 -23
- data/lib/redis_queued_locks/acquirer/locks.rb +6 -2
- data/lib/redis_queued_locks/acquirer/queue_info.rb +30 -7
- data/lib/redis_queued_locks/acquirer/queues.rb +17 -6
- data/lib/redis_queued_locks/acquirer/release_all_locks.rb +6 -2
- data/lib/redis_queued_locks/acquirer/release_locks_of.rb +6 -2
- data/lib/redis_queued_locks/client.rb +66 -24
- data/lib/redis_queued_locks/config.rb +93 -1
- data/lib/redis_queued_locks/resource.rb +124 -9
- data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +6 -5
- data/lib/redis_queued_locks/swarm/zombie_info.rb +6 -2
- data/lib/redis_queued_locks/version.rb +2 -2
- data/rbs_collection.lock.yaml +4 -0
- data/rbs_collection.yaml +2 -1
- data/sig/manifest.yml +1 -0
- data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue/log_visitor.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rbs +12 -6
- data/sig/redis_queued_locks/acquirer/acquire_lock/log_visitor.rbs +21 -14
- data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rbs +61 -40
- data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +59 -11
- data/sig/redis_queued_locks/acquirer/acquire_lock/{yield_with_expire → yield_expire}/log_visitor.rbs +4 -4
- data/sig/redis_queued_locks/acquirer/acquire_lock/yield_expire.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock.rbs +5 -3
- data/sig/redis_queued_locks/acquirer/clear_dead_requests.rbs +14 -1
- data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/queues.rbs +1 -1
- data/sig/redis_queued_locks/client.rbs +10 -7
- data/sig/redis_queued_locks/config.rbs +54 -1
- data/sig/redis_queued_locks/instrument/sampler.rbs +3 -1
- data/sig/redis_queued_locks/instrument/void_notifier.rbs +3 -1
- data/sig/redis_queued_locks/logging/sampler.rbs +3 -1
- data/sig/redis_queued_locks/resource.rbs +22 -12
- data/sig/redis_queued_locks/swarm/swarm_element/threaded.rbs +1 -1
- data/sig/redis_queued_locks/utilities.rbs +2 -1
- data/sig/redis_queued_locks.rbs +7 -0
- metadata +4 -3
- /data/sig/redis_queued_locks/{acquier.rbs → acquirer.rbs} +0 -0
data/README.md
CHANGED
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
|
|
5
5
|
**RedisQueuedLocks** (aka **RQL**, aka **RQL-lock**) - <a href="https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/">distributed locks</a> with "prioritized lock acquisition queue" capabilities based on the Redis Database.
|
|
6
6
|
|
|
7
|
-
Each lock request is put into the request queue (each lock is hosted by its own queue separately from other queues) and processed in order of their priority (FIFO
|
|
7
|
+
Each lock request is put into the request queue (each lock is hosted by its own queue separately from other queues) and processed in order of their priority: requests of a higher [priority](#lock-request-prioritization) go first, requests of the same priority are processed in FIFO order. Each lock request lives some period of time (RTTL) (with requeue capabilities) which guarantees the request queue will never be stacked.
|
|
8
8
|
|
|
9
9
|
In addition to the classic `queued` (FIFO) strategy RQL supports `random` (RANDOM) lock obtaining strategy when any acquirer from the lock queue can obtain the lock regardless the position in the queue.
|
|
10
10
|
|
|
11
|
-
Supports `Read`/`Write` locks and `Reentrant` locks.
|
|
11
|
+
Supports [lock request prioritization](#lock-request-prioritization), `Read`/`Write` locks and `Reentrant` locks.
|
|
12
12
|
|
|
13
13
|
Provides flexible invocation flow, parametrized limits (lock request ttl, lock ttl, queue ttl, lock attempts limit, fast failing, etc), **zombie locks elimination**, **[lock indexing](#lock-index)**, logging and instrumentation (and much more).
|
|
14
14
|
|
|
@@ -71,6 +71,7 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
|
|
|
71
71
|
- [Lock Access Strategies](#lock-access-strategies)
|
|
72
72
|
- [queued](#lock-access-strategies)
|
|
73
73
|
- [random](#lock-access-strategies)
|
|
74
|
+
- [Lock Request Prioritization](#lock-request-prioritization)
|
|
74
75
|
- [Read/Write Locks](#readwrite-locks)
|
|
75
76
|
- [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks)
|
|
76
77
|
- [Logging](#logging)
|
|
@@ -125,7 +126,7 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
|
|
|
125
126
|
|
|
126
127
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
|
127
128
|
|
|
128
|
-
> Each lock request is put into the request queue (each lock is hosted by its own queue separately from other queues) and processed in order of their priority (FIFO
|
|
129
|
+
> Each lock request is put into the request queue (each lock is hosted by its own queue separately from other queues) and processed in order of their priority: requests of a higher [priority](#lock-request-prioritization) go first, requests of the same priority are processed in FIFO order. Each lock request lives some period of time (RTTL) which guarantees that the request queue will never be stacked.
|
|
129
130
|
|
|
130
131
|
> In addition to the classic "queued" (FIFO) strategy RQL supports "random" (RANDOM) lock obtaining strategy when any acquirer from the lock queue can obtain the lock regardless the position in the queue.
|
|
131
132
|
|
|
@@ -205,7 +206,8 @@ client = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
|
205
206
|
config['default_lock_ttl'] = 5_000
|
|
206
207
|
|
|
207
208
|
# (seconds) (default: 15)
|
|
208
|
-
# - lock request timeout. after this timeout your lock request in queue will be requeued with new position (at the end of the
|
|
209
|
+
# - lock request timeout. after this timeout your lock request in queue will be requeued with new position (at the end of the
|
|
210
|
+
# lock requests of its priority);
|
|
209
211
|
# - (the enabled lock index) the lock request queue lives as long as its latest lock request (the queue TTL is refreshed with
|
|
210
212
|
# the queue ttl on each lock request): queues of dead acquirers expire by themselves;
|
|
211
213
|
# - should be a positive Integer
|
|
@@ -249,6 +251,30 @@ client = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
|
249
251
|
# retries, timeouts and so on). if lock is free to obtain - it will be obtained;
|
|
250
252
|
config['default_access_strategy'] = :queued
|
|
251
253
|
|
|
254
|
+
# (hash) (default: { lowest: -100, low: -50, normal: 0, high: 50, highest: 100 })
|
|
255
|
+
# - Lock request priority names: a priority name (Symbol) => its priority (Integer from `-100` to `100`);
|
|
256
|
+
# - The names can be used instead of the Integer priorities in the `:priority` attribute and in
|
|
257
|
+
# `config['default_lock_priority']`: `rql.lock('name', priority: :high) { ... }`;
|
|
258
|
+
# - Define the names of your application (keep the default names with `merge`):
|
|
259
|
+
# `config['lock_priorities'] = config['lock_priorities'].merge(payment: 90, email: -10)`;
|
|
260
|
+
# - The default value is frozen (it is shared by all clients): assign a new hash, don't change it in place;
|
|
261
|
+
# - See [Lock Request Prioritization](#lock-request-prioritization) documentation section for details;
|
|
262
|
+
config['lock_priorities'] = { lowest: -100, low: -50, normal: 0, high: 50, highest: 100 }
|
|
263
|
+
|
|
264
|
+
# (integer/symbol) (default: 0)
|
|
265
|
+
# - The default lock request priority: an Integer from `-100` (the lowest) to `100` (the highest)
|
|
266
|
+
# or a priority name of `config['lock_priorities']`;
|
|
267
|
+
# - The priority name should be a key of `config['lock_priorities']` (`RedisQueuedLocks::ConfigValidationError` otherwise):
|
|
268
|
+
# it is checked once after the configuration block (`Client.new { ... }`, `#configure { ... }`: the keys can be assigned
|
|
269
|
+
# in any order; an inconsistent pair is never applied: both keys keep their previous values, also when the block fails)
|
|
270
|
+
# and on a direct `config[...] =` assignment (before the value is applied: change the priority names and the default
|
|
271
|
+
# name together in a configuration block);
|
|
272
|
+
# - Can be customized in methods `#lock`, `#lock!`, `#lock_series` and `#lock_series!` via `:priority` attribute;
|
|
273
|
+
# - `:queued` access strategy: the lock request is placed in the lock request queue after all requests of the same
|
|
274
|
+
# or higher priority and before all requests of the lower priority (FIFO inside the same priority);
|
|
275
|
+
# - See [Lock Request Prioritization](#lock-request-prioritization) documentation section for details;
|
|
276
|
+
config['default_lock_priority'] = 0
|
|
277
|
+
|
|
252
278
|
# (symbol) (default: :wait_for_lock)
|
|
253
279
|
# - Global default conflict strategy mode;
|
|
254
280
|
# - Can be customized in methods `#lock` and `#lock!` via `:conflict_strategy` attribute (see method signatures of #lock and #lock! methods);
|
|
@@ -339,19 +365,19 @@ client = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
|
339
365
|
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
|
340
366
|
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
|
341
367
|
# - at this moment the only debug logs are realised in following cases:
|
|
342
|
-
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
343
|
-
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
344
|
-
# - "[redis_queued_locks.dead_score_reached__reset_acquirer_position]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
345
|
-
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
346
|
-
# - "[redis_queued_locks.lock_obtaining_failed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time", "result");
|
|
347
|
-
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
348
|
-
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
368
|
+
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
369
|
+
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
370
|
+
# - "[redis_queued_locks.dead_score_reached__reset_acquirer_position]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
371
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
372
|
+
# - "[redis_queued_locks.lock_obtaining_failed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result");
|
|
373
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
374
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
349
375
|
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
350
376
|
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
351
377
|
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
352
|
-
# - "[redis_queued_locks.start_lock_series_obtaining]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
353
|
-
# - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
354
|
-
# - "[redis_queued_locks.expire_lock_series]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
378
|
+
# - "[redis_queued_locks.start_lock_series_obtaining]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
379
|
+
# - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
380
|
+
# - "[redis_queued_locks.expire_lock_series]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
355
381
|
# - "[redis_queued_locks.explicit_lock_release]" (logs "lock_key", "lock_key_queue", "rel_time", "at");
|
|
356
382
|
# - "[redis_queued_locks.explicit_read_lock_release]" (logs "lock_key", "acq_id", "lock_res", "rel_time", "at");
|
|
357
383
|
# - "[redis_queued_locks.explicit_all_locks_release]" (logs "at", "rel_time", "rel_key_cnt");
|
|
@@ -376,15 +402,16 @@ client = RedisQueuedLocks::Client.new(redis_client) do |config|
|
|
|
376
402
|
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
377
403
|
# - "[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
378
404
|
# - "[redis_queued_locks.try_lock.single_process_lock_conflict__lock_upgrade]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
379
|
-
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
405
|
+
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
380
406
|
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
381
|
-
# - "[redis_queued_locks.try_lock.
|
|
407
|
+
# - "[redis_queued_locks.try_lock.dead_request_removed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "dead_acq_id", "dead_priority");
|
|
408
|
+
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue");
|
|
382
409
|
# - "[redis_queued_locks.try_lock.exit__queue_ttl_reached]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
383
|
-
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "<current_lock_data>");
|
|
384
|
-
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
|
385
|
-
# - "[redis_queued_locks.try_lock.exit__write_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id");
|
|
386
|
-
# - "[redis_queued_locks.try_lock.exit__read_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id");
|
|
387
|
-
# - "[redis_queued_locks.try_lock.exit__read_lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue");
|
|
410
|
+
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue", "<current_lock_data>");
|
|
411
|
+
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
|
412
|
+
# - "[redis_queued_locks.try_lock.exit__write_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id", "ahead_priority");
|
|
413
|
+
# - "[redis_queued_locks.try_lock.exit__read_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id", "ahead_priority");
|
|
414
|
+
# - "[redis_queued_locks.try_lock.exit__read_lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue");
|
|
388
415
|
# - "[redis_queued_locks.try_lock.obtain__free_to_acquire]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
389
416
|
# - "rw_mode" is the requested lock mode ('write'/'read');
|
|
390
417
|
config['log_lock_try'] = false
|
|
@@ -503,6 +530,7 @@ def lock(
|
|
|
503
530
|
conflict_strategy: config['default_conflict_strategy'],
|
|
504
531
|
read_write_mode: :write,
|
|
505
532
|
access_strategy: config['default_access_strategy'],
|
|
533
|
+
priority: config['default_lock_priority'],
|
|
506
534
|
identity: uniq_identity, # (attr_accessor) calculated during client instantiation via config['uniq_identifier'] proc;
|
|
507
535
|
meta: nil,
|
|
508
536
|
detailed_acq_timeout_error: config['detailed_acq_timeout_error'],
|
|
@@ -563,7 +591,7 @@ def lock(
|
|
|
563
591
|
- `instrumenter` - (optional) `[#notify]`
|
|
564
592
|
- See RedisQueuedLocks::Instrument::ActiveSupport for example;
|
|
565
593
|
- See [Instrumentation](#instrumentation) section of docs;
|
|
566
|
-
- pre-configured in `config['
|
|
594
|
+
- pre-configured in `config['instrumenter']` with void notifier (`RedisQueuedLocks::Instrument::VoidNotifier`);
|
|
567
595
|
- `instrument` - (optional) `[NilClass,Any]`
|
|
568
596
|
- Custom instrumentation data wich will be passed to the instrumenter's payload with :instrument key;
|
|
569
597
|
- `nil` by default (means "no custom instrumentation data");
|
|
@@ -585,6 +613,19 @@ def lock(
|
|
|
585
613
|
if lock is free to obtain - it will be obtained;
|
|
586
614
|
- pre-configured in `config['default_access_strategy']`;
|
|
587
615
|
- See [Lock Access Strategies](#lock-access-strategies) documentation section for details;
|
|
616
|
+
- `priority` - (optional) - `[Integer,Symbol]`
|
|
617
|
+
- The lock request priority: an Integer from `-100` (the lowest) to `100` (the highest), `0` by default,
|
|
618
|
+
or a priority name of `config['lock_priorities']` (`:lowest`, `:low`, `:normal`, `:high`, `:highest` by default,
|
|
619
|
+
the names of your application can be configured);
|
|
620
|
+
- `:queued` access strategy: the lock request is placed in the lock request queue after all requests of the same or
|
|
621
|
+
higher priority and before all requests of the lower priority (strict priority, FIFO inside the same priority;
|
|
622
|
+
read and write lock requests too);
|
|
623
|
+
- `:random` access strategy: the lock request does not wait for its turn (it ignores the order of the lock request queues
|
|
624
|
+
and the priorities of other requests), but it is queued at the position of its priority like a `:queued` one: `:queued`
|
|
625
|
+
lock requests wait for it;
|
|
626
|
+
- the obtained lock is never taken away: the priority orders the waiting lock requests only;
|
|
627
|
+
- pre-configured in `config['default_lock_priority']`;
|
|
628
|
+
- See [Lock Request Prioritization](#lock-request-prioritization) documentation section for details;
|
|
588
629
|
- `conflict_strategy` - (optional) - `[Symbol]`
|
|
589
630
|
- The conflict strategy mode for cases when the process that obtained the lock
|
|
590
631
|
want to acquire this lock again;
|
|
@@ -834,7 +875,7 @@ rql.lock_info("my_lock")
|
|
|
834
875
|
|
|
835
876
|
- (`:queue_ttl`) setting a short limit of time to the lock request queue position (if a process fails to acquire
|
|
836
877
|
the lock within this period of time (and before timeout/retry_count limits occurs of course) -
|
|
837
|
-
its lock request will be moved to the end of queue):
|
|
878
|
+
its lock request will be moved to the end of queue: the end of the lock requests of its priority):
|
|
838
879
|
|
|
839
880
|
```ruby
|
|
840
881
|
rql.lock("my_lock", queue_ttl: 5, timeout: 10_000, retry_count: nil)
|
|
@@ -946,6 +987,7 @@ def lock!(
|
|
|
946
987
|
instrument: nil,
|
|
947
988
|
instrumenter: config['instrumenter'],
|
|
948
989
|
access_strategy: config['default_access_strategy'],
|
|
990
|
+
priority: config['default_lock_priority'],
|
|
949
991
|
conflict_strategy: config['default_conflict_strategy'],
|
|
950
992
|
read_write_mode: :write,
|
|
951
993
|
log_sampling_enabled: config['log_sampling_enabled'],
|
|
@@ -994,6 +1036,7 @@ applied with the same value to the each lock acquirement process (for each lock
|
|
|
994
1036
|
- reentrant locks (already obtained by your outer logic, `conflict_strategy: :work_through`/`:extendable_work_through`) are kept;
|
|
995
1037
|
- a lock that is expired during your block and obtained by another acquirer is kept;
|
|
996
1038
|
- `read_write_mode:` is the same for all locks of the series (see [Read/Write Locks](#readwrite-locks));
|
|
1039
|
+
- `priority:` is the same for all locks of the series (see [Lock Request Prioritization](#lock-request-prioritization));
|
|
997
1040
|
- if you did not pass a block - all locks will be released with their forumla-based TTL described above;
|
|
998
1041
|
- **HOW WILL THIS CASE WORK IN FUTURE RELESAE**: all locks will have the same TTL (`5 seconds` for our example, not `15/10/5 seconds`);
|
|
999
1042
|
|
|
@@ -1074,9 +1117,9 @@ client.lock_seires('c', 'a', 'b') # ... `a` is still obtained
|
|
|
1074
1117
|
|
|
1075
1118
|
```shell
|
|
1076
1119
|
# NEW logs (examples)
|
|
1077
|
-
[redis_queued_locks.start_lock_series_obtaining] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write'
|
|
1078
|
-
[redis_queued_locks.lock_series_obtained] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' acq_time => 2.55 (ms)
|
|
1079
|
-
[redis_queued_locks.expire_lock_series] lock_keys => '["rql:lock:x", "rql:lock:y", "rql:lock:z"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write'
|
|
1120
|
+
[redis_queued_locks.start_lock_series_obtaining] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0
|
|
1121
|
+
[redis_queued_locks.lock_series_obtained] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0 acq_time => 2.55 (ms)
|
|
1122
|
+
[redis_queued_locks.expire_lock_series] lock_keys => '["rql:lock:x", "rql:lock:y", "rql:lock:z"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0
|
|
1080
1123
|
```
|
|
1081
1124
|
|
|
1082
1125
|
----
|
|
@@ -1256,15 +1299,18 @@ lock acquirers and their position in queue. The queue of write lock requests is
|
|
|
1256
1299
|
- get the lock queue information;
|
|
1257
1300
|
- queue represents the ordered set of lock key reqests:
|
|
1258
1301
|
- set is ordered by score in ASC manner (inside the Redis Set);
|
|
1259
|
-
- score is represented as a timestamp when the lock request was made
|
|
1302
|
+
- score is represented as a timestamp when the lock request was made shifted by the
|
|
1303
|
+
[priority](#lock-request-prioritization) band (`-priority * 2**32` seconds: the default priority `0` keeps the timestamp);
|
|
1260
1304
|
- represents the acquirer identifier and their score as an array of hashes;
|
|
1261
1305
|
- returns `nil` if lock queues (write and read) do not exist;
|
|
1262
1306
|
- lock queue data (`Hash<String,String|Array<Hash<String|Numeric>>`):
|
|
1263
1307
|
- `"lock_queue"` - `string` - lock queue key in redis (write lock requests);
|
|
1264
1308
|
- `"queue"` - `array` - an array of lock requests (array of hashes):
|
|
1265
1309
|
- `"acq_id"` - `string` - acquirer identifier (process_id/thread_id/fiber_id/ractor_id/identity by default);
|
|
1266
|
-
- `"score"` - `float`/`epoch` - time when the lock request was made (epoch)
|
|
1310
|
+
- `"score"` - `float`/`epoch` - the position of the lock request: time when the lock request was made (epoch) shifted by
|
|
1311
|
+
`-priority * 2**32` seconds (the time itself for the default priority `0`);
|
|
1267
1312
|
- `"rw_mode"` - `string` - the requested lock mode (`"write"`);
|
|
1313
|
+
- `"priority"` - `integer` - the lock request priority (from `-100` to `100`, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
1268
1314
|
- `"read_lock_queue"`, `"read_queue"` - (only when read lock requests exist) the read lock queue key and read lock requests
|
|
1269
1315
|
with the same structure (`"rw_mode"` - `"read"`), see [Read/Write Locks](#readwrite-locks);
|
|
1270
1316
|
|
|
@@ -1275,15 +1321,16 @@ rql.queue_info("your_lock_name")
|
|
|
1275
1321
|
{
|
|
1276
1322
|
"lock_queue" => "rql:lock_queue:your_lock_name",
|
|
1277
1323
|
"queue" => [
|
|
1278
|
-
|
|
1279
|
-
{ "acq_id" => "rql:acq:123/567/
|
|
1280
|
-
{ "acq_id" => "rql:acq:
|
|
1324
|
+
# (a lock request of the high priority: ahead of the requests of the default priority)
|
|
1325
|
+
{ "acq_id" => "rql:acq:123/456/567/678/fa76df9cc2", "score" => -2583360655.459158, "rw_mode" => "write", "priority" => 1 },
|
|
1326
|
+
{ "acq_id" => "rql:acq:123/567/456/679/c7bfcaf4f9", "score" => 1711606640.540906, "rw_mode" => "write", "priority" => 0 },
|
|
1327
|
+
{ "acq_id" => "rql:acq:555/329/523/127/7329553b11", "score" => 1711606640.540963, "rw_mode" => "write", "priority" => 0 },
|
|
1281
1328
|
# ...etc
|
|
1282
1329
|
],
|
|
1283
1330
|
# (when read lock requests exist)
|
|
1284
1331
|
"read_lock_queue" => "rql:lock_read_queue:your_lock_name",
|
|
1285
1332
|
"read_queue" => [
|
|
1286
|
-
{ "acq_id" => "rql:acq:123/456/567/700/fa76df9cc2", "score" => 1711606640.541011, "rw_mode" => "read" },
|
|
1333
|
+
{ "acq_id" => "rql:acq:123/456/567/700/fa76df9cc2", "score" => 1711606640.541011, "rw_mode" => "read", "priority" => 0 },
|
|
1287
1334
|
# ...etc
|
|
1288
1335
|
]
|
|
1289
1336
|
}
|
|
@@ -1471,7 +1518,7 @@ rql.unlock_read('report', acquirer_id: reader['acq_id'])
|
|
|
1471
1518
|
- pre-configured value in `config['logger']`;
|
|
1472
1519
|
- `:instrumenter` - (optional) `[#notify]`
|
|
1473
1520
|
- custom instrumenter object;
|
|
1474
|
-
- pre-configured value in `config['
|
|
1521
|
+
- pre-configured value in `config['instrumenter']`;
|
|
1475
1522
|
- `:instrument` - (optional) `[NilClass,Any]`
|
|
1476
1523
|
- custom instrumentation data wich will be passed to the instrumenter's payload with `:instrument` key;
|
|
1477
1524
|
- `:log_sampling_enabled` - (optional) `[Boolean]`
|
|
@@ -1554,7 +1601,7 @@ rql.clear_locks
|
|
|
1554
1601
|
- pre-configured value in `config['logger']`;
|
|
1555
1602
|
- `:instrumenter` - (optional) `[#notify]`
|
|
1556
1603
|
- custom instrumenter object;
|
|
1557
|
-
- pre-configured value in `config['
|
|
1604
|
+
- pre-configured value in `config['instrumenter']`;
|
|
1558
1605
|
- `:instrument` - (optional) `[NilClass,Any]`
|
|
1559
1606
|
- custom instrumentation data wich will be passed to the instrumenter's payload with `:instrument` key;
|
|
1560
1607
|
- `:log_sampling_enabled` - (optional) `[Boolean]`
|
|
@@ -1662,7 +1709,7 @@ rql.clear_current_locks
|
|
|
1662
1709
|
- pre-configured value in `config['logger']`;
|
|
1663
1710
|
- `:instrumenter` - (optional) `[#notify]`
|
|
1664
1711
|
- custom instrumenter object;
|
|
1665
|
-
- pre-configured value in `config['
|
|
1712
|
+
- pre-configured value in `config['instrumenter']`;
|
|
1666
1713
|
- `:instrument` - (optional) `[NilClass,Any]`
|
|
1667
1714
|
- custom instrumentation data wich will be passed to the instrumenter's payload with `:instrument` key;
|
|
1668
1715
|
- `:log_sampling_enabled` - (optional) `[Boolean]`
|
|
@@ -1972,8 +2019,8 @@ rql.locks_info # or rql.locks_info(scan_size: 123)
|
|
|
1972
2019
|
- returns `Set<Hash<Symbol,Any>>` (see [#queue_info](#queue_info) and examples below for details).
|
|
1973
2020
|
- contained data: `{ queue: String, requests: Array<Hash<String,Any>> }`
|
|
1974
2021
|
- `:queue` - `String` - lock key queue in Redis;
|
|
1975
|
-
- `:requests` - `Array<Hash<String,Any>>` - lock requests in the que with their acquirer id, score
|
|
1976
|
-
(`"rw_mode"` - `"write"` for `rql:lock_queue:*`, `"read"` for read lock queues `rql:lock_read_queue:*`).
|
|
2022
|
+
- `:requests` - `Array<Hash<String,Any>>` - lock requests in the que with their acquirer id, score, requested lock mode
|
|
2023
|
+
(`"rw_mode"` - `"write"` for `rql:lock_queue:*`, `"read"` for read lock queues `rql:lock_read_queue:*`) and priority.
|
|
1977
2024
|
|
|
1978
2025
|
```ruby
|
|
1979
2026
|
rql.queues_info # or rql.qeuues_info(scan_size: 123)
|
|
@@ -1981,13 +2028,13 @@ rql.queues_info # or rql.qeuues_info(scan_size: 123)
|
|
|
1981
2028
|
=> #<Set:
|
|
1982
2029
|
{{:queue=>"rql:lock_queue:some-lock-123",
|
|
1983
2030
|
:requests=>
|
|
1984
|
-
[{"acq_id"=>"rql:acq:38529/4500/4520/4360/66093702f24a3129", "score"=>1711606640.540842, "rw_mode"=>"write"},
|
|
1985
|
-
{"acq_id"=>"rql:acq:38529/4580/4600/4360/66093702f24a3129", "score"=>1711606640.540906, "rw_mode"=>"write"},
|
|
1986
|
-
{"acq_id"=>"rql:acq:38529/4620/4640/4360/66093702f24a3129", "score"=>1711606640.5409632, "rw_mode"=>"write"}]},
|
|
2031
|
+
[{"acq_id"=>"rql:acq:38529/4500/4520/4360/66093702f24a3129", "score"=>1711606640.540842, "rw_mode"=>"write", "priority"=>0},
|
|
2032
|
+
{"acq_id"=>"rql:acq:38529/4580/4600/4360/66093702f24a3129", "score"=>1711606640.540906, "rw_mode"=>"write", "priority"=>0},
|
|
2033
|
+
{"acq_id"=>"rql:acq:38529/4620/4640/4360/66093702f24a3129", "score"=>1711606640.5409632, "rw_mode"=>"write", "priority"=>0}]},
|
|
1987
2034
|
{:queue=>"rql:lock_read_queue:some-lock-456",
|
|
1988
2035
|
:requests=>
|
|
1989
|
-
[{"acq_id"=>"rql:acq:38529/4380/4400/4360/66093702f24a3129", "score"=>1711606640.540722, "rw_mode"=>"read"},
|
|
1990
|
-
{"acq_id"=>"rql:acq:38529/4420/4440/4360/66093702f24a3129", "score"=>1711606640.5407748, "rw_mode"=>"read"}]},
|
|
2036
|
+
[{"acq_id"=>"rql:acq:38529/4380/4400/4360/66093702f24a3129", "score"=>1711606640.540722, "rw_mode"=>"read", "priority"=>0},
|
|
2037
|
+
{"acq_id"=>"rql:acq:38529/4420/4440/4360/66093702f24a3129", "score"=>1711606640.5407748, "rw_mode"=>"read", "priority"=>0}]},
|
|
1991
2038
|
...}>
|
|
1992
2039
|
```
|
|
1993
2040
|
---
|
|
@@ -1999,8 +2046,10 @@ rql.queues_info # or rql.qeuues_info(scan_size: 123)
|
|
|
1999
2046
|
In some cases your lock requests may become "dead". It means that your lock request lives in lock queue in Redis without
|
|
2000
2047
|
any processing. It can happen when your processs that are enqueeud to the lock queue is failed unexpectedly (for some reason)
|
|
2001
2048
|
before the lock acquire moment occurs and when no any other process does not need this lock anymore.
|
|
2002
|
-
For this case your lock
|
|
2003
|
-
|
|
2049
|
+
For this case your lock request will be cleared only when another lock request meets it: a lock request of the same
|
|
2050
|
+
[priority](#lock-request-prioritization) (lock acquirement drops the expired requests of its priority) or a `:queued` lock request
|
|
2051
|
+
it blocks (the head of the queue, an earlier request of the other read/write mode); the dead requests of other priorities stay
|
|
2052
|
+
in the queue.
|
|
2004
2053
|
|
|
2005
2054
|
NOTE: with the enabled [Lock Index](#lock-index) the lock request queue lives as long as its latest lock request (the queue TTL
|
|
2006
2055
|
is refreshed with the request `queue_ttl` on each lock request), so the queue of dead requests expires by itself (queues of older
|
|
@@ -2009,7 +2058,12 @@ RQL versions and queues created with the disabled lock index have no TTL: see [#
|
|
|
2009
2058
|
In order to help with these dead requests you may periodically call `#clear_dead_requests`
|
|
2010
2059
|
with corresponding `:dead_ttl` option, that is pre-configured by default via `config['dead_request_ttl']`.
|
|
2011
2060
|
Lock queues are found via the redis `SCAN` (one pass) or via the [Lock Index](#lock-index) (one `ZSCAN` pass, the keyspace is
|
|
2012
|
-
never scanned: the enabled lock index).
|
|
2061
|
+
never scanned: the enabled lock index). Each batch of lock queues takes two round trips: the lock requests are read, then the dead
|
|
2062
|
+
requests of each [priority](#lock-request-prioritization) found in the queues are dropped (no second round trip without dead requests).
|
|
2063
|
+
With `by_priority: true` the lock requests are not read: only the first lock request of each priority is read (the dead requests
|
|
2064
|
+
of a priority are found by its first lock request): a batch takes one round trip per priority of its lock queue with the most
|
|
2065
|
+
priorities + 1 (the dead requests of a priority are dropped in the round trip that reads the next priority), so the transfer does not
|
|
2066
|
+
depend on the length of the lock queues: use it for long lock queues with few priorities.
|
|
2013
2067
|
|
|
2014
2068
|
`:dead_ttl` option is required because of it is no any **fast** and **resource-free** way to understand which request
|
|
2015
2069
|
is dead now and is it really dead cuz each request queue can host their requests with
|
|
@@ -2022,14 +2076,20 @@ Accepts:
|
|
|
2022
2076
|
- `:scan_size` - (optional) `[Integer]`
|
|
2023
2077
|
- the batch of scanned keys for Redis'es SCAN command;
|
|
2024
2078
|
- (the enabled lock index) the `ZSCAN` count of the [Lock Index](#lock-index) pass and the number of locks (both lock queues
|
|
2025
|
-
of each lock) processed in one
|
|
2079
|
+
of each lock) processed in one batch;
|
|
2026
2080
|
- has a preconfigured valie in `config['lock_release_batch_size']`;
|
|
2081
|
+
- `:by_priority` - (optional) `[Boolean]`
|
|
2082
|
+
- read only the first lock request of each [priority](#lock-request-prioritization) in the lock queues instead of all lock
|
|
2083
|
+
requests: a batch of lock queues takes one round trip per priority of its lock queue with the most priorities + 1 (instead
|
|
2084
|
+
of two round trips), the transfer does not depend on the length of the lock queues (for long lock queues with few
|
|
2085
|
+
priorities);
|
|
2086
|
+
- `false` by default (all lock requests are read);
|
|
2027
2087
|
- `:logger` - (optional) `[::Logger,#debug]`
|
|
2028
2088
|
- custom logger object;
|
|
2029
2089
|
- pre-configured in `config['logger']`;
|
|
2030
2090
|
- `:instrumenter` - (optional) `[#notify]`
|
|
2031
2091
|
- custom instrumenter object;
|
|
2032
|
-
- pre-configured in `config['
|
|
2092
|
+
- pre-configured in `config['instrumenter']`;
|
|
2033
2093
|
- `:instrument` - (optional) `[NilClass,Any]`
|
|
2034
2094
|
- custom instrumentation data wich will be passed to the instrumenter's payload with :instrument key;
|
|
2035
2095
|
- `nil` by default (no additional data);
|
|
@@ -2079,6 +2139,9 @@ rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000) # 1 hour in milliseconds
|
|
|
2079
2139
|
}>
|
|
2080
2140
|
}
|
|
2081
2141
|
}
|
|
2142
|
+
|
|
2143
|
+
# long lock queues with few priorities: only the first lock request of each priority is read
|
|
2144
|
+
rql.clear_dead_requests(dead_ttl: 60 * 60 * 1000, by_priority: true)
|
|
2082
2145
|
```
|
|
2083
2146
|
|
|
2084
2147
|
---
|
|
@@ -2598,13 +2661,115 @@ rql.possible_host_ids
|
|
|
2598
2661
|
- can be customized in methods `#lock` and `#lock!` via `:access_strategy` attribute (see method signatures of #lock and #lock! methods);
|
|
2599
2662
|
- supports different strategies:
|
|
2600
2663
|
- `:queued` (FIFO): the classic queued behavior (default), your lock will be obitaned if you are first in queue and the required lock is free;
|
|
2664
|
+
the queue is ordered by the [lock request priorities](#lock-request-prioritization) (FIFO inside the same priority);
|
|
2601
2665
|
- `:random` (RANDOM): obtain a lock without checking the positions in the queue (but with checking the limist, retries, timeouts and so on). if lock is free to obtain - it will be obtained;
|
|
2666
|
+
(the priorities of other lock requests are ignored too; the lock request itself is queued at the position of its
|
|
2667
|
+
[priority](#lock-request-prioritization) like a `:queued` one: `:queued` lock requests wait for it);
|
|
2602
2668
|
- for current implementation detalis check:
|
|
2603
2669
|
- [Configuration](#configuration) documentation: see `config['default_access_strategy']` config docs;
|
|
2604
2670
|
- [#lock](#lock---obtain-a-lock) method documentation: see `access_strategy` attribute docs;
|
|
2605
2671
|
|
|
2606
2672
|
---
|
|
2607
2673
|
|
|
2674
|
+
## Lock Request Prioritization
|
|
2675
|
+
|
|
2676
|
+
<sup>\[[back to top](#table-of-contents)\]</sup>
|
|
2677
|
+
|
|
2678
|
+
- `priority:` attribute of `#lock`, `#lock!`, `#lock_series` and `#lock_series!` (`config['default_lock_priority']` by default):
|
|
2679
|
+
an Integer from `-100` (the lowest priority) to `100` (the highest priority), `0` - (**default**) normal priority,
|
|
2680
|
+
or a priority name (Symbol) of `config['lock_priorities']`;
|
|
2681
|
+
- priority names (`config['lock_priorities']`, a priority name => its Integer priority):
|
|
2682
|
+
- default names: `:lowest` (`-100`), `:low` (`-50`), `:normal` (`0`), `:high` (`50`), `:highest` (`100`);
|
|
2683
|
+
- name the priorities of your application (keep the default names with `merge`; the default value is frozen: assign
|
|
2684
|
+
a new hash);
|
|
2685
|
+
- a name is resolved to its Integer priority on each lock acquirement: the lock request queues, `#queue_info`/`#queues_info`,
|
|
2686
|
+
logs and instrumentation have the Integer priority; an unknown name fails with `RedisQueuedLocks::ArgumentError`;
|
|
2687
|
+
- `config['default_lock_priority']` can be a priority name too: an unknown one fails the configuration with
|
|
2688
|
+
`RedisQueuedLocks::ConfigValidationError` (checked after the configuration block, the keys can be assigned in any order,
|
|
2689
|
+
and on a direct `config[...] =` assignment);
|
|
2690
|
+
- `:queued` access strategy (default): the lock request is placed in the lock request queue after all requests of the same
|
|
2691
|
+
or higher priority and before all requests of the lower priority: strict priority between priorities, FIFO inside the same
|
|
2692
|
+
priority (read and write lock requests too, see [Read/Write Locks](#readwrite-locks));
|
|
2693
|
+
|
|
2694
|
+
```ruby
|
|
2695
|
+
# a balance operation goes ahead of the email notifications that are already waiting for the lock
|
|
2696
|
+
rql.lock("user_#{user.id}", priority: 50) { process_balance }
|
|
2697
|
+
|
|
2698
|
+
# a payment goes ahead of the balance operations
|
|
2699
|
+
rql.lock("user_#{user.id}", priority: 90) { process_payment }
|
|
2700
|
+
|
|
2701
|
+
# email notifications wait for all lock requests of higher priorities
|
|
2702
|
+
rql.lock("user_#{user.id}", priority: -10) { send_email }
|
|
2703
|
+
|
|
2704
|
+
# normal priority by default
|
|
2705
|
+
rql.lock("user_#{user.id}") { update_profile } # priority: 0
|
|
2706
|
+
|
|
2707
|
+
# the default priority of the client (for example, in background workers)
|
|
2708
|
+
rql.configure { |config| config['default_lock_priority'] = -10 }
|
|
2709
|
+
|
|
2710
|
+
# priority names: the default ones (:lowest, :low, :normal, :high, :highest) and the names of your application
|
|
2711
|
+
rql.configure do |config|
|
|
2712
|
+
config['lock_priorities'] = config['lock_priorities'].merge(payment: 90, balance: 50, email: -10)
|
|
2713
|
+
end
|
|
2714
|
+
rql.lock("user_#{user.id}", priority: :payment) { process_payment } # priority: 90
|
|
2715
|
+
rql.lock("user_#{user.id}", priority: :email) { send_email } # priority: -10
|
|
2716
|
+
rql.lock("user_#{user.id}", priority: :high) { update_profile } # priority: 50
|
|
2717
|
+
```
|
|
2718
|
+
|
|
2719
|
+
- how it works:
|
|
2720
|
+
- the position of the lock request in the queue (the score) is the request time shifted by the priority band:
|
|
2721
|
+
`request time - priority * 2**32 seconds` (each priority has its own range of positions; the default priority `0` keeps
|
|
2722
|
+
the request time: the positions of RQL versions without priorities);
|
|
2723
|
+
- every lock request drops dead requests (older than `queue_ttl`) of its own priority in both lock queues (in the same round
|
|
2724
|
+
trip as before priorities); dead requests of other priorities that block it (the head of the queue and the earlier request
|
|
2725
|
+
of the other read/write mode: the priority and the request time are decoded from the position) are dropped with all dead
|
|
2726
|
+
requests of their priorities in one more round trip of the same attempt, so a dead request of any priority never blocks the
|
|
2727
|
+
queue and the cost of an attempt does not depend on the number of priorities in use;
|
|
2728
|
+
- dead requests of lower priorities never block: they are dropped by the lock requests of their priority (or lower ones)
|
|
2729
|
+
and by `#clear_dead_requests` (it drops the dead requests of every priority found in the queues);
|
|
2730
|
+
- lock requests at infinite positions (`-inf`/`+inf`: manual writes of the lock queues, RQL never writes them) have the
|
|
2731
|
+
bound priorities (`100`/`-100`) in `#queue_info`/`#queues_info` and logs; a request at `-inf` is a dead one (like before
|
|
2732
|
+
priorities): it is dropped when it blocks a lock request and by `#clear_dead_requests`; a request at `+inf` is behind all
|
|
2733
|
+
requests and is never dropped as a dead one;
|
|
2734
|
+
- `#queue_info`/`#queues_info` return the priority of each lock request (`"priority"`); the lock acquirement logs and
|
|
2735
|
+
instrumentation events (classic, reentrant and lock series ones) have the `priority` of the lock request, the `log_lock_try`
|
|
2736
|
+
logs also have the priorities of the requests ahead (`first_priority_in_queue`, `ahead_priority`, `dead_priority`;
|
|
2737
|
+
`first_priority_in_queue => nil` when the lock queue has no requests);
|
|
2738
|
+
- limitations and recommendations:
|
|
2739
|
+
- the priority orders the waiting lock requests only: the obtained lock is never taken away (no preemption);
|
|
2740
|
+
- strict priority: a continuous flow of requests of a higher priority can starve the requests of lower priorities (they
|
|
2741
|
+
reach their `timeout`/`retry_count` limits): give them longer limits;
|
|
2742
|
+
- the released lock is passed by polling: the next lock request notices the released lock on its next attempt
|
|
2743
|
+
(`retry_delay`): use a shorter `retry_delay` for the important logic;
|
|
2744
|
+
- the default `retry_count` (`3`) gives a short waiting time: the priority matters for the requests that wait longer
|
|
2745
|
+
(`retry_count: nil` with `timeout`);
|
|
2746
|
+
- requests of the same priority made within ~61 µs of each other are ordered by their acquirer ids (the precision of
|
|
2747
|
+
the shifted positions);
|
|
2748
|
+
- each lock request drops dead requests with its own `queue_ttl` (the requests of its priority and the requests that block it):
|
|
2749
|
+
a request that waits longer than the shorter `queue_ttl` of such requests is dropped by their attempts and comes back at its
|
|
2750
|
+
old position on its own attempts (its own `queue_ttl` is not reached yet), so the requests behind it with a shorter
|
|
2751
|
+
`queue_ttl` don't wait for it: it loses its place (and the advantage of its priority) to them, the lock goes to the request
|
|
2752
|
+
whose attempt finds the lock free first; use the same `queue_ttl` for the lock requests of a lock (a longer `queue_ttl` does
|
|
2753
|
+
not keep a request in the queue);
|
|
2754
|
+
- `:random` access strategy: the lock request does not wait for its turn (the order of the queues and the priorities of other
|
|
2755
|
+
requests are ignored), but it is queued at the position of its priority: `:queued` lock requests wait for it (a `:random`
|
|
2756
|
+
request of a high priority holds back the waiting `:queued` requests of lower priorities until it obtains the lock or
|
|
2757
|
+
leaves the queue);
|
|
2758
|
+
- dead lock requests are dropped by the lock requests that meet them (the requests of their priority and the requests they
|
|
2759
|
+
block): the dead requests of the priorities that no lock request uses any more (and the dead requests met only by `:random`
|
|
2760
|
+
lock requests) stay in the queues; they never block (an accepted trade-off of the head-driven cleanup: the cost of an
|
|
2761
|
+
attempt does not depend on the number of priorities); without the [Lock Index](#lock-index) (its queues expire) and the
|
|
2762
|
+
[swarm](#swarm-mode-and-zombie-locks) (it drops the requests of dead processes) call `#clear_dead_requests` periodically
|
|
2763
|
+
(e.g. from a scheduled job) to drop them (`by_priority: true` for long lock queues: only the first lock request of each
|
|
2764
|
+
priority is read);
|
|
2765
|
+
- all processes that share locks should use RQL versions with priorities before positive priorities are used (`:high`,
|
|
2766
|
+
`:highest`, any priority above `0`): older versions drop such lock requests as dead ones (the order is broken; the mutual
|
|
2767
|
+
exclusion is not affected: lock queues never take part in it); the default priority (`0`) and negative priorities
|
|
2768
|
+
(`:low`, `:lowest`) work with older versions too: older versions keep such requests behind their own ones and never drop
|
|
2769
|
+
them as dead ones (the dead ones are dropped by the upgraded processes);
|
|
2770
|
+
|
|
2771
|
+
---
|
|
2772
|
+
|
|
2608
2773
|
## Read/Write Locks
|
|
2609
2774
|
|
|
2610
2775
|
<sup>\[[back to top](#table-of-contents)\]</sup>
|
|
@@ -2627,8 +2792,11 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
|
|
|
2627
2792
|
- lock request ordering (`access_strategy`):
|
|
2628
2793
|
- `:queued` (default) - FIFO between modes: read lock request waits for every earlier write lock request,
|
|
2629
2794
|
write lock request waits for every earlier read and write lock request (so writers are not starved by a
|
|
2630
|
-
continuous flow of readers and consecutive readers obtain the lock together);
|
|
2631
|
-
|
|
2795
|
+
continuous flow of readers and consecutive readers obtain the lock together); with
|
|
2796
|
+
[lock request priorities](#lock-request-prioritization) "earlier" means "ahead in the queue": a request of a higher
|
|
2797
|
+
priority is ahead of all requests of lower priorities in both modes (FIFO inside the same priority);
|
|
2798
|
+
- `:random` - the order of the lock queues is ignored: any free lock is obtained (a continuous flow of readers can starve
|
|
2799
|
+
writers); the lock request itself is queued like any lock request (`:queued` lock requests wait for it);
|
|
2632
2800
|
- read locks:
|
|
2633
2801
|
- each read lock has its own `ttl` and is released (after the block execution) or expired independently of other read locks;
|
|
2634
2802
|
- `ttl` should be a positive `Integer`;
|
|
@@ -2727,13 +2895,13 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
|
|
|
2727
2895
|
- default logs (raised from `#lock`/`#lock!`):
|
|
2728
2896
|
|
|
2729
2897
|
```ruby
|
|
2730
|
-
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2731
|
-
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2732
|
-
"[redis_queued_locks.dead_score_reached__reset_acquirer_position]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2733
|
-
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2734
|
-
"[redis_queued_locks.lock_obtaining_failed]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time", "result");
|
|
2735
|
-
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2736
|
-
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2898
|
+
"[redis_queued_locks.start_lock_obtaining]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2899
|
+
"[redis_queued_locks.start_try_to_lock_cycle]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2900
|
+
"[redis_queued_locks.dead_score_reached__reset_acquirer_position]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2901
|
+
"[redis_queued_locks.lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2902
|
+
"[redis_queued_locks.lock_obtaining_failed]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result");
|
|
2903
|
+
"[redis_queued_locks.extendable_reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2904
|
+
"[redis_queued_locks.reentrant_lock_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2737
2905
|
"[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2738
2906
|
"[redis_queued_locks.expire_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2739
2907
|
"[redis_queued_locks.decrease_lock]" # (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
@@ -2750,24 +2918,25 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
|
|
|
2750
2918
|
"[redis_queued_locks.try_lock.reentrant_lock__work_through]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
2751
2919
|
"[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
2752
2920
|
"[redis_queued_locks.try_lock.single_process_lock_conflict__lock_upgrade]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
2753
|
-
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2921
|
+
"[redis_queued_locks.try_lock.acq_added_to_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2754
2922
|
"[redis_queued_locks.try_lock.remove_expired_acqs]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2755
|
-
"[redis_queued_locks.try_lock.
|
|
2923
|
+
"[redis_queued_locks.try_lock.dead_request_removed]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "dead_acq_id", "dead_priority");
|
|
2924
|
+
"[redis_queued_locks.try_lock.get_first_from_queue]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue");
|
|
2756
2925
|
"[redis_queued_locks.try_lock.exit__queue_ttl_reached]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2757
|
-
"[redis_queued_locks.try_lock.exit__no_first]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "<current_lock_data>");
|
|
2758
|
-
"[redis_queued_locks.try_lock.exit__lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
|
2759
|
-
"[redis_queued_locks.try_lock.exit__write_request_ahead]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id");
|
|
2760
|
-
"[redis_queued_locks.try_lock.exit__read_request_ahead]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id");
|
|
2761
|
-
"[redis_queued_locks.try_lock.exit__read_lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue");
|
|
2926
|
+
"[redis_queued_locks.try_lock.exit__no_first]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue", "<current_lock_data>");
|
|
2927
|
+
"[redis_queued_locks.try_lock.exit__lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
|
2928
|
+
"[redis_queued_locks.try_lock.exit__write_request_ahead]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id", "ahead_priority");
|
|
2929
|
+
"[redis_queued_locks.try_lock.exit__read_request_ahead]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id", "ahead_priority");
|
|
2930
|
+
"[redis_queued_locks.try_lock.exit__read_lock_still_obtained]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue");
|
|
2762
2931
|
"[redis_queued_locks.try_lock.obtain__free_to_acquire]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2763
2932
|
```
|
|
2764
2933
|
|
|
2765
2934
|
- lock series logs (raised from `#lock_series`/`#lock_series!`, PoC; `rw_mode` is the mode of the series):
|
|
2766
2935
|
|
|
2767
2936
|
```ruby
|
|
2768
|
-
"[redis_queued_locks.start_lock_series_obtaining]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2769
|
-
"[redis_queued_locks.lock_series_obtained]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2770
|
-
"[redis_queued_locks.expire_lock_series]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2937
|
+
"[redis_queued_locks.start_lock_series_obtaining]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2938
|
+
"[redis_queued_locks.lock_series_obtained]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2939
|
+
"[redis_queued_locks.expire_lock_series]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2771
2940
|
```
|
|
2772
2941
|
|
|
2773
2942
|
- operation logs (one log line per operation that changes Redis keys, with the time of the whole operation; the same data as
|
|
@@ -2808,19 +2977,19 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
|
|
|
2808
2977
|
# - should implement `debug(progname = nil, &block)` (minimal requirement) or be an instance of Ruby's `::Logger` class/subclass;
|
|
2809
2978
|
# - supports `SemanticLogger::Logger` (see "semantic_logger" gem)
|
|
2810
2979
|
# - at this moment the only debug logs are realised in following cases:
|
|
2811
|
-
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2812
|
-
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2813
|
-
# - "[redis_queued_locks.dead_score_reached__reset_acquirer_position]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2814
|
-
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2815
|
-
# - "[redis_queued_locks.lock_obtaining_failed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time", "result");
|
|
2816
|
-
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2817
|
-
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2980
|
+
# - "[redis_queued_locks.start_lock_obtaining]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2981
|
+
# - "[redis_queued_locks.start_try_to_lock_cycle]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2982
|
+
# - "[redis_queued_locks.dead_score_reached__reset_acquirer_position]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2983
|
+
# - "[redis_queued_locks.lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2984
|
+
# - "[redis_queued_locks.lock_obtaining_failed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result");
|
|
2985
|
+
# - "[redis_queued_locks.extendable_reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2986
|
+
# - "[redis_queued_locks.reentrant_lock_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2818
2987
|
# - "[redis_queued_locks.fail_fast_or_limits_reached_or_deadlock__dequeue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2819
2988
|
# - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2820
2989
|
# - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2821
|
-
# - "[redis_queued_locks.start_lock_series_obtaining]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2822
|
-
# - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "acq_time");
|
|
2823
|
-
# - "[redis_queued_locks.expire_lock_series]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2990
|
+
# - "[redis_queued_locks.start_lock_series_obtaining]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2991
|
+
# - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
|
|
2992
|
+
# - "[redis_queued_locks.expire_lock_series]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2824
2993
|
# - "[redis_queued_locks.explicit_lock_release]" (logs "lock_key", "lock_key_queue", "rel_time", "at");
|
|
2825
2994
|
# - "[redis_queued_locks.explicit_read_lock_release]" (logs "lock_key", "acq_id", "lock_res", "rel_time", "at");
|
|
2826
2995
|
# - "[redis_queued_locks.explicit_all_locks_release]" (logs "at", "rel_time", "rel_key_cnt");
|
|
@@ -2845,15 +3014,16 @@ config['logger'] = RedisQueuedLocks::Logging::VoidLogger
|
|
|
2845
3014
|
# - "[redis_queued_locks.try_lock.reentrant_lock__work_through]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
2846
3015
|
# - "[redis_queued_locks.try_lock.single_process_lock_conflict__dead_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
2847
3016
|
# - "[redis_queued_locks.try_lock.single_process_lock_conflict__lock_upgrade]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "spc_status", "last_spc_ts");
|
|
2848
|
-
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
3017
|
+
# - "[redis_queued_locks.try_lock.acq_added_to_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
|
|
2849
3018
|
# - "[redis_queued_locks.try_lock.remove_expired_acqs]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2850
|
-
# - "[redis_queued_locks.try_lock.
|
|
3019
|
+
# - "[redis_queued_locks.try_lock.dead_request_removed]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "dead_acq_id", "dead_priority");
|
|
3020
|
+
# - "[redis_queued_locks.try_lock.get_first_from_queue]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue");
|
|
2851
3021
|
# - "[redis_queued_locks.try_lock.exit__queue_ttl_reached]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2852
|
-
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "<current_lock_data>");
|
|
2853
|
-
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
|
2854
|
-
# - "[redis_queued_locks.try_lock.exit__write_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id");
|
|
2855
|
-
# - "[redis_queued_locks.try_lock.exit__read_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id");
|
|
2856
|
-
# - "[redis_queued_locks.try_lock.exit__read_lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue");
|
|
3022
|
+
# - "[redis_queued_locks.try_lock.exit__no_first]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue", "<current_lock_data>");
|
|
3023
|
+
# - "[redis_queued_locks.try_lock.exit__lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue", "locked_by_acq_id", "<current_lock_data>");
|
|
3024
|
+
# - "[redis_queued_locks.try_lock.exit__write_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id", "ahead_priority");
|
|
3025
|
+
# - "[redis_queued_locks.try_lock.exit__read_request_ahead]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "ahead_acq_id", "ahead_priority");
|
|
3026
|
+
# - "[redis_queued_locks.try_lock.exit__read_lock_still_obtained]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "first_acq_id_in_queue", "first_priority_in_queue");
|
|
2857
3027
|
# - "[redis_queued_locks.try_lock.obtain__free_to_acquire]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
|
|
2858
3028
|
# - "rw_mode" is the requested lock mode ('write'/'read');
|
|
2859
3029
|
config['log_lock_try'] = false
|
|
@@ -2991,6 +3161,7 @@ Detalized event semantics and payload structure:
|
|
|
2991
3161
|
- `:hst_id` - `string` - lock's host identifier;
|
|
2992
3162
|
- `:lock_key` - `string` - lock name;
|
|
2993
3163
|
- `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3164
|
+
- `:priority` - `integer` - lock request priority: from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
2994
3165
|
- `:ts` - `numeric`/`epoch` - the time when the lock was obtaiend;
|
|
2995
3166
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
|
2996
3167
|
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
|
@@ -3003,6 +3174,7 @@ Detalized event semantics and payload structure:
|
|
|
3003
3174
|
- payload:
|
|
3004
3175
|
- `:lock_key` - `string` - lock name;
|
|
3005
3176
|
- `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3177
|
+
- `:priority` - `integer` - lock request priority: from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3006
3178
|
- `:ttl` - `integer`/`milliseconds` - requested lock ttl;
|
|
3007
3179
|
- `:acq_id` - `string` - lock acquirer identifier;
|
|
3008
3180
|
- `:hst_id` - `string` - lock's host identifier;
|
|
@@ -3020,6 +3192,7 @@ Detalized event semantics and payload structure:
|
|
|
3020
3192
|
- `:lock_key` - `string` - lock name;
|
|
3021
3193
|
- `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3022
3194
|
a read lock request under the write lock of the same acquirer works through the write lock (`:rw_mode` is `:read`);
|
|
3195
|
+
- `:priority` - `integer` - requested lock request priority: from `-100` to `100` (a priority name is reported as its integer priority; a reentrant lock request is not queued, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3023
3196
|
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
|
3024
3197
|
- `:acq_id` - `string` - lock acquirer identifier;
|
|
3025
3198
|
- `:hst_id` - `string` - lock's host identifier;
|
|
@@ -3035,6 +3208,7 @@ Detalized event semantics and payload structure:
|
|
|
3035
3208
|
- `:lock_key` - `string` - lock name;
|
|
3036
3209
|
- `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3037
3210
|
a read lock request under the write lock of the same acquirer works through the write lock (`:rw_mode` is `:read`);
|
|
3211
|
+
- `:priority` - `integer` - requested lock request priority: from `-100` to `100` (a priority name is reported as its integer priority; a reentrant lock request is not queued, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3038
3212
|
- `:ttl` - `integer`/`milliseconds` - last lock ttl by reentrant locking;
|
|
3039
3213
|
- `:acq_id` - `string` - lock acquirer identifier;
|
|
3040
3214
|
- `:hst_id` - `string` - lock's host identifier;
|
|
@@ -3053,6 +3227,7 @@ Detalized event semantics and payload structure:
|
|
|
3053
3227
|
- `:hst_id` - `string` - lock's host identifier;
|
|
3054
3228
|
- `:lock_key` - `string` - lock name;
|
|
3055
3229
|
- `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3230
|
+
- `:priority` - `integer` - lock request priority: from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3056
3231
|
- `:ts` - `numeric`/`epoch` - the time when lock was obtained;
|
|
3057
3232
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
|
3058
3233
|
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
|
@@ -3071,6 +3246,7 @@ Detalized event semantics and payload structure:
|
|
|
3071
3246
|
- `:lock_key` - `string` - lock name;
|
|
3072
3247
|
- `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3073
3248
|
a read lock request under the write lock of the same acquirer works through the write lock (`:rw_mode` is `:read`);
|
|
3249
|
+
- `:priority` - `integer` - requested lock request priority: from `-100` to `100` (a priority name is reported as its integer priority; a reentrant lock request is not queued, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3074
3250
|
- `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
|
|
3075
3251
|
- `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
|
|
3076
3252
|
|
|
@@ -3081,6 +3257,7 @@ Detalized event semantics and payload structure:
|
|
|
3081
3257
|
- payload:
|
|
3082
3258
|
- `:lock_keys` - `array<string>` - lock names of the series;
|
|
3083
3259
|
- `:rw_mode` - `symbol` - the mode of the series (`read_write_mode`): `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3260
|
+
- `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3084
3261
|
- `:ttl` - `integer`/`milliseconds` - the `ttl` option of the series (each lock of the series gets its own ttl,
|
|
3085
3262
|
see [lock_series](#lock_series---poc-acquire-a-series-of-locks));
|
|
3086
3263
|
- `:acq_id` - `string` - lock acquirer identifier;
|
|
@@ -3096,6 +3273,7 @@ Detalized event semantics and payload structure:
|
|
|
3096
3273
|
- payload:
|
|
3097
3274
|
- `:lock_keys` - `array<string>` - lock names of the series;
|
|
3098
3275
|
- `:rw_mode` - `symbol` - the mode of the series (`read_write_mode`): `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
|
|
3276
|
+
- `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
|
|
3099
3277
|
- `:hold_time` - `float`/`milliseconds` - the series hold time;
|
|
3100
3278
|
- `:ttl` - `integer`/`milliseconds` - the `ttl` option of the series;
|
|
3101
3279
|
- `:acq_id` - `string` - lock acquirer identifier;
|
|
@@ -3203,7 +3381,6 @@ Detalized event semantics and payload structure:
|
|
|
3203
3381
|
- isolated timeouts which are independent of internal Ruby's timeout implementation (where all timeouts are hostend inside
|
|
3204
3382
|
the global "timeout request" queue and managed by a single global "timeout wathcer" thread). it should prevent any logic and timeout intersections,
|
|
3205
3383
|
some GVL-related things and problem situations when the global watcher thread is "dead";
|
|
3206
|
-
- lock request prioritization;
|
|
3207
3384
|
- **strict redlock algorithm support** (support for many `RedisClient` instances that are fully independent (distributed redis instances));
|
|
3208
3385
|
- #lock_series` - acquire a series of locks:
|
|
3209
3386
|
- (NOTE: "Proof of Concept" implementation is ready to use);
|
|
@@ -3221,6 +3398,7 @@ Detalized event semantics and payload structure:
|
|
|
3221
3398
|
Mutex (instantiated inside the main Ractor) from the our non-main Ractor (swar-mode works under the non-main ractor) and this process fails\
|
|
3222
3399
|
on non-sharable memory error);
|
|
3223
3400
|
- **Minor**:
|
|
3401
|
+
- `*_status` aliases for `*_info` methods;
|
|
3224
3402
|
- an ability to return all insturmentation metrics from the `lock` invocation (and after block `yield`ing);
|
|
3225
3403
|
- think about a PG-like `stats`-data/table, that should be helpful for lock history acquirement analyzation :thinking: (suitable for cases when we want
|
|
3226
3404
|
to check if the lock was acquired by some different parallel process durign the some period of time in the past or not);
|
|
@@ -3269,6 +3447,17 @@ Detalized event semantics and payload structure:
|
|
|
3269
3447
|
bundle exec rake rspec
|
|
3270
3448
|
```
|
|
3271
3449
|
|
|
3450
|
+
- Tests in parallel processes (`RSpec`, ~6x faster): each process (worker) works with its own pair of Redis databases
|
|
3451
|
+
(the specs use db 0 and db 1), so `WORKERS=n` flushes db `0..2n-1` of the local Redis (the default number of workers:
|
|
3452
|
+
the Redis `databases` / 2, up to the CPU count); the examples are distributed by their durations of the previous runs
|
|
3453
|
+
(`tmp/rspec_parallel/durations.json`), the output of each worker is in `tmp/rspec_parallel/worker-<n>.log`, the coverage
|
|
3454
|
+
report of the whole run is collated into `coverage/` after all workers:
|
|
3455
|
+
```shell
|
|
3456
|
+
bundle exec rake rspec:parallel
|
|
3457
|
+
WORKERS=4 bundle exec rake rspec:parallel
|
|
3458
|
+
bundle exec rake "rspec:parallel[spec/redis_queued_locks_spec.rb]" # spec files/filters
|
|
3459
|
+
```
|
|
3460
|
+
|
|
3272
3461
|
- Linting (`Rubocop`+`RBS`):
|
|
3273
3462
|
```shell
|
|
3274
3463
|
bundle exec rake rubocop
|
|
@@ -3281,7 +3470,8 @@ bundle exec rake steep:check
|
|
|
3281
3470
|
|
|
3282
3471
|
- `Runtime` TypeChecking (`RBS`):
|
|
3283
3472
|
```shell
|
|
3284
|
-
bundle exec rbs collection install && RBS_TEST_RAISE=true RUBYOPT='-rrbs/test/setup' RBS_TEST_OPT='-I sig' RBS_TEST_LOGLEVEL=error RBS_TEST_TARGET='RedisQueuedLocks::*' bundle exec rspec
|
|
3473
|
+
bundle exec rbs collection install && RBS_TEST_RAISE=true RUBYOPT='-rbundler/setup -rrbs/test/setup' RBS_TEST_OPT='-I sig' RBS_TEST_LOGLEVEL=error RBS_TEST_TARGET='RedisQueuedLocks::*' RBS_TEST_SKIP='::RedisQueuedLocks::Swarm::FlushZombies,::RedisQueuedLocks::Swarm::SwarmElement::Isolated,::RedisQueuedLocks::Swarm::RedisClientBuilder' bundle exec rspec
|
|
3474
|
+
# or in parallel processes: the same environment with `bundle exec rake rspec:parallel` instead of `bundle exec rspec`
|
|
3285
3475
|
```
|
|
3286
3476
|
|
|
3287
3477
|
---
|