redis_queued_locks 1.22.0 → 1.23.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/project-overview.md +90 -38
  3. data/.claude/rules/acquirer.md +12 -7
  4. data/.claude/rules/arguments.md +2 -2
  5. data/.claude/rules/logic.md +5 -4
  6. data/.claude/rules/tests.md +4 -4
  7. data/.claude/rules/type-checking.md +2 -3
  8. data/.claude/rules/visitors.md +8 -6
  9. data/.claude/skills/rql-code-review/SKILL.md +25 -20
  10. data/.ruby-version +1 -1
  11. data/CHANGELOG.md +30 -1
  12. data/CLAUDE.md +5 -4
  13. data/README.md +259 -103
  14. data/lib/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rb +4 -4
  15. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +582 -276
  16. data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire.rb +29 -12
  17. data/lib/redis_queued_locks/acquirer/acquire_lock.rb +36 -27
  18. data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +13 -13
  19. data/lib/redis_queued_locks/acquirer/lock_series/instr_visitor.rb +243 -0
  20. data/lib/redis_queued_locks/acquirer/lock_series/log_visitor.rb +472 -0
  21. data/lib/redis_queued_locks/acquirer/lock_series/release_lock_series.rb +181 -0
  22. data/lib/redis_queued_locks/acquirer/lock_series/try_to_lock_series.rb +1051 -0
  23. data/lib/redis_queued_locks/acquirer/lock_series.rb +957 -0
  24. data/lib/redis_queued_locks/acquirer.rb +1 -1
  25. data/lib/redis_queued_locks/client.rb +55 -25
  26. data/lib/redis_queued_locks/errors.rb +8 -0
  27. data/lib/redis_queued_locks/version.rb +2 -2
  28. data/redis_queued_locks.gemspec +21 -14
  29. data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rbs +1 -1
  30. data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +106 -5
  31. data/sig/redis_queued_locks/acquirer/acquire_lock.rbs +1 -1
  32. data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +2 -2
  33. data/sig/redis_queued_locks/acquirer/lock_series/instr_visitor.rbs +102 -0
  34. data/sig/redis_queued_locks/acquirer/lock_series/log_visitor.rbs +154 -0
  35. data/sig/redis_queued_locks/acquirer/lock_series/release_lock_series.rbs +37 -0
  36. data/sig/redis_queued_locks/acquirer/lock_series/try_to_lock_series.rbs +198 -0
  37. data/sig/redis_queued_locks/acquirer/lock_series.rbs +106 -0
  38. data/sig/redis_queued_locks/client.rbs +70 -3
  39. data/sig/redis_queued_locks/errors.rbs +6 -0
  40. metadata +32 -11
  41. data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +0 -61
  42. data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +0 -104
  43. data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +0 -513
data/README.md CHANGED
@@ -29,8 +29,8 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
29
29
  - [Usage](#usage)
30
30
  - [lock](#lock---obtain-a-lock)
31
31
  - [lock!](#lock---exceptional-lock-obtaining)
32
- - [lock_series](#lock_series---poc-acquire-a-series-of-locks) (PoC)
33
- - [lock_series!](#lock_series---poc-exceptional-lock_series) (PoC)
32
+ - [lock_series](#lock_series---acquire-a-series-of-locks)
33
+ - [lock_series!](#lock_series---exceptional-lock_series)
34
34
  - [lock_info](#lock_info)
35
35
  - [queue_info](#queue_info)
36
36
  - [locked?](#locked)
@@ -85,8 +85,12 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
85
85
  - ["redis_queued_locks.reentrant_lock_obtained"](#redis_queued_locksreentrant_lock_obtained)
86
86
  - ["redis_queued_locks.lock_hold_and_release"](#redis_queued_lockslock_hold_and_release)
87
87
  - ["redis_queued_locks.reentrant_lock_hold_completes"](#redis_queued_locksreentrant_lock_hold_completes)
88
+ - ["redis_queued_locks.lock_series_intermediate_lock_obtained"](#redis_queued_lockslock_series_intermediate_lock_obtained)
89
+ - ["redis_queued_locks.lock_series_lock_obtained"](#redis_queued_lockslock_series_lock_obtained)
90
+ - ["redis_queued_locks.lock_series_lock_obtaining_failed"](#redis_queued_lockslock_series_lock_obtaining_failed)
88
91
  - ["redis_queued_locks.lock_series_obtained"](#redis_queued_lockslock_series_obtained)
89
92
  - ["redis_queued_locks.lock_series_hold_and_release"](#redis_queued_lockslock_series_hold_and_release)
93
+ - ["redis_queued_locks.lock_series_obtaining_failed"](#redis_queued_lockslock_series_obtaining_failed)
90
94
  - ["redis_queued_locks.explicit_lock_release"](#redis_queued_locksexplicit_lock_release)
91
95
  - ["redis_queued_locks.explicit_read_lock_release"](#redis_queued_locksexplicit_read_lock_release)
92
96
  - ["redis_queued_locks.explicit_all_locks_release"](#redis_queued_locksexplicit_all_locks_release)
@@ -376,8 +380,16 @@ client = RedisQueuedLocks::Client.new(redis_client) do |config|
376
380
  # - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
377
381
  # - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
378
382
  # - "[redis_queued_locks.start_lock_series_obtaining]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
379
- # - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
383
+ # - "[redis_queued_locks.start_try_to_lock_series_cycle]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
384
+ # - "[redis_queued_locks.dead_score_reached__reset_series_position]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
385
+ # - "[redis_queued_locks.lock_series_intermediate_lock_obtained]" (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
386
+ # - "[redis_queued_locks.lock_series_lock_obtained]" (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
387
+ # - "[redis_queued_locks.lock_series_lock_obtaining_failed]" (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result");
388
+ # - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "reentrant_lock_keys");
389
+ # - "[redis_queued_locks.lock_series_obtaining_failed]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result", "failed_lock_key");
390
+ # - "[redis_queued_locks.dequeue_from_lock_series_queues]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
380
391
  # - "[redis_queued_locks.expire_lock_series]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
392
+ # - "[redis_queued_locks.decrease_lock_series]" (logs "lock_keys", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
381
393
  # - "[redis_queued_locks.explicit_lock_release]" (logs "lock_key", "lock_key_queue", "rel_time", "at");
382
394
  # - "[redis_queued_locks.explicit_read_lock_release]" (logs "lock_key", "acq_id", "lock_res", "rel_time", "at");
383
395
  # - "[redis_queued_locks.explicit_all_locks_release]" (logs "at", "rel_time", "rel_key_cnt");
@@ -468,8 +480,8 @@ end
468
480
 
469
481
  - [lock](#lock---obtain-a-lock)
470
482
  - [lock!](#lock---exceptional-lock-obtaining)
471
- - [lock_series](#lock_series---poc-acquire-a-series-of-locks) (PoC)
472
- - [lock_series!](#lock_series---poc-exceptional-lock_series) (PoC)
483
+ - [lock_series](#lock_series---acquire-a-series-of-locks)
484
+ - [lock_series!](#lock_series---exceptional-lock_series)
473
485
  - [lock_info](#lock_info)
474
486
  - [queue_info](#queue_info)
475
487
  - [locked?](#locked)
@@ -528,7 +540,7 @@ def lock(
528
540
  raise_errors: false,
529
541
  fail_fast: false,
530
542
  conflict_strategy: config['default_conflict_strategy'],
531
- read_write_mode: :write,
543
+ rw_mode: :write,
532
544
  access_strategy: config['default_access_strategy'],
533
545
  priority: config['default_lock_priority'],
534
546
  identity: uniq_identity, # (attr_accessor) calculated during client instantiation via config['uniq_identifier'] proc;
@@ -577,6 +589,11 @@ def lock(
577
589
  - the block is interrupted by an internal `RedisQueuedLocks::Exception` (an `::Exception`, like `Timeout::ExitException` of Ruby's `Timeout`): `rescue => e`,
578
590
  `rescue StandardError`, `rescue Timeout::Error` and `rescue nil` inside the block can not swallow it (only `rescue Exception`),
579
591
  `ensure` of the block still runs;
592
+ - the block gets the lock's TTL minus the time from the start of the successful acquirement attempt and minus a 2 ms margin;
593
+ with no time left the block is not invoked (`RedisQueuedLocks::LockTtlTimeBudgetIsSpentError`);
594
+ - both failures have the common superclass `RedisQueuedLocks::TimedLockTimeoutError`: `RedisQueuedLocks::TimedBlockExceedsLockTtlError` (the block exceeded the time) and `RedisQueuedLocks::LockTtlTimeBudgetIsSpentError` (the block was not invoked);
595
+ - (limitation) `timed` can only interrupt the block: a long `ensure` of the block, `rescue Exception` in the block and the code that
596
+ can not be interrupted (a C extension holding the GVL) can outlive the lock;
580
597
  - pre-configured in `config['is_timed_by_default']`;
581
598
  - `false` by default;
582
599
  - `retry_count` - (optional) `[Integer,NilClass]`
@@ -637,7 +654,7 @@ def lock(
637
654
  - `:wait_for_lock` - (default) - work in classic way (with timeouts, retry delays, retry limits, etc - in classic way :));
638
655
  - `:dead_locking` - fail with deadlock exception;
639
656
  - See [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks) documentation section for details;
640
- - `read_write_mode` - (optional) - `[Symbol]`
657
+ - `rw_mode` - (optional) - `[Symbol]`
641
658
  - The lock mode (read/write lock semantics);
642
659
  - `:write` by default;
643
660
  - Modes:
@@ -799,7 +816,7 @@ Return value:
799
816
  { ok: false, result: :timeout_reached } # see <timeout> option
800
817
  { ok: false, result: :retry_limit_reached } # see <retry_count> option
801
818
  { ok: false, result: :conflict_dead_lock } # see <conflict_strategy> option for details (:dead_locking strategy)
802
- { ok: false, result: :conflict_lock_upgrade } # read-to-write lock upgrade (see <read_write_mode> and <conflict_strategy> options)
819
+ { ok: false, result: :conflict_lock_upgrade } # read-to-write lock upgrade (see <rw_mode> and <conflict_strategy> options)
803
820
  { ok: false, result: :fail_fast_no_try } # see <fail_fast> option
804
821
  { ok: false, result: :fail_fast_after_try } # see <fail_fast> option
805
822
  ```
@@ -825,7 +842,7 @@ rql.lock("my_lock", ttl: 5_000, timed: true) { sleep(4) }
825
842
  # => OK
826
843
 
827
844
  rql.lock("my_lock", ttl: 5_000, timed: true) { sleep(6) }
828
- # => fails with RedisQueuedLocks::TimedLockTimeoutError
845
+ # => fails with RedisQueuedLocks::TimedBlockExceedsLockTtlError (a RedisQueuedLocks::TimedLockTimeoutError)
829
846
  ```
830
847
 
831
848
  - infinite lock obtaining (no retry limit, no timeout limit):
@@ -989,7 +1006,7 @@ def lock!(
989
1006
  access_strategy: config['default_access_strategy'],
990
1007
  priority: config['default_lock_priority'],
991
1008
  conflict_strategy: config['default_conflict_strategy'],
992
- read_write_mode: :write,
1009
+ rw_mode: :write,
993
1010
  log_sampling_enabled: config['log_sampling_enabled'],
994
1011
  log_sampling_percent: config['log_sampling_percent'],
995
1012
  log_sampler: config['log_sampler'],
@@ -1006,39 +1023,56 @@ See `#lock` method [documentation](#lock---obtain-a-lock).
1006
1023
 
1007
1024
  ---
1008
1025
 
1009
- #### #lock_series - (PoC) acquire a series of locks
1026
+ #### #lock_series - acquire a series of locks
1010
1027
 
1011
1028
  <sup>\[[back to top](#usage)\]</sup>
1012
1029
 
1013
- > (IMPORTANT!) "Proof of Concept" realization. Will be reworked in the future.
1014
-
1015
- `lock_series` - acquire a series of locks simultaniously and hold them all while your block of code is executing
1016
- or (if block is not passed) while all locks will not expire.
1017
-
1018
- If it is not possible to obtain all locks at the same time (taking into queue ttl, retry attempts count for each lock, etc) no any lock will be acquired.
1019
-
1020
- Method options is the same as for `lock` method: `retry_count`, `retry_jitter`, `ttl`, `queue_ttl`, etc. Each option will be
1021
- applied with the same value to the each lock acquirement process (for each lock in a series separately).
1022
-
1023
- **IMPORTANT DETAILS AND LIMITATIONS**:
1024
- - (this behavior will be reworked with another algorithm in future);
1025
- - each lock will be released with the next formula:
1026
- - `lock position with reverse numbering` * `ttl`
1027
- - (positions exmaple: ["a", "b", "c"] => "a" position is 3, "b" position is 2, "c" position is 1);
1028
- - this means that in case of 3 locks (`lock_series("x", "y", "z", ttl: 5_000`) with `5 seconds TTL` your locks will be with the next TTL:
1029
- - `"x"` - 15 seconds
1030
- - `"y"` - 10 seconds
1031
- - `"z"` - 5 seconds
1032
- - when your block of code has completed their execution - all locks will be released immediatly;
1033
- - when the series fails (any lock can not be obtained, with or without an exception) - the locks that are already
1034
- obtained by the series are released immediatly (`acquired_locks` of the failure result lists them);
1030
+ `lock_series` - obtain a group of locks atomically: all locks of the series are obtained at the same moment with the same TTL
1031
+ and are held together while your block of code is executing (or, if a block is not passed, while the locks live).
1032
+
1033
+ **Guarantees**:
1034
+ - **all or nothing**: one transaction obtains all locks of the series; an attempt that can not obtain some lock obtains nothing,
1035
+ so a failed series leaves no obtained locks (and no lock requests in the lock queues);
1036
+ - **all locks at the same time**: all locks are obtained with the same TTL (`ttl:`) at the same moment, so the block of code
1037
+ never works with a partially held series and `timed: true` limits it by the real lifetime of the locks;
1038
+ - **no hold-and-wait**: nothing is held while the series waits for its locks: the free locks of the series are not locked
1039
+ and their TTL is not spent on the waiting;
1040
+ - **no deadlocks between lock series**: the order of the lock names does not matter (`lock_series("a", "b")` and
1041
+ `lock_series("b", "a")` work in parallel threads): the lock request of the series has one position in all its lock queues,
1042
+ so the lock requests wait only for earlier lock requests. Your own nested logic is out of it: code that holds a lock and
1043
+ then waits for a series still can wait for a lock that is needed by another acquirer (the waiting is limited by the limits
1044
+ of the series);
1045
+
1046
+ **How the options work**:
1047
+ - method options are the same as for the `#lock` method: `ttl`, `queue_ttl`, `retry_count`, `retry_delay`, `retry_jitter`, etc;
1048
+ - limits are the limits of **the whole series**, not of each lock: `timeout:` is one time period for all locks, `retry_count:` is
1049
+ the number of attempts to obtain all locks, `queue_ttl:` is the lifetime of the lock request of the series;
1050
+ - `ttl:` is the TTL of each lock of the series (the same value for all locks);
1051
+ - `rw_mode:` is the same for all locks of the series (see [Read/Write Locks](#readwrite-locks));
1052
+ - `priority:` is the same for all locks of the series (see [Lock Request Prioritization](#lock-request-prioritization)):
1053
+ the lock request of the series has this priority in all its lock queues;
1054
+ - `conflict_strategy:` works per lock: reentrant locks (already obtained by your outer logic, `:work_through`/`:extendable_work_through`)
1055
+ are not obtained and not released by the series: the extended ones (`:extendable_work_through`) get their TTL back after the block,
1056
+ like in the `#lock` method; a lock with a failed conflict (`:dead_locking`, read-to-write lock upgrade) fails the whole series;
1057
+ - `fail_fast: true` fails the series on the first lock that is already obtained;
1058
+ - a duplicated lock name is the same lock (`lock_series("a", "b", "a")` works with two locks); at least one lock name is required
1059
+ (`RedisQueuedLocks::ArgumentError`);
1060
+
1061
+ **Details and limitations**:
1062
+ - when your block of code has completed its execution (with or without an exception) all locks obtained by the series are released;
1063
+ - an interrupted series (`Timeout.timeout` of your logic, `Thread#kill`, a Redis error) releases its locks and drops its lock requests,
1064
+ when it is possible (a dead process or an unavailable Redis: the locks live till their TTL, the lock requests till their `queue_ttl`);
1035
1065
  - the series releases only locks obtained by itself and only if they are still owned by the current acquirer:
1036
- - reentrant locks (already obtained by your outer logic, `conflict_strategy: :work_through`/`:extendable_work_through`) are kept;
1037
- - a lock that is expired during your block and obtained by another acquirer is kept;
1038
- - `read_write_mode:` is the same for all locks of the series (see [Read/Write Locks](#readwrite-locks));
1039
- - `priority:` is the same for all locks of the series (see [Lock Request Prioritization](#lock-request-prioritization));
1040
- - if you did not pass a block - all locks will be released with their forumla-based TTL described above;
1041
- - **HOW WILL THIS CASE WORK IN FUTURE RELESAE**: all locks will have the same TTL (`5 seconds` for our example, not `15/10/5 seconds`);
1066
+ a lock that is expired during your block and obtained by another acquirer is kept;
1067
+ - if you did not pass a block - all locks live till their TTL;
1068
+ - locks of the series are regular locks with regular keys: `#lock_info`, `#unlock`, `#extend_lock_ttl`, `#clear_locks` and the swarm
1069
+ work with each of them separately;
1070
+ - `:queued` access strategy: a series that is ahead of other lock requests makes them wait even for its free locks (the waiting
1071
+ series is the next obtainer of these locks): one long-waiting series delays the lock requests of all its locks for its waiting time
1072
+ (no longer than its `queue_ttl`, `timeout` and `retry_count` limits allow);
1073
+ - `:random` access strategy: the series is obtained only when all its locks are free at the same moment, so a big series can
1074
+ wait for a long time under load (lock requests of the `:random` strategy have no reserved turn);
1075
+ - all processes that work with the same locks should use the same RQL version: older versions obtain the series lock by lock;
1042
1076
 
1043
1077
  ##### How to use:
1044
1078
 
@@ -1049,12 +1083,27 @@ applied with the same value to the each lock acquirement process (for each lock
1049
1083
  client.lock_series("a", "b", "c") { ...some_code... }
1050
1084
 
1051
1085
  # result-oriented way:
1052
- # - (block is passed)
1086
+ # - (block is not passed)
1087
+ client.lock_series('x', 'y', 'z', ttl: 5_000)
1088
+ # =>
1089
+ {
1090
+ ok: true,
1091
+ result: {
1092
+ lock_series: ["x", "y", "z"], # your locks
1093
+ rql_lock_series: ["rql:lock:x", "rql:lock:y", "rql:lock:z"], # internal RQL lock-keys of your locks
1094
+ acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", # the acquirer of the locks
1095
+ hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", # the host of the locks
1096
+ ts: 1769506100.676884, # the time when the locks were obtained, epoch
1097
+ ttl: 5000 # the ttl of each lock, milliseconds
1098
+ }
1099
+ }
1100
+
1101
+ # - (detailed result, block is passed)
1053
1102
  detailed_result = client.lock_series('x', 'y', 'z', detailed_result: true) { 1 + 2 }
1054
1103
  detailed_result # =>
1055
1104
  {
1056
1105
  yield_result: 3, # result of your block of code
1057
- lock_release_strategy: :immediate_release_after_yield, # (block is passed)
1106
+ locks_release_strategy: :immediate_release_after_yield, # (block is passed)
1058
1107
  locks_released_at: 1769506100.676884, # (instrumentation) release time, epoch
1059
1108
  locks_acq_time: 7.0, # (instrumentation) time spent to acquire all locks, milliseconds
1060
1109
  locks_hold_time: 0.65, # (instrumentation) lock series hold period (hold preiod of all required locks), milliseconds
@@ -1062,12 +1111,12 @@ detailed_result # =>
1062
1111
  rql_lock_series: ["rql:lock:x", "rql:lock:y", "rql:lock:z"] # internal RQL lock-keys of your locks
1063
1112
  }
1064
1113
 
1065
- # - (block is not passed)
1114
+ # - (detailed result, block is not passed)
1066
1115
  detailed_result = client.lock_series('x', 'y', 'z', detailed_result: true)
1067
1116
  detailed_result # =>
1068
1117
  {
1069
1118
  yield_result: nil, # block is not given - nothing to return
1070
- lock_release_strategy: :redis_key_ttl, # block is not given - locks are living in redis with their TTL
1119
+ locks_release_strategy: :redis_key_ttl, # block is not given - locks are living in redis with their TTL
1071
1120
  locks_released_at: nil, # (instrumentation) # block is not given - locks are living in redis with their TTL
1072
1121
  locks_acq_time: 7.0, # (instrumentation) time spent to acquire all locks, milliseconds
1073
1122
  locks_hold_time: nil, # (instrumentation) # block is not given - locks are living in redis with their TTL
@@ -1084,53 +1133,71 @@ detailed_result # =>
1084
1133
  client.lock('a', ttl: 10_000_000)
1085
1134
 
1086
1135
  # - try to bobtain series of locks with `a` inside the series (`a` can not be obtained for now)
1087
- client.lock_seires('c', 'a', 'b') # ... `a` is still obtained
1136
+ client.lock_series('c', 'a', 'b') # ... `a` is still obtained
1088
1137
  # =>
1089
1138
  {
1090
1139
  ok: false,
1091
1140
  result: {
1092
1141
  error: :failed_to_acquire_lock_series,
1093
- detailed_errors: { 'a' => { ok: false, result: :retry_limit_reached } },
1142
+ detailed_errors: { 'a' => { ok: false, result: :retry_limit_reached } }, # (empty without `failed_on_lock`)
1094
1143
  lock_series: ["c", "a", "b"],
1095
- acquired_locks: ["c"],
1096
- missing_locks: ["a", "b"],
1097
- failed_on_lock: ["a"],
1144
+ acquired_locks: [], # always empty: nothing is obtained by the failed series
1145
+ missing_locks: ["c", "a", "b"],
1146
+ failed_on_lock: "a", # the first lock that blocked the last completed attempt (`nil`: unknown)
1098
1147
  }
1099
1148
  }
1100
1149
  ```
1101
1150
 
1102
- ###### New instrumentaiton events:
1103
-
1104
- - `rw_mode` is the mode of the series (`read_write_mode`, the same for all locks of the series);
1151
+ ###### Instrumentation events:
1152
+
1153
+ - `rw_mode` is the mode of the series (the same for all locks of the series);
1154
+ - `#lock` events (`lock_obtained`, `lock_obtaining_failed`, ...) are not raised for the locks of the series, each lock has its own series events
1155
+ with the series context in the name (`lock_key` is the lock, `lock_keys` are all locks of the series):
1156
+ - `lock_series_intermediate_lock_obtained` - intermediate obtaining: the lock passed all checks of an attempt of the series (the lock is
1157
+ free and the lock request of the series is next). Nothing is held yet: the series can still wait for its other locks or fail.
1158
+ Raised once per lock of the series (not per attempt), in the order of the lock names up to the lock that blocks the series;
1159
+ - `lock_series_lock_obtained` - full obtaining: the lock is really obtained, all locks of the series are held;
1160
+ - `lock_series_lock_obtaining_failed` - the lock that blocked the failed series, once per series (not raised when the blocking lock
1161
+ is unknown: no attempt found it, see `failed_lock_key` of `lock_series_obtaining_failed`);
1162
+ - reentrant locks are not obtained by the series: they have no lock events (see `reentrant_lock_keys` of the series events);
1105
1163
  - see [Instrumentation Events](#instrumentation-events) for the payload details;
1106
1164
 
1107
1165
  ```ruby
1108
- # NEW instrumentation events (examples)
1109
- "redis_queued_locks.lock_series_obtained" => # {lock_keys: ["rql:lock:s", "rql:lock:t", "rql:lock:u"], rw_mode: :write, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4688692, acq_time: 2.61, instrument: nil}
1110
- "redis_queued_locks.lock_series_hold_and_release" => # {lock_keys: ["rql:lock:x", "rql:lock:y", "rql:lock:z"], rw_mode: :write, hold_time: 0.29, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4717052, acq_time: 4.26, instrument: nil}
1166
+ # instrumentation events (examples)
1167
+ "redis_queued_locks.lock_series_intermediate_lock_obtained" => # {lock_key: "rql:lock:s", lock_keys: ["rql:lock:s", "rql:lock:t"], rw_mode: :write, priority: 0, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4688692, acq_time: 2.61, instrument: nil}
1168
+ "redis_queued_locks.lock_series_lock_obtained" => # {lock_key: "rql:lock:s", lock_keys: ["rql:lock:s", "rql:lock:t"], rw_mode: :write, priority: 0, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4688692, acq_time: 2.61, instrument: nil}
1169
+ "redis_queued_locks.lock_series_lock_obtaining_failed" => # {lock_key: "rql:lock:y", lock_keys: ["rql:lock:x", "rql:lock:y"], rw_mode: :write, priority: 0, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4717052, acq_time: 4.26, result: :retry_limit_reached, instrument: nil}
1170
+ "redis_queued_locks.lock_series_obtained" => # {lock_keys: ["rql:lock:s", "rql:lock:t", "rql:lock:u"], rw_mode: :write, priority: 0, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4688692, acq_time: 2.61, reentrant_lock_keys: [], instrument: nil}
1171
+ "redis_queued_locks.lock_series_hold_and_release" => # {lock_keys: ["rql:lock:x", "rql:lock:y", "rql:lock:z"], rw_mode: :write, priority: 0, hold_time: 0.29, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4717052, acq_time: 4.26, reentrant_lock_keys: [], instrument: nil}
1172
+ "redis_queued_locks.lock_series_obtaining_failed" => # {lock_keys: ["rql:lock:x", "rql:lock:y"], rw_mode: :write, priority: 0, ttl: 5000, acq_id: "rql:acq:4486/1696/1704/1712/e6fd0da7991e4303", hst_id: "rql:hst:4486/1696/1712/e6fd0da7991e4303", ts: 1769627106.4717052, acq_time: 4.26, result: :retry_limit_reached, failed_lock_key: "rql:lock:y", instrument: nil}
1111
1173
  ```
1112
1174
 
1113
- ###### New logs:
1175
+ ###### Logs:
1114
1176
 
1115
- - `rw_mode` is the mode of the series (`read_write_mode`, the same for all locks of the series);
1177
+ - `rw_mode` is the mode of the series (the same for all locks of the series);
1116
1178
  - see [Logging](#logging) for the logged keys;
1117
1179
 
1118
1180
  ```shell
1119
- # NEW logs (examples)
1181
+ # logs (examples)
1120
1182
  [redis_queued_locks.start_lock_series_obtaining] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0
1121
- [redis_queued_locks.lock_series_obtained] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0 acq_time => 2.55 (ms)
1122
- [redis_queued_locks.expire_lock_series] lock_keys => '["rql:lock:x", "rql:lock:y", "rql:lock:z"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0
1183
+ [redis_queued_locks.start_try_to_lock_series_cycle] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0
1184
+ [redis_queued_locks.lock_series_intermediate_lock_obtained] lock_key => 'rql:lock:a' lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0 acq_time => 2.55 (ms)
1185
+ [redis_queued_locks.lock_series_lock_obtained] lock_key => 'rql:lock:a' lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0 acq_time => 2.55 (ms)
1186
+ [redis_queued_locks.lock_series_obtained] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0 acq_time => 2.55 (ms) reentrant_lock_keys => '[]'
1187
+ [redis_queued_locks.expire_lock_series] lock_keys => '["rql:lock:a", "rql:lock:b"]' queue_ttl => 15 acq_id => 'rql:acq:4486/1696/1704/1712/e6fd0da7991e4303' hst_id => 'rql:hst:4486/1696/1712/e6fd0da7991e4303' acs_strat => 'queued' rw_mode => 'write' priority => 0
1123
1188
  ```
1124
1189
 
1125
1190
  ----
1126
1191
 
1127
- #### #lock_series! - (PoC) exceptional `lock_series`
1192
+ #### #lock_series! - exceptional `lock_series`
1128
1193
 
1129
1194
  <sup>\[[back to top](#usage)\]</sup>
1130
1195
 
1131
- > (IMPORTANT!) "Proof of Concept" realization. Will be reworked in the future (very-very soon)
1196
+ Raises the same errors as `#lock!` when the series can not be obtained (`RedisQueuedLocks::LockAlreadyObtainedError`,
1197
+ `RedisQueuedLocks::ConflictLockObtainError`, `RedisQueuedLocks::LockAcquirementRetryLimitError`,
1198
+ `RedisQueuedLocks::LockAcquirementTimeoutError`).
1132
1199
 
1133
- For more details see `lock_series` [docs](#lock_series---poc-acquire-a-series-of-locks)
1200
+ For more details see `lock_series` [docs](#lock_series---acquire-a-series-of-locks)
1134
1201
 
1135
1202
  ```ruby
1136
1203
  client.lock_series!("x", "y", "z") { ...some_code... }
@@ -1229,18 +1296,18 @@ rql.lock_info("your_lock_name")
1229
1296
 
1230
1297
  ```ruby
1231
1298
  # <for read/write locks>
1232
- # (see `read_write_mode:` kwarg attribute of #lock/#lock! methods and "Read/Write Locks" documentation section)
1299
+ # (see `rw_mode:` kwarg attribute of #lock/#lock! methods and "Read/Write Locks" documentation section)
1233
1300
  # (lock requests are not included: use #queue_info to see the queued readers and writers)
1234
1301
 
1235
1302
  # -> case 1: the lock is held by readers, writers and readers are waiting in the lock queues;
1236
1303
 
1237
1304
  # process#1 and process#2 (readers, work in parallel):
1238
- rql.lock("your_lock_name", read_write_mode: :read, ttl: 5_000, meta: { "kek" => "pek" }) { ... }
1239
- rql.lock("your_lock_name", read_write_mode: :read, ttl: 3_000) { ... }
1305
+ rql.lock("your_lock_name", rw_mode: :read, ttl: 5_000, meta: { "kek" => "pek" }) { ... }
1306
+ rql.lock("your_lock_name", rw_mode: :read, ttl: 3_000) { ... }
1240
1307
  # process#3 (writer, waits for all readers in the write lock queue):
1241
- rql.lock("your_lock_name", read_write_mode: :write) { ... }
1308
+ rql.lock("your_lock_name", rw_mode: :write) { ... }
1242
1309
  # process#4 (reader, waits for the earlier writer in the read lock queue):
1243
- rql.lock("your_lock_name", read_write_mode: :read) { ... }
1310
+ rql.lock("your_lock_name", rw_mode: :read) { ... }
1244
1311
 
1245
1312
  rql.lock_info("your_lock_name")
1246
1313
 
@@ -1445,7 +1512,7 @@ rql.unlock("your_lock_name")
1445
1512
 
1446
1513
  - releases the read lock of the current acquirer only (see [Read/Write Locks](#readwrite-locks)):
1447
1514
  other read locks, the write lock and lock requests are not affected (`#unlock` releases the lock entirely);
1448
- - suitable for read locks obtained without a block of code (`lock(..., read_write_mode: :read)` without a block);
1515
+ - suitable for read locks obtained without a block of code (`lock(..., rw_mode: :read)` without a block);
1449
1516
  - the current acquirer is calculated in the same way as for `#lock` (`#current_acquirer_id`: process, thread, fiber, ractor, identity),
1450
1517
  so call it from the same thread/fiber that obtained the read lock;
1451
1518
  - the read lock of a different acquirer can be released via `:acquirer_id` (for example, extracted from `#lock_info`/`#locks_info`);
@@ -1463,7 +1530,7 @@ rql.unlock("your_lock_name")
1463
1530
  - raises the `"redis_queued_locks.explicit_read_lock_release"` instrumentation event;
1464
1531
 
1465
1532
  ```ruby
1466
- rql.lock('report', read_write_mode: :read, ttl: 30_000) # => obtained without a block
1533
+ rql.lock('report', rw_mode: :read, ttl: 30_000) # => obtained without a block
1467
1534
  # ...
1468
1535
  rql.unlock_read('report')
1469
1536
  # =>
@@ -1779,7 +1846,7 @@ rql.release_locks_of(host_id: rql.current_host_id, acquirer_id: rql.current_acqu
1779
1846
  - `milliseconds` - (required) `[Integer]`
1780
1847
  - how many milliseconds should be added to the lock's TTL;
1781
1848
  - should be a positive `Integer`;
1782
- - `:read_write_mode` - (optional) `[Symbol]`
1849
+ - `:rw_mode` - (optional) `[Symbol]`
1783
1850
  - `:write` (default) - extends the write lock (the behavior described below);
1784
1851
  - `:read` - extends the read lock of the current acquirer only (see [Read/Write Locks](#readwrite-locks)):
1785
1852
  - the read lock expiration, the read locks registry TTL and the read lock data TTL are extended;
@@ -2774,19 +2841,19 @@ rql.lock("user_#{user.id}", priority: :high) { update_profile } # priority: 50
2774
2841
 
2775
2842
  <sup>\[[back to top](#table-of-contents)\]</sup>
2776
2843
 
2777
- - `read_write_mode:` attribute of `#lock` and `#lock!`:
2844
+ - `rw_mode:` attribute of `#lock` and `#lock!`:
2778
2845
  - `:write` (**default**) - exclusive lock (the classic RQL lock): waits for the write lock and for all read locks;
2779
2846
  - `:read` - shared lock: waits for the write lock only (read locks of different acquirers work in parallel);
2780
2847
 
2781
2848
  ```ruby
2782
2849
  # readers work in parallel
2783
- rql.lock('report', read_write_mode: :read) { read_report }
2850
+ rql.lock('report', rw_mode: :read) { read_report }
2784
2851
 
2785
2852
  # writer waits for all readers (and readers wait for the writer)
2786
- rql.lock('report', read_write_mode: :write) { rebuild_report }
2853
+ rql.lock('report', rw_mode: :write) { rebuild_report }
2787
2854
 
2788
2855
  # write mode is default:
2789
- rql.lock('report') { rebuild_report } # read_write_mode: :write by default
2856
+ rql.lock('report') { rebuild_report } # rw_mode: :write by default
2790
2857
  ```
2791
2858
 
2792
2859
  - lock request ordering (`access_strategy`):
@@ -2807,11 +2874,11 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
2807
2874
  - `#unlock` releases the lock entirely: the write lock, all read locks (with their data) and all lock requests;
2808
2875
  - `#unlock_read` (aka `#release_read_lock`) releases the read lock of the current acquirer
2809
2876
  (suitable for read locks obtained without a block), or the read lock of the acquirer passed via `acquirer_id:`;
2810
- - `#extend_lock_ttl(lock_name, milliseconds, read_write_mode: :read)` extends the read lock of the current acquirer only;
2877
+ - `#extend_lock_ttl(lock_name, milliseconds, rw_mode: :read)` extends the read lock of the current acquirer only;
2811
2878
  `all_read_locks: true` extends all live read locks of the lock;
2812
- - `#lock_series` with `read_write_mode: :read` obtains a series of read locks (the mode is the same for all locks of the series):
2879
+ - `#lock_series` with `rw_mode: :read` obtains a series of read locks (the mode is the same for all locks of the series):
2813
2880
  - read lock series of different acquirers work in parallel, writers wait for them;
2814
- - after the block (or when the series fails) only the read locks of the current acquirer are released;
2881
+ - after the block only the read locks of the current acquirer are released;
2815
2882
  - reentrant read locks of the series (already obtained by the outer logic of the same acquirer) are not released by the series;
2816
2883
  - reentrant locks (the same acquirer, see [Deadlocks and Reentrant locks](#deadlocks-and-reentrant-locks)):
2817
2884
  - read lock request under the obtained read lock or write lock is processed as a reentrant lock of the obtained lock
@@ -2931,12 +2998,21 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
2931
2998
  "[redis_queued_locks.try_lock.obtain__free_to_acquire]" # (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
2932
2999
  ```
2933
3000
 
2934
- - lock series logs (raised from `#lock_series`/`#lock_series!`, PoC; `rw_mode` is the mode of the series):
3001
+ - lock series logs (raised from `#lock_series`/`#lock_series!`; `rw_mode` is the mode of the series; with `log_lock_try` each
3002
+ attempt of the series writes the `try_lock.*` logs of the list above with its own `lock_key`, one log per lock of the series):
2935
3003
 
2936
3004
  ```ruby
2937
3005
  "[redis_queued_locks.start_lock_series_obtaining]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
2938
- "[redis_queued_locks.lock_series_obtained]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
3006
+ "[redis_queued_locks.start_try_to_lock_series_cycle]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
3007
+ "[redis_queued_locks.dead_score_reached__reset_series_position]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
3008
+ "[redis_queued_locks.lock_series_intermediate_lock_obtained]" # (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
3009
+ "[redis_queued_locks.lock_series_lock_obtained]" # (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
3010
+ "[redis_queued_locks.lock_series_lock_obtaining_failed]" # (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result");
3011
+ "[redis_queued_locks.lock_series_obtained]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "reentrant_lock_keys");
3012
+ "[redis_queued_locks.lock_series_obtaining_failed]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result", "failed_lock_key");
3013
+ "[redis_queued_locks.dequeue_from_lock_series_queues]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
2939
3014
  "[redis_queued_locks.expire_lock_series]" # (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
3015
+ "[redis_queued_locks.decrease_lock_series]" # (logs "lock_keys", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
2940
3016
  ```
2941
3017
 
2942
3018
  - operation logs (one log line per operation that changes Redis keys, with the time of the whole operation; the same data as
@@ -2988,8 +3064,16 @@ rql.lock('report') { rebuild_report } # read_write_mode: :write by default
2988
3064
  # - "[redis_queued_locks.expire_lock]" (logs "lock_key", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
2989
3065
  # - "[redis_queued_locks.decrease_lock]" (logs "lock_key", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode");
2990
3066
  # - "[redis_queued_locks.start_lock_series_obtaining]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
2991
- # - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
3067
+ # - "[redis_queued_locks.start_try_to_lock_series_cycle]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
3068
+ # - "[redis_queued_locks.dead_score_reached__reset_series_position]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
3069
+ # - "[redis_queued_locks.lock_series_intermediate_lock_obtained]" (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
3070
+ # - "[redis_queued_locks.lock_series_lock_obtained]" (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time");
3071
+ # - "[redis_queued_locks.lock_series_lock_obtaining_failed]" (logs "lock_key", "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result");
3072
+ # - "[redis_queued_locks.lock_series_obtained]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "reentrant_lock_keys");
3073
+ # - "[redis_queued_locks.lock_series_obtaining_failed]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority", "acq_time", "result", "failed_lock_key");
3074
+ # - "[redis_queued_locks.dequeue_from_lock_series_queues]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
2992
3075
  # - "[redis_queued_locks.expire_lock_series]" (logs "lock_keys", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
3076
+ # - "[redis_queued_locks.decrease_lock_series]" (logs "lock_keys", "decreased_ttl", "queue_ttl", "acq_id", "hst_id", "acs_strat", "rw_mode", "priority");
2993
3077
  # - "[redis_queued_locks.explicit_lock_release]" (logs "lock_key", "lock_key_queue", "rel_time", "at");
2994
3078
  # - "[redis_queued_locks.explicit_read_lock_release]" (logs "lock_key", "acq_id", "lock_res", "rel_time", "at");
2995
3079
  # - "[redis_queued_locks.explicit_all_locks_release]" (logs "at", "rel_time", "rel_key_cnt");
@@ -3131,8 +3215,12 @@ List of instrumentation events
3131
3215
  - ["redis_queued_locks.reentrant_lock_obtained"](#redis_queued_locksreentrant_lock_obtained)
3132
3216
  - ["redis_queued_locks.lock_hold_and_release"](#redis_queued_lockslock_hold_and_release)
3133
3217
  - ["redis_queued_locks.reentrant_lock_hold_completes"](#redis_queued_locksreentrant_lock_hold_completes)
3218
+ - ["redis_queued_locks.lock_series_intermediate_lock_obtained"](#redis_queued_lockslock_series_intermediate_lock_obtained)
3219
+ - ["redis_queued_locks.lock_series_lock_obtained"](#redis_queued_lockslock_series_lock_obtained)
3220
+ - ["redis_queued_locks.lock_series_lock_obtaining_failed"](#redis_queued_lockslock_series_lock_obtaining_failed)
3134
3221
  - ["redis_queued_locks.lock_series_obtained"](#redis_queued_lockslock_series_obtained)
3135
3222
  - ["redis_queued_locks.lock_series_hold_and_release"](#redis_queued_lockslock_series_hold_and_release)
3223
+ - ["redis_queued_locks.lock_series_obtaining_failed"](#redis_queued_lockslock_series_obtaining_failed)
3136
3224
  - ["redis_queued_locks.explicit_lock_release"](#redis_queued_locksexplicit_lock_release)
3137
3225
  - ["redis_queued_locks.explicit_read_lock_release"](#redis_queued_locksexplicit_read_lock_release)
3138
3226
  - ["redis_queued_locks.explicit_all_locks_release"](#redis_queued_locksexplicit_all_locks_release)
@@ -3142,8 +3230,8 @@ List of instrumentation events
3142
3230
  - ["redis_queued_locks.reindex_locks"](#redis_queued_locksreindex_locks)
3143
3231
 
3144
3232
  What is instrumented (each event has a log line with the same name, see [Logging](#logging)):
3145
- - the lock acquirement (`#lock`/`#lock!`, `#lock_series`/`#lock_series!`): each meaningful outcome (the lock is obtained,
3146
- not obtained, held and released);
3233
+ - the lock acquirement (`#lock`/`#lock!`, `#lock_series`/`#lock_series!`): each meaningful outcome (the lock or the whole
3234
+ series is obtained, not obtained, held and released);
3147
3235
  - each operation that changes Redis keys (`#unlock`, `#unlock_read`, `#clear_locks`, `#clear_locks_of`/`#clear_current_locks`,
3148
3236
  `#extend_lock_ttl`, `#clear_dead_requests`, `#reindex_locks`): one event with the time of the whole operation;
3149
3237
  - lock information queries (`#lock_info`, `#locks`, `#queues`, `#keys`, ...) are not instrumented;
@@ -3169,8 +3257,7 @@ Detalized event semantics and payload structure:
3169
3257
  ##### `"redis_queued_locks.lock_obtaining_failed"`
3170
3258
  - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3171
3259
  - a moment when the lock acquirement is failed (the lock is not obtained);
3172
- - raised from `#lock`/`#lock!` (with `raise_errors: true` - before the error is raised) and from `#lock_series`/`#lock_series!`
3173
- (for the lock of the series that is not obtained);
3260
+ - raised from `#lock`/`#lock!` (with `raise_errors: true` - before the error is raised);
3174
3261
  - payload:
3175
3262
  - `:lock_key` - `string` - lock name;
3176
3263
  - `:rw_mode` - `symbol` - requested lock mode: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
@@ -3250,36 +3337,110 @@ Detalized event semantics and payload structure:
3250
3337
  - `:acq_time` - `float`/`milliseconds` - time spent on lock acquiring;
3251
3338
  - `:instrument` - `nil`/`Any` - custom data passed to the `#lock`/`#lock!` method as `:instrument` attribute;
3252
3339
 
3340
+ ##### `"redis_queued_locks.lock_series_intermediate_lock_obtained"`
3341
+ - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3342
+ - intermediate obtaining: a moment when the lock of the series passed all checks of an attempt of the series (the lock is free and the lock
3343
+ request of the series is next). Nothing is held yet: the series can still wait for its other locks or fail;
3344
+ - raised from `#lock_series`/`#lock_series!` once per lock of the series (not per attempt), in the order of the lock names up to the lock
3345
+ that blocks the series; reentrant locks of the series are not reported;
3346
+ - payload:
3347
+ - `:lock_key` - `string` - the lock of the series;
3348
+ - `:lock_keys` - `array<string>` - all locks of the series;
3349
+ - `:rw_mode` - `symbol` - the mode of the series: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3350
+ - `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
3351
+ - `:ttl` - `integer`/`milliseconds` - the ttl of each lock of the series;
3352
+ - `:acq_id` - `string` - lock acquirer identifier;
3353
+ - `:hst_id` - `string` - lock's host identifier;
3354
+ - `:ts` - `float`/`epoch` - the time of the event;
3355
+ - `:acq_time` - `float`/`milliseconds` - time spent on the series acquiring up to this moment;
3356
+ - `:instrument` - `nil`/`Any` - custom data passed to the `#lock_series`/`#lock_series!` method as `:instrument` attribute;
3357
+
3358
+ ##### `"redis_queued_locks.lock_series_lock_obtained"`
3359
+ - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3360
+ - full obtaining: a moment when the lock of the series was really obtained (all locks of the series are held);
3361
+ - raised from `#lock_series`/`#lock_series!` for each lock obtained by the series, right before `lock_series_obtained`; reentrant locks of
3362
+ the series are not reported (see `:reentrant_lock_keys` of `lock_series_obtained`);
3363
+ - payload:
3364
+ - `:lock_key` - `string` - the lock of the series;
3365
+ - `:lock_keys` - `array<string>` - all locks of the series;
3366
+ - `:rw_mode` - `symbol` - the mode of the series: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3367
+ - `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
3368
+ - `:ttl` - `integer`/`milliseconds` - the ttl of each lock of the series;
3369
+ - `:acq_id` - `string` - lock acquirer identifier;
3370
+ - `:hst_id` - `string` - lock's host identifier;
3371
+ - `:ts` - `float`/`epoch` - the time when the series was obtained;
3372
+ - `:acq_time` - `float`/`milliseconds` - time spent on the series acquiring;
3373
+ - `:instrument` - `nil`/`Any` - custom data passed to the `#lock_series`/`#lock_series!` method as `:instrument` attribute;
3374
+
3375
+ ##### `"redis_queued_locks.lock_series_lock_obtaining_failed"`
3376
+ - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3377
+ - a moment when the series acquirement is failed: the lock of the series that blocked it;
3378
+ - raised from `#lock_series`/`#lock_series!` once per failed series (with `raise_errors: true` - before the error is raised), right before
3379
+ `lock_series_obtaining_failed`; it is not raised when the blocking lock is unknown (no attempt found it);
3380
+ - payload:
3381
+ - `:lock_key` - `string` - the lock that blocked the series;
3382
+ - `:lock_keys` - `array<string>` - all locks of the series;
3383
+ - `:rw_mode` - `symbol` - the mode of the series: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3384
+ - `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
3385
+ - `:ttl` - `integer`/`milliseconds` - the ttl of each lock of the series;
3386
+ - `:acq_id` - `string` - lock acquirer identifier;
3387
+ - `:hst_id` - `string` - lock's host identifier;
3388
+ - `:ts` - `float`/`epoch` - the time when the series acquirement was failed;
3389
+ - `:acq_time` - `float`/`milliseconds` - time spent on the series acquiring;
3390
+ - `:result` - `symbol` - the failure result (see `lock_series_obtaining_failed`);
3391
+ - `:instrument` - `nil`/`Any` - custom data passed to the `#lock_series`/`#lock_series!` method as `:instrument` attribute;
3392
+
3253
3393
  ##### `"redis_queued_locks.lock_series_obtained"`
3254
3394
  - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3255
- - (PoC) a moment when all locks of the series were obtained;
3395
+ - a moment when all locks of the series were obtained;
3256
3396
  - raised from `#lock_series`/`#lock_series!`;
3257
3397
  - payload:
3258
3398
  - `:lock_keys` - `array<string>` - lock names of the series;
3259
- - `:rw_mode` - `symbol` - the mode of the series (`read_write_mode`): `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3399
+ - `:rw_mode` - `symbol` - the mode of the series: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3260
3400
  - `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
3261
- - `:ttl` - `integer`/`milliseconds` - the `ttl` option of the series (each lock of the series gets its own ttl,
3262
- see [lock_series](#lock_series---poc-acquire-a-series-of-locks));
3401
+ - `:ttl` - `integer`/`milliseconds` - the ttl of each lock of the series (see [lock_series](#lock_series---acquire-a-series-of-locks));
3263
3402
  - `:acq_id` - `string` - lock acquirer identifier;
3264
3403
  - `:hst_id` - `string` - lock's host identifier;
3265
3404
  - `:ts` - `numeric`/`epoch` - the time when the series was obtained;
3266
3405
  - `:acq_time` - `float`/`milliseconds` - time spent on the series acquiring;
3406
+ - `:reentrant_lock_keys` - `array<string>` - the locks of the series that are obtained by your outer logic (reentrant locks:
3407
+ they are not obtained and not released by the series);
3267
3408
  - `:instrument` - `nil`/`Any` - custom data passed to the `#lock_series`/`#lock_series!` method as `:instrument` attribute;
3268
3409
 
3269
3410
  ##### `"redis_queued_locks.lock_series_hold_and_release"`
3270
3411
  - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3271
- - (PoC) an event signalizes about the "hold+and+release" process of the series is finished;
3412
+ - an event signalizes about the "hold+and+release" process of the series is finished;
3272
3413
  - raised from `#lock_series`/`#lock_series!` when invoked with a block of code;
3273
3414
  - payload:
3274
3415
  - `:lock_keys` - `array<string>` - lock names of the series;
3275
- - `:rw_mode` - `symbol` - the mode of the series (`read_write_mode`): `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3416
+ - `:rw_mode` - `symbol` - the mode of the series: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3276
3417
  - `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
3277
3418
  - `:hold_time` - `float`/`milliseconds` - the series hold time;
3278
- - `:ttl` - `integer`/`milliseconds` - the `ttl` option of the series;
3419
+ - `:ttl` - `integer`/`milliseconds` - the ttl of each lock of the series;
3279
3420
  - `:acq_id` - `string` - lock acquirer identifier;
3280
3421
  - `:hst_id` - `string` - lock's host identifier;
3281
3422
  - `:ts` - `float`/`epoch` - the time when the series was released;
3282
3423
  - `:acq_time` - `float`/`milliseconds` - time spent on the series acquiring;
3424
+ - `:reentrant_lock_keys` - `array<string>` - the locks of the series that are obtained by your outer logic (reentrant locks:
3425
+ they are not obtained and not released by the series);
3426
+ - `:instrument` - `nil`/`Any` - custom data passed to the `#lock_series`/`#lock_series!` method as `:instrument` attribute;
3427
+
3428
+ ##### `"redis_queued_locks.lock_series_obtaining_failed"`
3429
+ - <sup>\[[back to the list](#instrumentation-events)\]</sup>
3430
+ - a moment when the series acquirement is failed (no lock of the series is obtained);
3431
+ - raised from `#lock_series`/`#lock_series!` (with `raise_errors: true` - before the error is raised);
3432
+ - payload:
3433
+ - `:lock_keys` - `array<string>` - lock names of the series;
3434
+ - `:rw_mode` - `symbol` - the mode of the series: `:write` or `:read` (see [Read/Write Locks](#readwrite-locks));
3435
+ - `:priority` - `integer` - the lock request priority of the series (`priority`): from `-100` to `100` (a priority name is reported as its integer priority, see [Lock Request Prioritization](#lock-request-prioritization));
3436
+ - `:ttl` - `integer`/`milliseconds` - the ttl of each lock of the series;
3437
+ - `:acq_id` - `string` - lock acquirer identifier;
3438
+ - `:hst_id` - `string` - lock's host identifier;
3439
+ - `:ts` - `float`/`epoch` - the time when the series acquirement was failed;
3440
+ - `:acq_time` - `float`/`milliseconds` - time spent on the series acquiring;
3441
+ - `:result` - `symbol` - the failure result: `:timeout_reached`, `:retry_limit_reached`, `:fail_fast_no_try`, `:fail_fast_after_try`, `:conflict_dead_lock`, `:conflict_lock_upgrade`;
3442
+ - `:failed_lock_key` - `string`/`nil` - the first lock that blocked the last completed attempt of the series (`nil`: unknown:
3443
+ no attempt was completed or some lock state was changed during the attempt);
3283
3444
  - `:instrument` - `nil`/`Any` - custom data passed to the `#lock_series`/`#lock_series!` method as `:instrument` attribute;
3284
3445
 
3285
3446
  ##### `"redis_queued_locks.explicit_lock_release"`
@@ -3372,6 +3533,12 @@ Detalized event semantics and payload structure:
3372
3533
  <sup>\[[back to top](#table-of-contents)\]</sup>
3373
3534
 
3374
3535
  - **Major**:
3536
+ - **Experimental feature**: (non-`timed` locks): per-ruby-block-holding-the-lock sidecar `Ractor` and `in progress queue` in RedisDB that will extend
3537
+ the acquired lock for long-running blocks of code (that invoked "under" the lock
3538
+ whose ttl may expire before the block execution completes). It makes sense for non-`timed` locks *only*;
3539
+ - think about "thread priority" configuration :thinking:;
3540
+ - add `hst_id` to all methods that works with queues info;
3541
+ - try to return the `fiber object id` to the lock host identifier (we cant use fiber object id cuz `ObjectSpace` has no access to the fiber object space after the any ractor object initialization)
3375
3542
  - Swarm:
3376
3543
  - circuit-breaker for long-living failures of your infrastructure inside the swarm elements and supervisor:
3377
3544
  the supervisor will stop (for some period of time or while the some factor will return `true`)
@@ -3382,11 +3549,6 @@ Detalized event semantics and payload structure:
3382
3549
  the global "timeout request" queue and managed by a single global "timeout wathcer" thread). it should prevent any logic and timeout intersections,
3383
3550
  some GVL-related things and problem situations when the global watcher thread is "dead";
3384
3551
  - **strict redlock algorithm support** (support for many `RedisClient` instances that are fully independent (distributed redis instances));
3385
- - #lock_series` - acquire a series of locks:
3386
- - (NOTE: "Proof of Concept" implementation is ready to use);
3387
- ```ruby
3388
- rql.lock_series('lock_a', 'lock_b', 'lock_c') { puts 'locked' }
3389
- ```
3390
3552
  - an ability to release all locks and all requests of the concrete acquirer id or host id (or both in validation-orianted combination);
3391
3553
  - detailed lock informotion inside the error object in cases of exceptions (at the moment we have this info inside the error message only that hard to analyze in work);
3392
3554
  - a convenient way to mark any `lock` invocation as "non-instrumentable" / "non-loggable" (as an alternative to `VoidNotifier` and to `VoidLogger`);
@@ -3402,9 +3564,6 @@ Detalized event semantics and payload structure:
3402
3564
  - an ability to return all insturmentation metrics from the `lock` invocation (and after block `yield`ing);
3403
3565
  - think about a PG-like `stats`-data/table, that should be helpful for lock history acquirement analyzation :thinking: (suitable for cases when we want
3404
3566
  to check if the lock was acquired by some different parallel process durign the some period of time in the past or not);
3405
- - think about "thread priority" configuration :thinking:;
3406
- - add `hst_id` to all methods that works with queues info;
3407
- - try to return the `fiber object id` to the lock host identifier (we cant use fiber object id cuz `ObjectSpace` has no access to the fiber object space after the any ractor object initialization)
3408
3567
  - named RQL's threads (`Thread#name`) and RQL's ractors (`Ractor#name`) in order to have an ability to find and work with RQL's threads and ractors outside of RQL logic (stop threads before process forking, for example);
3409
3568
  - `#lock`/`#lock!` - `timeout:` option: support for granular periods (it supports only `seconds` at the moment, but we need `milliseconds`);
3410
3569
  - Add `after timeout hook` ability for timeout errors with an ability to access the binding context (if it possibly) of the thread where it failed;
@@ -3416,9 +3575,6 @@ Detalized event semantics and payload structure:
3416
3575
  - support for `Dragonfly` database backend (https://github.com/dragonflydb/dragonfly) (https://www.dragonflydb.io/);
3417
3576
  - (_research_) GVL-isolation for lock acquirement logic (try to bypass Ruby's "context-switching" in order to prevent any time-based affects on lock-acquirement logic);
3418
3577
  - Semantic error objects for unexpected Redis errors;
3419
- - **Experimental feature**: (non-`timed` locks): per-ruby-block-holding-the-lock sidecar `Ractor` and `in progress queue` in RedisDB that will extend
3420
- the acquired lock for long-running blocks of code (that invoked "under" the lock
3421
- whose ttl may expire before the block execution completes). It makes sense for non-`timed` locks *only*;
3422
3578
  - sized lock queues (with an ability of dynamically growing size);
3423
3579
  - better code stylization (+ some refactorings);
3424
3580
  - `RedisQueuedLocks::Acquirer::Try.try_to_lock` - detailed successful result analization;
@@ -3435,7 +3591,7 @@ Detalized event semantics and payload structure:
3435
3591
  - add a library-level exception, when RQL-related key in Redis (required for its logic) has incompatible type (means: some other program uses our key with their own type and logic and RQL can't work properly);
3436
3592
  - yardoc docs with CI check (full doc coverage check);
3437
3593
  - split *exception* inheritance to the groups: `lock obtaining errors`, `block invocation errors`, `swarm errors`, and other groups (research possible groups):
3438
- - in some cases we need to intercept "Lock Obtaining Process Erros", in other cases: "Block Invocation Errors", and so on (in `Lock Serirs PoC` for example);
3594
+ - in some cases we need to intercept "Lock Obtaining Process Erros", in other cases: "Block Invocation Errors", and so on;
3439
3595
  - think about `#debounce` and `#throttle` mechanisms;
3440
3596
 
3441
3597
  ---