redis_queued_locks 1.21.0 → 1.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (78) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/project-overview.md +294 -51
  3. data/.claude/rules/acquirer.md +16 -11
  4. data/.claude/rules/arguments.md +2 -2
  5. data/.claude/rules/changelog.md +27 -0
  6. data/.claude/rules/git.md +1 -1
  7. data/.claude/rules/logic.md +14 -15
  8. data/.claude/rules/swarm.md +3 -4
  9. data/.claude/rules/tests.md +9 -9
  10. data/.claude/rules/type-checking.md +14 -16
  11. data/.claude/rules/visitors.md +11 -9
  12. data/.claude/skills/rql-code-review/SKILL.md +129 -47
  13. data/.ruby-version +1 -1
  14. data/CHANGELOG.md +122 -245
  15. data/CLAUDE.md +12 -10
  16. data/README.md +516 -175
  17. data/Rakefile +11 -0
  18. data/lib/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rb +4 -4
  19. data/lib/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rb +28 -12
  20. data/lib/redis_queued_locks/acquirer/acquire_lock/log_visitor.rb +38 -16
  21. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rb +92 -15
  22. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +945 -272
  23. data/lib/redis_queued_locks/acquirer/acquire_lock.rb +82 -32
  24. data/lib/redis_queued_locks/acquirer/clear_dead_requests.rb +240 -45
  25. data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +15 -13
  26. data/lib/redis_queued_locks/acquirer/keys.rb +6 -2
  27. data/lib/redis_queued_locks/acquirer/lock_series/instr_visitor.rb +243 -0
  28. data/lib/redis_queued_locks/acquirer/lock_series/log_visitor.rb +472 -0
  29. data/lib/redis_queued_locks/acquirer/lock_series/release_lock_series.rb +181 -0
  30. data/lib/redis_queued_locks/acquirer/lock_series/try_to_lock_series.rb +1051 -0
  31. data/lib/redis_queued_locks/acquirer/lock_series.rb +948 -0
  32. data/lib/redis_queued_locks/acquirer/locks.rb +6 -2
  33. data/lib/redis_queued_locks/acquirer/queue_info.rb +30 -7
  34. data/lib/redis_queued_locks/acquirer/queues.rb +17 -6
  35. data/lib/redis_queued_locks/acquirer/release_all_locks.rb +6 -2
  36. data/lib/redis_queued_locks/acquirer/release_locks_of.rb +6 -2
  37. data/lib/redis_queued_locks/acquirer.rb +1 -1
  38. data/lib/redis_queued_locks/client.rb +116 -44
  39. data/lib/redis_queued_locks/config.rb +93 -1
  40. data/lib/redis_queued_locks/resource.rb +124 -9
  41. data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +6 -5
  42. data/lib/redis_queued_locks/swarm/zombie_info.rb +6 -2
  43. data/lib/redis_queued_locks/version.rb +2 -2
  44. data/rbs_collection.lock.yaml +4 -0
  45. data/rbs_collection.yaml +2 -1
  46. data/redis_queued_locks.gemspec +21 -14
  47. data/sig/manifest.yml +1 -0
  48. data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue/log_visitor.rbs +2 -2
  49. data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rbs +2 -2
  50. data/sig/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rbs +12 -6
  51. data/sig/redis_queued_locks/acquirer/acquire_lock/log_visitor.rbs +21 -14
  52. data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rbs +61 -40
  53. data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +160 -11
  54. data/sig/redis_queued_locks/acquirer/acquire_lock/{yield_with_expire → yield_expire}/log_visitor.rbs +4 -4
  55. data/sig/redis_queued_locks/acquirer/acquire_lock/yield_expire.rbs +2 -2
  56. data/sig/redis_queued_locks/acquirer/acquire_lock.rbs +5 -3
  57. data/sig/redis_queued_locks/acquirer/clear_dead_requests.rbs +14 -1
  58. data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +2 -2
  59. data/sig/redis_queued_locks/acquirer/lock_series/instr_visitor.rbs +102 -0
  60. data/sig/redis_queued_locks/acquirer/lock_series/log_visitor.rbs +154 -0
  61. data/sig/redis_queued_locks/acquirer/lock_series/release_lock_series.rbs +37 -0
  62. data/sig/redis_queued_locks/acquirer/lock_series/try_to_lock_series.rbs +198 -0
  63. data/sig/redis_queued_locks/acquirer/lock_series.rbs +106 -0
  64. data/sig/redis_queued_locks/acquirer/queues.rbs +1 -1
  65. data/sig/redis_queued_locks/client.rbs +77 -7
  66. data/sig/redis_queued_locks/config.rbs +54 -1
  67. data/sig/redis_queued_locks/instrument/sampler.rbs +3 -1
  68. data/sig/redis_queued_locks/instrument/void_notifier.rbs +3 -1
  69. data/sig/redis_queued_locks/logging/sampler.rbs +3 -1
  70. data/sig/redis_queued_locks/resource.rbs +22 -12
  71. data/sig/redis_queued_locks/swarm/swarm_element/threaded.rbs +1 -1
  72. data/sig/redis_queued_locks/utilities.rbs +2 -1
  73. data/sig/redis_queued_locks.rbs +7 -0
  74. metadata +35 -13
  75. data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +0 -56
  76. data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +0 -95
  77. data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +0 -484
  78. /data/sig/redis_queued_locks/{acquier.rbs → acquirer.rbs} +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e7aeb9d0b5ea57783c70a285b753616abf277955029184236ed7746f127148e4
4
- data.tar.gz: 9a5771fd5f0df1edad7eb6270a211576fbcd33f4049a5fb040a65e334f92a32f
3
+ metadata.gz: 0c06bae2bc7f0b6164847caa6b7adb5c77a7aab5a85dc5f3ef8c45ac19b8ed80
4
+ data.tar.gz: 6a73267ecd535b74c98f489066ee5cc6f2a8c2ea261eab6d5b5f6b84e4748585
5
5
  SHA512:
6
- metadata.gz: 2c2328b3187ce087a694d3169aaaf9362ffb48557e626ab32933ffa4b70e015121edababddaeb3d2e24df7bfcb2c77980c54f1ed315c7a08d2d41e23e631351e
7
- data.tar.gz: 4a380d09471c48bc00ef7454e5755420738c5f1233c576d10d115c6ca973ecb505ce77b037d805f0299964da1eae31f02dd5999e2e1945cfef6aa5508e3b7116
6
+ metadata.gz: b3f34d6976337bcdfe4732b3c58034ed455922450f98758a0f4c90e36a885721d847e648c5dfa64357aac9b1b401138787649bda0491693777581669f932a187
7
+ data.tar.gz: 7d4e8769653ddfaf0f483a413c08ba10ea15d4f4d324e57dee1009e27222f7c2473789b0221380fc68a0047a5383083761963ed541179f7ab633876f68b3e891
@@ -1,13 +1,13 @@
1
1
  # redis_queued_locks: Project Overview
2
2
 
3
3
  Ruby gem providing **distributed locks on Redis with a prioritized lock-acquisition queue**.
4
- Each lock has its own request queue; requests are processed FIFO (`:queued`) or first-come
5
- (`:random`). Requests have TTLs (with requeue), so queues never get stuck. Includes reentrancy
4
+ Each lock has its own request queue; requests are processed FIFO (`:queued`, ordered by the lock request
5
+ priority first: `priority` -100..100) or first-come (`:random`). Requests have TTLs (with requeue), so queues never get stuck. Includes reentrancy
6
6
  (conflict) strategies, timed locks, retries/fail-fast, logging, instrumentation with sampling,
7
7
  and a background "swarm" that detects dead hosts and flushes their locks.
8
8
 
9
9
  - Repo: https://github.com/0exp/redis_queued_locks (MIT); main branch `master`
10
- - Runtime dependency: `redis-client ~> 0.20` (locked 0.30.1), nothing else
10
+ - Runtime dependency: `redis-client ~> 0.20` (locked 0.31.0), nothing else
11
11
  - Ruby `>= 4.0` (`.ruby-version` = 4.0.7)
12
12
 
13
13
  ---
@@ -25,6 +25,8 @@ Client (client.rb) public API facade
25
25
  │
26
26
  ▼
27
27
  RedisClient (redis-client gem) WATCH/MULTI, Lua (EVAL), ZSET, HASH, SCAN
28
+
29
+ Resource (resource.rb) keys, acquirer/host ids, queue score layout (priority bands)
28
30
  ```
29
31
 
30
32
  `Client.new(redis_client) { |config| ... }`:
@@ -35,28 +37,129 @@ Client (client.rb) public API facade
35
37
  Every other public method is a thin wrapper: it fills option defaults from `config[...]` and calls
36
38
  an `Acquirer::*` module function with `redis_client`.
37
39
 
38
- Operation modules are independent of each other (all `Acquirer::*` except the `AcquireLock` core
39
- and PoC modules such as `LockSeriesPoC`, which may reuse other modules and `AcquireLock`): no
40
+ Operation modules are independent of each other (all `Acquirer::*` except the lock acquisition core:
41
+ `AcquireLock` and `LockSeries`, which reuses the `AcquireLock` mixins): no
40
42
  cross-calls and no shared RBS types; common logic lives in `Resource`/`Utilities`, similar logic is
41
43
  duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatting, `Queues` the
42
44
  `QueueInfo` request formatting). Known debt: `AcquireLock::WithAcqTimeout` calls
43
45
  `LockInfo`/`QueueInfo` for detailed timeout errors.
44
46
 
47
+ ### Read/Write locks in the architecture
48
+
49
+ `rw_mode: :write` (default) is the exclusive classic lock (its keys and data are unchanged); `:read` is a shared
50
+ lock: readers share the lock with each other, a writer waits for the write lock and for all live readers, a reader waits
51
+ for the write lock only. Mutual exclusion comes only from WATCH/MULTI on the lock state (no Lua in the RW path); the
52
+ request queues only order the requests (FIFO between modes by `(score, acq_id)`).
53
+
54
+ Keys of one lock name (all of them match `KEY_PATTERN`, classified by `Resource.lock_key_family`):
55
+
56
+ ```
57
+ rql:lock:<name> HASH write lock: owner + data write lock state
58
+ rql:lock_readers:<name> ZSET readers registry: acq_id => expiry (Redis ms) read lock state (the source of truth)
59
+ rql:lock_reader:<name>:<acq_id> HASH reader data (the write lock format without ids: a data carrier only)
60
+ rql:lock_queue:<name> ZSET write lock requests ordering only
61
+ rql:lock_read_queue:<name> ZSET read lock requests (the same scores) ordering only
62
+ ```
63
+
64
+ The path of the mode through the components:
65
+
66
+ ```
67
+ Client#lock / #lock! (rw_mode:) #lock_series / #lock_series! (one mode per series),
68
+ │ #unlock_read / #release_read_lock, #extend_lock_ttl(rw_mode:,
69
+ │ all_read_locks:)
70
+ ▼
71
+ AcquireLock Step 0.1 (RW) validates the mode; the keys are computed once (read_lock_key: readers only)
72
+ ▼
73
+ TryToLock (each attempt and pre-check) WATCH: writers [lock_key, lock_readers_key], readers [lock_key] only
74
+ │ (readers never abort each other); one state pipeline: HGET acq_id,
75
+ │ TIME, own reader score, longest reader (writers); the conditions:
76
+ │ write = no write lock + no live reader (score > Redis now),
77
+ │ read = no write lock; the order: write = head of the write queue +
78
+ │ no earlier read request, read = no earlier write request; the
79
+ │ obtaining MULTI: write HSET + PEXPIRE, read ZADD expiry + registry
80
+ │ PEXPIRE NX + GT + reader data; same-acquirer conflicts by
81
+ │ conflict_strategy (held mode `sp_conflict_rw_mode`; read->write
82
+ │ upgrade never works through)
83
+ ▼
84
+ YieldExpire releases the held mode (`held_rw_mode`): write EXPIRE 0, read ZREM +
85
+ DEL of the own reader data (owner-safe)
86
+ ReleaseLock (unlock) write lock + readers + both queues in one MULTI, then one DEL of the
87
+ released readers' data (only when readers existed)
88
+ ReleaseReadLock / ExtendLockTTL one read lock (own or `acquirer_id:`) / own or all live read locks
89
+ Info, iteration, cleanup, swarm the read family everywhere: lock_info readers, queue_info read_queue,
90
+ locks/queues/keys, clear_*, clear_dead_requests, zombies
91
+ LogVisitor / InstrVisitor (+ series, try-lock) rw_mode in every lock log/event (requested; YieldExpire: held; series)
92
+ ```
93
+
94
+ - Invariants for any lock-touching code: cover the read family next to `rql:lock:*`/`rql:lock_queue:*`; writers WATCH
95
+ the registry, readers never; reader liveness only by the Redis-time score (`score > now`), never `ZCARD`/`EXISTS`;
96
+ registry and reader data TTLs keep the max with `PEXPIRE NX` + `PEXPIRE GT`; write-only result shapes stay unchanged
97
+ (read keys/fields only when read data exists); the lock index covers the read keys (enabled mode).
98
+ - Details (reentrancy matrix, fail_fast, results, release semantics, design decisions): "Read/Write locks" below; the
99
+ public description: README "Read/Write Locks".
100
+
101
+ ### Lock request prioritization in the architecture
102
+
103
+ The lock request priority orders the waiting requests of a lock (`:queued`): strict priority between priorities, FIFO
104
+ inside a priority, in both read/write modes. It is carried by the queue score only (no new keys, fields or commands per
105
+ request) and never takes part in mutual exclusion (WATCH/MULTI on the lock state decides it, as before). The path of the
106
+ priority through the components:
107
+
108
+ ```
109
+ Client#lock / #lock! / #lock_series / #lock_series! priority: (Integer or a name), default config['default_lock_priority'];
110
+ │ lock_priorities: config['lock_priorities'] (internal keyword)
111
+ ▼
112
+ Config (config.rb) lock_priorities (name => Integer) + default_lock_priority validators,
113
+ │ the default name checked against the names (configure / []=)
114
+ │
115
+ ▼
116
+ AcquireLock Step 0.1 / LockSeries Step 0.1 name -> Integer, range -100..100 (ArgumentError); Integer only below
117
+ │
118
+ ├──▶ Resource the single source of the score layout:
119
+ │ calc_initial_acquirer_position(priority) score = enqueue time - priority * 2**32 (one band per priority)
120
+ │ dead_score_reached?(position, priority, ttl) requeue check (decodes the enqueue time)
121
+ │ lock_priority_band_floor / _dead_score the bounds of the dead requests of a band
122
+ │ lock_request_priority(score) the priority decoded from a score
123
+ │
124
+ ├──▶ TryToLock (each attempt and pre-check) queueing: the own band pruned in both queues + ZADD NX with the
125
+ │ shifted score (queue_lock_request, no extra round trip); Step 2.2:
126
+ │ dead blocking requests of other bands dropped with their bands
127
+ │ (drop_dead_requests_ahead, dead_request_priority: one round trip
128
+ │ per removal, none while nothing dead blocks); ordering: queue heads
129
+ │ and the cross-mode (score, acq_id) comparison on shifted scores
130
+ │
131
+ ├──▶ ClearDeadRequests per-band cleanup of the bands found in the queues (<= 2 round trips/batch;
132
+ │ by_priority: a walk over the first request of each band, the most
133
+ │ bands of a batch queue + 1 round trips, no full queue read)
134
+ ├──▶ QueueInfo / Queues 'priority' of each request decoded from its score
135
+ └──▶ LogVisitor / InstrVisitor (+ series, try-lock) priority in every lock acquirement log/event; try-lock logs decode
136
+ the priorities of the requests ahead (first/ahead/dead)
137
+ ```
138
+
139
+ - Unchanged by prioritization: the lock keys and their data, queue members (`acq_id`), dequeue (`ZREM`), `unlock` /
140
+ `clear_locks` (full queue clears), the swarm (hosts and zombies come from acquirer ids), the lock index, the obtaining
141
+ transactions.
142
+ - Invariants for any code that touches lock queues: never prune a queue with one `-inf..now-ttl` range (prune bands with
143
+ the `Resource` bounds); never enumerate the priority range per attempt or per queue (decode the bands from the data);
144
+ read priorities only through `Resource.lock_request_priority`; the attempt cost must not depend on the priority range.
145
+ - Details (encoding, liveness, cleanup, limitations, mixed versions): "Lock request priorities" below; the public
146
+ description: README "Lock Request Prioritization".
147
+
45
148
  ### Client → implementation map
46
149
 
47
150
  | Client method | Implementation |
48
151
  |---|---|
49
152
  | `lock` / `lock!` | `acquirer/acquire_lock.rb` + `acquirer/acquire_lock/*` |
50
- | `lock_series` / `lock_series!` | `acquirer/lock_series_poc.rb` (proof of concept; one `read_write_mode` for the whole series) |
153
+ | `lock_series` / `lock_series!` | `acquirer/lock_series.rb` + `acquirer/lock_series/*` (atomic, see "Lock series"; one `rw_mode`, `priority` and `ttl` for the whole series) |
51
154
  | `unlock` | `acquirer/release_lock.rb` |
52
155
  | `unlock_read` / `release_read_lock` | `acquirer/release_read_lock.rb` (own read lock or `acquirer_id:` read lock only) |
53
- | `extend_lock_ttl` | `acquirer/extend_lock_ttl.rb` (`read_write_mode: :read` extends own read lock, `+ all_read_locks: true` all live read locks) |
156
+ | `extend_lock_ttl` | `acquirer/extend_lock_ttl.rb` (`rw_mode: :read` extends own read lock, `+ all_read_locks: true` all live read locks) |
54
157
  | `locked?` / `queued?` | `acquirer/is_locked.rb` / `acquirer/is_queued.rb` |
55
158
  | `lock_info` / `queue_info` | `acquirer/lock_info.rb` / `acquirer/queue_info.rb` |
56
159
  | `locks`, `locks_info`, `queues`, `queues_info`, `keys` | `acquirer/locks.rb`, `queues.rb`, `keys.rb` (keyspace SCAN or lock index: `lock_index_enabled`) |
57
160
  | `clear_locks` | `acquirer/release_all_locks.rb` |
58
161
  | `clear_locks_of`, `clear_current_locks` | `acquirer/release_locks_of.rb` |
59
- | `clear_dead_requests` | `acquirer/clear_dead_requests.rb` |
162
+ | `clear_dead_requests` | `acquirer/clear_dead_requests.rb` (`by_priority: true` reads the first request of each priority band instead of all requests) |
60
163
  | `reindex_locks` | `acquirer/reindex_locks.rb` (adds keys without lock index entries to the lock index: one SCAN pass; works in both modes) |
61
164
  | `current_acquirer_id`, `current_host_id`, `possible_host_ids` | `resource.rb` |
62
165
  | `swarmize!`, `deswarmize!`, `swarm_status`, `swarm_info`, `probe_hosts`, `flush_zombies`, `zombie_locks`, `zombie_acquirers`, `zombie_hosts`, `zombies_info` | `swarm.rb` + `swarm/*` |
@@ -67,27 +170,81 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
67
170
 
68
171
  | Mixin | Role |
69
172
  |---|---|
70
- | `TryToLock` (`try_to_lock.rb`) | One attempt: `multi(watch:)` (write: lock key + readers registry, read: lock key only), one pipelined read of the lock state (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only), the lock request queueing in one pipelined round trip (`queue_lock_request`: `ZADD NX` into the queue of the requested mode (timestamp score), prune both queues, both queue heads, then (the enabled lock index only) the lock index commands: request queue `PEXPIRE NX`+`GT` (queue ttl), `ZADD GT` of the lock name into the lock index (queue ttl), `ZREMRANGEBYSCORE` of expired lock names), take the lock if allowed (see Read/Write locks; with the enabled lock index the obtaining `MULTI` ends with the lock index `ZADD GT` (lock ttl), the `:extendable_work_through` extension `MULTI` with `ZADD XX INCR`). Eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends the queueing commands in the lock state round trip: `lock {}` = 4 round trips (`WATCH`, state + queueing, `MULTI/EXEC`, release), a failed attempt = 3. Pre-check (`precheck_lock`, retries and `fail_fast` attempts): the lock state (+ the queueing for retries) without `WATCH` in one round trip; an attempt that can not obtain the lock fails there (same results and log events: `lock_obstacle`/`log_lock_obstacle`), otherwise the attempt continues with `WATCH`/`MULTI` (a failed retry = 1 round trip, a successful one = 4; `fail_fast` on an obtained lock = 1); same-process conflicts are never decided by the pre-check. Handles reentrant conflicts (incl. read/write ones); write TTL extension via inline Lua (`PTTL` + `PEXPIRE`). |
173
+ | `TryToLock` (`try_to_lock.rb`) | One attempt: `multi(watch:)` (write: lock key + readers registry, read: lock key only), one pipelined read of the lock state (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only), the lock request queueing in one pipelined round trip (`queue_lock_request`: prune the priority band of the request in both queues (first: the old dead position of a requeued request is dropped before `ZADD NX` adds its new position), `ZADD NX` into the queue of the requested mode (timestamp score shifted by the priority band, see "Lock request priorities"), both queue heads with scores, then (the enabled lock index only) the lock index commands: request queue `PEXPIRE NX`+`GT` (queue ttl), `ZADD GT` of the lock name into the lock index (queue ttl), `ZREMRANGEBYSCORE` of expired lock names), drop the dead requests of other priority bands that block the request (the write queue head and the earlier opposite-mode request) with their bands in one more round trip that re-reads the heads (`drop_dead_requests_ahead`, Step 2.2, also in the pre-check: up to `MAX_DEAD_REQUEST_REMOVALS` = 5 such round trips per attempt, none while no dead request blocks), take the lock if allowed (see Read/Write locks; with the enabled lock index the obtaining `MULTI` ends with the lock index `ZADD GT` (lock ttl), the `:extendable_work_through` extension `MULTI` with `ZADD XX INCR`). Eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends the queueing commands in the lock state round trip: `lock {}` = 4 round trips (`WATCH`, state + queueing, `MULTI/EXEC`, release), a failed attempt = 3. Pre-check (`precheck_lock`, retries and `fail_fast` attempts): the lock state (+ the queueing for retries) without `WATCH` in one round trip; an attempt that can not obtain the lock fails there (same results and log events in the same order: `lock_obstacle`/`log_lock_obstacle`, the dropped dead requests after the queueing events: `log_dropped_requests`), otherwise the attempt continues with `WATCH`/`MULTI` (a failed retry = 1 round trip, a successful one = 4; `fail_fast` on an obtained lock = 1); same-process conflicts are never decided by the pre-check. Handles reentrant conflicts (incl. read/write ones); write TTL extension via inline Lua (`PTTL` + `PEXPIRE`). |
71
174
  | `WithAcqTimeout` | Global acquisition timeout (`timeout`). |
72
175
  | `DelayExecution` | Retry delay + jitter between attempts. |
73
176
  | `DequeueFromLockQueue` | Removes the acquirer from the queue of the requested mode on timeout/failure. |
74
177
  | `YieldExpire` | Runs the user block (optionally `timed`), then releases/expires the lock of the held mode (write: `EXPIRE 0`; read: `ZREM` own reader + `DEL` own read lock data; extendable reentrant: decrease). Known limitation (kept by the user's decision, documented in README): the write release and the write ttl decrease have no owner check, so a block longer than the ttl can delete/shorten the lock of another acquirer. |
75
178
  | `LogVisitor` / `InstrVisitor` | One method per lifecycle event for logs and instrumentation. |
76
179
 
180
+ ### Lock series (`acquirer/lock_series.rb`)
181
+
182
+ The second part of the lock acquisition core (not an operation module): `LockSeries` extends `TryToLockSeries` (it includes
183
+ `AcquireLock::TryToLock` and `ReleaseLockSeries`), `AcquireLock::DelayExecution` and `AcquireLock::YieldExpire` (`yield_with_timeout`).
184
+
185
+ - Semantics: all or nothing; all locks are obtained by one transaction with the same `ttl` at the same moment; nothing is held while
186
+ the series waits (no hold-and-wait, no ttl staircase); `timeout`, `retry_count`, `queue_ttl` are limits of the whole series; one
187
+ `rw_mode`, `priority`, `ttl`, `meta` per series; lock names: at least one (`ArgumentError`), `uniq(&:to_s)`.
188
+ - Deadlock freedom: one `acquirer_position` in all request queues of the series, reset for all queues at once when the dead score is
189
+ reached: all requests of a lock are ordered by `(score, acq_id)` in the same way in all queues, a request waits only for earlier
190
+ requests, so a wait cycle is impossible for any order of the lock names (nested holders of the user's logic are out of it).
191
+ - Attempt (`TryToLockSeries#try_to_lock_series`): the single lock attempt for N locks, built from its helpers: `multi(watch: watched_keys)`
192
+ (write: all lock keys + all registries, read: all lock keys) → one pipeline with `pipeline_lock_state` of each lock (`pipeline_series_lock_state`,
193
+ each lock has the same number of replies: `series_lock_state_replies`, read by the `LOCK_STATE_*` constants: `series_lock_states`) →
194
+ conflicts per lock (`same_process_conflict_status`: `:conflict_dead_lock`/`:conflict_lock_upgrade` fail the series, work through locks are
195
+ not queued and not obtained) → fail-fast check → the queueing of the other locks (eager queueing or `queue_series_lock_requests`) →
196
+ `drop_series_dead_requests_ahead` (batch Step 2.2: one round trip for all locks, up to `MAX_DEAD_REQUEST_REMOVALS`, none while nothing dead
197
+ blocks) → `find_series_lock_obstacle` (the first obstacle in the order of the lock names, `lock_obstacle` per lock) → one `MULTI`:
198
+ (transaction layout) read lock extensions first (`READ_LOCK_EXTENSION_COMMANDS` + 1 with the lock index, `ZADD XX INCR` first: its `nil` reply =
199
+ expired during the extension, the transaction is applied and can not be rolled back: the series releases its locks and fails the
200
+ attempt with `:read_lock_is_expired_during_extension`), then the commands of the other locks (`transact_*` helpers of `TryToLock`, shared with the single lock attempt;
201
+ reentrant locks also `ZREM` their request: the pre-check of a retry queues all locks).
202
+ - Pre-check (`precheck_lock_series`): retries and `fail_fast`, the lock state of all locks without `WATCH` in one round trip (with the queueing
203
+ for retries); a same-process conflict of any lock continues the attempt (no decision), an obstacle fails it; never obtains locks.
204
+ - Round trips (any number of locks): obtained series with a block: write = 6 (`WATCH`, state + queueing, `MULTI/EXEC`, release: `WATCH`, owners,
205
+ `MULTI/EXEC`), read = 4; failed attempt = 3; failed retry = 1 (pre-check); `fail_fast` on an obtained lock = 1; the dequeue of a failed series = 1.
206
+ - Failures: `drop_failed_lock_series` runs in `ensure` for every series that is not obtained (a failure, the `timeout`, `Timeout.timeout` of the
207
+ outer logic, Redis errors, `Thread#kill`): best effort (`run_non_critical`, never replaces the original error): the locks of the obtaining
208
+ transaction whose reply is lost (`acq_process[:lock_processes]`, set before `EXEC`: owner-checked release of the `:lock_obtaining` locks) and the
209
+ requests in all queues (one pipeline of `ZREM`, except `:fail_fast_no_try`: nothing is queued). The result is built after it, in one place
210
+ (`lock_series_obtaining_failed` log + event, then `raise_lock_series_error` with `raise_errors` or the failure result). Both releases go through
211
+ `complete_lock_series_release`: an interrupted release is repeated once (`ensure`, `run_non_critical`: an interruption is delivered once, the release is idempotent). All of it is best
212
+ effort: an interruption right at the entry of `ensure`, a second interruption or a dead process leave the locks till their ttl (the safe direction).
213
+ - Release (`ReleaseLockSeries#release_lock_series`): by `lock_processes` and `held_rw_modes`: `:lock_obtaining` write locks: compare-and-delete in
214
+ one transaction (`WATCH` own keys + pipelined `HGET acq_id` + `DEL`, retried on abort with the keys still owned), read locks: `ZREM` + `DEL` of
215
+ the reader data; `:extendable_conflict_work_through`: the ttl is decreased by `ttl - spent time - REDIS_TIMESHIFT_ERROR` (write: `EVAL DECREASE_LOCK_PTTL`
216
+ in the same owner-checked transaction, read: `ZADD XX INCR`); `:conflict_work_through` locks are kept as is.
217
+ - Results: with a block: the block result; without: `{ ok: true, result: { lock_series:, rql_lock_series:, acq_id:, hst_id:, ts:, ttl: } }`;
218
+ `detailed_result: true`: `yield_result`, `locks_release_strategy`, `locks_released_at`, `locks_acq_time`, `locks_hold_time`, `lock_series`, `rql_lock_series`;
219
+ failure: `{ ok: false, result: { error: :failed_to_acquire_lock_series, detailed_errors:, lock_series:, acquired_locks: [], missing_locks:, failed_on_lock: } }`
220
+ (`failed_on_lock`: the first lock that blocked the last attempt with a known lock, `nil` without such an attempt).
221
+ - Observability (`logic.md#16`, `visitors.md`): the `AcquireLock` events are not raised, the series has its own events, with the series context in their names:
222
+ of the series: `lock_series_obtained`, `lock_series_hold_and_release`, `lock_series_obtaining_failed`; of each lock (`lock_key` + `lock_keys`):
223
+ `lock_series_intermediate_lock_obtained` (the lock passed all checks of an attempt: `ready_locks` of the attempt, added by `find_series_lock_obstacle`, reported by
224
+ `lock_series` out of the WATCH block; nothing is held; once per lock, never per attempt: `reported_ready_locks`; in the order of the lock names up to the blocking lock, so a failed
225
+ attempt shows the same picture as the old sequential series; a failed conflict or `fail_fast` check reports nothing: these attempts fail before the checks), `lock_series_lock_obtained` (after the
226
+ transaction is applied, before `lock_series_obtained`, `:lock_obtaining` locks only), `lock_series_lock_obtaining_failed` (the blocking lock, once per failed series, before
227
+ `lock_series_obtaining_failed`; not raised without a known blocking lock). `log_lock_try` logs of each attempt are the `try_lock.*` logs of `TryToLock::LogVisitor`, one per lock
228
+ (per the blocking lock for the exit events); the detailed timeout error reads the raw state of the blocking lock (`blocking_lock_details`: no dependency on `LockInfo`/`QueueInfo`).
229
+ - Design decisions: a waiting series reserves its turn in the queues of its free locks (`:queued`: these locks stay free, new requests wait for the
230
+ series, bounded by its limits); `:random` needs all locks free at the same moment; a retry of a series with a reentrant lock has no one round
231
+ trip failure (the attempt decides conflicts); no Lua (WATCH/MULTI only).
232
+
77
233
  ### Main `lock` options (defaults from config)
78
234
 
79
235
  `ttl`, `queue_ttl`, `lock_index_grace_period` (used with the enabled lock index only; the mode itself, `config['lock_index_enabled']`,
80
236
  is client-wide: `Client` passes it to the acquirers, it is not a `lock` option), `timeout`, `timed`, `retry_count`, `retry_delay`, `retry_jitter`,
81
- `raise_errors`, `fail_fast`, `conflict_strategy`, `access_strategy`, `read_write_mode`
237
+ `raise_errors`, `fail_fast`, `conflict_strategy`, `access_strategy`, `priority` (`-100`..`100` or a priority name of `config['lock_priorities']`, default
238
+ `config['default_lock_priority']` = `0`, see "Lock request priorities"), `rw_mode`
82
239
  (`:write` (default, exclusive) | `:read` (shared)), `identity`, `meta`, `logger`, `log_lock_try`,
83
240
  `instrumenter`, `instrument`, log/instr sampling options, `log_sample_this`, `instr_sample_this`.
84
241
 
85
242
  - `access_strategy`: `:queued` (FIFO, default) or `:random`.
86
243
  - `conflict_strategy` (same process re-acquires its own lock): `:wait_for_lock` (default),
87
244
  `:work_through`, `:extendable_work_through`, `:dead_locking`.
88
- - `read_write_mode`: `:write` (default) | `:read`, see "Read/Write locks" below.
245
+ - `rw_mode`: `:write` (default) | `:read`, see "Read/Write locks" below.
89
246
 
90
- ### Read/Write locks (`read_write_mode`)
247
+ ### Read/Write locks (`rw_mode`)
91
248
 
92
249
  - Semantics: `:write` is exclusive (waits for the write lock and all live read locks); `:read` is
93
250
  shared (waits for the write lock only). Default `:write` keeps the classic behavior and keys.
@@ -98,8 +255,9 @@ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option),
98
255
  ignored (`score > now`) and pruned on reader acquisition; registry TTL = max reader TTL (`PEXPIRE NX`
99
256
  + `PEXPIRE GT` in the same MULTI).
100
257
  - Ordering (`:queued`): FIFO between modes by `(score, acq_id)`: read waits for earlier write requests;
101
- write must be the head of the write queue and waits for earlier read requests. Every attempt prunes
102
- dead requests in both queues. `:random` ignores queues (writers can starve). Ordering is best effort
258
+ write must be the head of the write queue and waits for earlier read requests. Scores are shifted by the priority
259
+ bands (see "Lock request priorities"), so "earlier" means "ahead": a higher priority is ahead in both modes. Every attempt prunes
260
+ dead requests of its priority band in both queues (and the dead requests of other bands that block it). `:random` requests ignore the order of the queues (writers can starve), but they are queued like any request (`:queued` requests wait for them). Ordering is best effort
103
261
  (client clocks, `unlock` clears queues); it never affects mutual exclusion.
104
262
  - Read lock data: `rql:lock_reader:<name>:<acq_id>` HASH with the write-lock format without `acq_id`/`hst_id`
105
263
  (`ts`, `ini_ttl`, `meta`, `spc_*` reentrant counters; acquirer and host are derived from the acquirer id and are
@@ -113,17 +271,17 @@ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option),
113
271
  - Try results (internal symbols): `:write_request_is_ahead`, `:read_request_is_ahead`,
114
272
  `:read_lock_is_still_acquired`, `:conflict_lock_upgrade`, `:read_lock_is_expired_during_extension`;
115
273
  try success result carries internal `rw_mode` (held mode), not exposed in the public `lock` result.
116
- - Validation (`AcquireLock`): `read_write_mode` must be `:read`/`:write`; `ttl` must be a positive
274
+ - Validation (`AcquireLock`): `rw_mode` must be `:read`/`:write`; `ttl` must be a positive
117
275
  Integer in both modes (`RedisQueuedLocks::ArgumentError`; write: `PEXPIRE` with a non-positive ttl would
118
276
  delete the lock key in the obtaining `MULTI`, read: the reader expiration is computed from it);
119
- `LockSeriesPoC` checks the series `ttl` the same way before computing the lock ttls (they include
120
- the retry delay and jitter); `config['default_lock_ttl']` must be positive too; `extend_lock_ttl`
277
+ `LockSeries` checks the same arguments itself (duplicated on purpose); `config['default_lock_ttl']` must be positive too; `extend_lock_ttl`
121
278
  `milliseconds` must be a positive Integer (a non-positive one would decrease/delete the lock);
122
279
  `meta` keys must not be reserved lock data keys, String or Symbol (Symbol keys are written as
123
- Strings: `Symbol#name`; the same check in `AcquireLock` and `LockSeriesPoC`); the other per-call
124
- numeric options are validated with the ranges of their config keys (`logic.md#7`): `queue_ttl` > 0,
280
+ Strings: `Symbol#name`; the same check in `AcquireLock` and `LockSeries`); the other per-call
281
+ numeric options are validated with the ranges of their config keys (`logic.md#7`): `priority` Integer from
282
+ `Resource::MIN_LOCK_PRIORITY` (-100) to `MAX_LOCK_PRIORITY` (100), `queue_ttl` > 0,
125
283
  `timeout`/`retry_count` nil or >= 0, `retry_delay`/`retry_jitter`/`lock_index_grace_period` >= 0
126
- (`AcquireLock`; `LockSeriesPoC` checks the retry delay/jitter before the lock ttl calculation),
284
+ (`AcquireLock`, `LockSeries`),
127
285
  scan/batch sizes, `dead_ttl`, reindexing `queue_ttl` > 0 (operation modules), `zombie_ttl` and the
128
286
  zombie scan sizes > 0 (`Swarm` facade: `SwarmArgumentError`).
129
287
  - RW-aware operations: `unlock` (write lock + readers + reader data + both queues + the lock index entry (the enabled lock index), same result shape; one release
@@ -135,41 +293,118 @@ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option),
135
293
  `'rw_mode' => 'read'`, `'rem_ttl'`, `'readers' => [reader data + rem_ttl]` when no write lock),
136
294
  `queue_info` (`'read_lock_queue'`/`'read_queue'` when present), `locks`/`locks_info`, `queues`/`queues_info`;
137
295
  write lock info in `lock_info`/`locks_info` has `'lock_key'` and `'rw_mode' => 'write'` (computed on formatting,
138
- not stored), so `'rw_mode'` is a reserved `meta` key (validated in `AcquireLock` and `LockSeriesPoC`);
296
+ not stored), so `'rw_mode'` is a reserved `meta` key (validated in `AcquireLock` and `LockSeries`);
139
297
  every request in `queue_info`/`queues_info` is `{ 'acq_id', 'score', 'rw_mode' => 'write'|'read' }`
140
298
  (`queues_info` derives the mode from the queue key via `Resource.lock_queue_rw_mode`).
141
- - `lock_series` (both modes): one mode per series; `release_lock_series` releases only locks the series
142
- obtained itself (`process == :lock_obtaining`, reentrant ones are kept) and only the current acquirer's:
143
- write = compare-and-delete in one transaction per series (`WATCH` the own lock keys + pipelined `HGET acq_id` + `DEL`,
144
- retried on abort with the keys still owned at the last check), read = `ZREM` + `DEL` reader data. It runs
145
- after the block and on any failure (exception or `raise_errors: false`), once per failed series.
299
+ - `lock_series` (both modes): one mode per series, see "Lock series".
146
300
  - Own read lock operations (current acquirer via `current_acquirer_id(identity:)`): `unlock_read` /
147
301
  `release_read_lock` (`ZREM` + `DEL` reader data, event `explicit_read_lock_release`, result
148
302
  `{ rel_time:, rel_key:, rel_acq_id:, lock_res: }`; `acquirer_id:` (default `nil` => current acquirer;
149
- when passed, `identity:` is ignored: the acquirer id already contains its identity) releases the read lock of a different acquirer) and `extend_lock_ttl(..., read_write_mode: :read)`
303
+ when passed, `identity:` is ignored: the acquirer id already contains its identity) releases the read lock of a different acquirer) and `extend_lock_ttl(..., rw_mode: :read)`
150
304
  (`multi(watch: [lock_key])`: extends only a live read lock while no write lock exists, never revives
151
305
  an expired one; registry and reader data TTLs via `PEXPIRE NX`+`GT`). `unlock` releases everything.
152
- - `extend_lock_ttl(..., read_write_mode: :read, all_read_locks: true)` (`false` by default, ignored for
306
+ - `extend_lock_ttl(..., rw_mode: :read, all_read_locks: true)` (`false` by default, ignored for
153
307
  `:write`; non-boolean in `:read` => `ArgumentError`): same `multi(watch: [lock_key])` (registry not
154
308
  watched, so concurrent readers don't abort it), `ZRANGE WITHSCORES` snapshot, every live reader gets
155
309
  `ZADD XX INCR` + reader data `PEXPIRE NX`+`GT`, registry TTL = longest extended reader;
156
310
  `extended_locks_count` = the number of non-nil `ZADD` results (0 => `:async_expire_or_no_lock`).
157
311
  `extend_lock_ttl` success result (all modes): `{ ok: true, result: { extended_locks_count: Integer } }`
158
- (write / own read = 1); failure stays `{ ok: false, result: :async_expire_or_no_lock }`. Internal param order: `read_write_mode, all_read_locks, acquirer_id`.
312
+ (write / own read = 1); failure stays `{ ok: false, result: :async_expire_or_no_lock }`. Internal param order: `rw_mode, all_read_locks, acquirer_id`.
159
313
  - Design decisions (release semantics, not limitations to "fix"): read->write upgrade is an error unless
160
314
  `:wait_for_lock`; nested reads of one acquirer go through `conflict_strategy` (no hold counting);
161
315
  `:random` gives no fairness between modes; cross-host FIFO is best effort; no Lua in the RW path.
162
- - RBS: `Client` types `read_write_mode` as `:read | :write`. Older gem versions ignore readers: all processes must be upgraded
316
+ - RBS: `Client` types `rw_mode` as `:read | :write`. Older gem versions ignore readers: all processes must be upgraded
163
317
  before read locks are used.
164
318
 
319
+ ### Lock request priorities (`priority`)
320
+
321
+ - Option `priority` of `lock`/`lock!`/`lock_series`/`lock_series!` (one value per series): Integer from
322
+ `Resource::MIN_LOCK_PRIORITY` (-100, the lowest) to `Resource::MAX_LOCK_PRIORITY` (100, the highest) or a priority
323
+ name (Symbol, RBS `RQL::lockPriority`), default `config['default_lock_priority']` (0, a name allowed); validated in
324
+ `AcquireLock` (Step 0.1), in `LockSeries` (Step 0.1: the same check, duplicated like its ttl checks) and by the
325
+ config validators.
326
+ - Priority names: `config['lock_priorities']` (Hash Symbol => Integer in the range, default frozen
327
+ `{ lowest: -100, low: -50, normal: 0, high: 50, highest: 100 }`, validated by the config) is read by `Client#lock` /
328
+ `#lock_series` and forwarded as the internal keyword `lock_priorities:` (`AcquireLock`, `LockSeries`; not a public
329
+ keyword: a client-wide registry); `AcquireLock` Step 0.1 resolves a Symbol `priority` to its Integer
330
+ (`lock_priorities.fetch(priority, priority)`, an unknown name stays a Symbol and fails the range check with
331
+ `ArgumentError`; `LockSeries` resolves it the same way), so everything after Step 0.1 (scores,
332
+ logs, events, `queue_info`) sees Integers only; a name in `default_lock_priority` must be a key of `lock_priorities`
333
+ (validators see one value: `Config#prevent_default_priority__unknown_name`, `ConfigValidationError`): `Config#configure`
334
+ checks the pair once after the block (the keys can be assigned in any order; on a failure both keys keep their values from
335
+ before the block, nested blocks: after the outer one), `Config#[]=` checks a direct assignment of either key (out of a
336
+ block) before it is applied.
337
+ - Semantics (`:queued`): strict priority, FIFO inside a priority, in both modes (the read and write queues use the same scores);
338
+ orders waiting requests only (no preemption); a `:random` request does not wait for its turn (it ignores the order and the
339
+ priorities of other requests), but it is queued at the position of its priority (`ZADD NX` like any request): `:queued`
340
+ requests wait for it; the obtained lock is never taken away.
341
+ - Encoding (`Resource`, the single source of the score layout): position (score) = enqueue time (client `Time.now.to_f`) -
342
+ `priority * Resource::LOCK_PRIORITY_BAND` (2**32 s: bands never overlap until 2106, the priority is decoded by
343
+ `Resource.lock_request_priority(score) = -floor(score / 2**32)`); priority 0 keeps the pre-priority score (old data and old
344
+ clients' default requests are unchanged). `calc_initial_acquirer_position(priority)` encodes, `dead_score_reached?(position,
345
+ priority, queue_ttl)` decodes the enqueue time for the requeue check.
346
+ - Liveness (score = order + liveness): dead requests are pruned per band, `ZREMRANGEBYSCORE <queue>
347
+ <Resource.lock_priority_band_floor(p)> <Resource.lock_priority_band_dead_score(now - queue_ttl, p)>` (the floor is `-p * 2**32`
348
+ for every band, inclusive: positions never reach band edges). Head-driven cleanup (the attempt cost does not depend on the
349
+ priority range): `queue_lock_request` prunes the band of the request in both queues before `ZADD NX` and the heads (5 queueing commands, 9
350
+ with the lock index, as before priorities) and reads both heads with scores; `drop_dead_requests_ahead` (Step 2.2 of the attempt
351
+ and of the pre-check, before the obstacle decision, `:queued` only) decodes the band and the enqueue time of the blocking requests
352
+ (the write queue head for writers, the earlier opposite-mode request for both modes) from their scores (`dead_request_priority`)
353
+ and prunes the bands of the dead ones (by the barrier of the current request: its `queue_ttl`) in both queues and re-reads both
354
+ heads in one round trip (one or two bands; a live write queue head stops the removal; `try_lock.dead_request_removed` log per
355
+ dropped request), up to `MAX_DEAD_REQUEST_REMOVALS` (5) round trips per attempt (more dead blocking requests: the next
356
+ attempts); no round trip while the blocking requests are alive or there are none. Any band is decoded (also out of the range:
357
+ newer versions). Dead requests of lower priorities never block: they are dropped by the attempts of their band (or lower),
358
+ `clear_dead_requests`, the queue TTL (enabled lock index) or the swarm. `ClearDeadRequests.drop_dead_requests` (both lookups):
359
+ one round trip reads the batch queues (`ZRANGE 0 -1 WITHSCORES`), one more prunes only the bands whose first request is dead
360
+ (the requests of a band are adjacent and ordered by time), none without dead requests; `processed_queues` of the lock index
361
+ lookup = non-empty queues. `clear_dead_requests(by_priority: true)` (`false` by default; `drop_dead_requests_by_priority`)
362
+ never reads a whole queue: a walk over the bands, each round trip reads the first request of the next band of each walked
363
+ queue (`ZRANGE <queue> <bound> +inf BYSCORE LIMIT 0 1 WITHSCORES`, from `-inf`, the next bound is
364
+ `lock_priority_band_floor(p - 1)`) and prunes the dead bands found by the previous one (the same removal commands, after the
365
+ reads); a queue is walked till a read finds nothing: the most bands of a batch queue + 1 round trips per batch, with or
366
+ without dead requests, the transfer depends on the bands, not on the requests (for long queues with few priorities);
367
+ `-inf` requests are dropped by their position (the walk goes on with `(-inf`), the walk of a queue stops at `+inf`; a
368
+ position out of the float precision of the band edges (manual writes) is passed with an exclusive bound (`(<position>`:
369
+ the next band edge rounds to the position itself); `processed_queues` of the lock index lookup = the queues with a first
370
+ request (non-empty).
371
+ - Design decision (the user's): the queue score is the only order and liveness of a lock request. A separate liveness registry
372
+ (the deadlines of the requests in another key next to the queues) was considered and rejected: a new key family for the lock
373
+ index, reindexing, release and zombie paths, more commands per attempt; the dead requests of other bands are dropped by the
374
+ head-driven removal, the attempts of their band and `clear_dead_requests` (see Limitations).
375
+ - Infinite positions (`-inf`/`+inf`: manual writes of the queues, RQL never writes them; out of all bands): the decoding never
376
+ raises (`Resource.lock_request_priority`: `-inf` => `MAX_LOCK_PRIORITY`, `+inf` => `MIN_LOCK_PRIORITY`: info results and logs);
377
+ a request at `-inf` is always dead (like before priorities: no enqueue time) and is dropped by its position
378
+ (`ZREMRANGEBYSCORE <queue> -inf -inf`, both queues) when it blocks an attempt (`drop_dead_requests_ahead`, the same round trip)
379
+ and by `clear_dead_requests` (the first requests of the queue); a request at `+inf` is never dead and never ahead of an RQL
380
+ request (kept like before priorities).
381
+ - Unchanged by priorities: keys, members (`acq_id`), `ZADD NX`, `ZRANGE 0 0`, the cross-mode `(score, acq_id)` comparison, dequeue,
382
+ `unlock`/`clear_locks` (`-inf..+inf`), swarm (hosts from acquirer ids), lock index.
383
+ - Info/observability: `queue_info`/`queues_info` requests have `'priority'` (decoded); every lock acquirement log and event
384
+ carries the Integer `priority` of the request (classic, reentrant: the requested one, they never queue; lock series: the
385
+ priority of the series); `log_lock_try` logs that name
386
+ another request carry its decoded priority (`first_priority_in_queue`, `ahead_priority`, `dead_priority`; decoded for logs
387
+ only: gated by `log_sampled && log_lock_try`, `TryToLock#logged_request_priority`; `first_priority_in_queue => nil` without the first request).
388
+ - Limitations: the priority range is a library constant that bounds the score shift (float precision of the shifted positions:
389
+ ~61 µs at ±100, ~0.5 ms at ±1000; requests of the same priority within this interval are ordered by acquirer ids); mixed
390
+ versions: older clients prune high-priority requests as dead ones (`-inf..now-ttl`) and never prune low-priority ones (always
391
+ behind): the order degrades, no stuck queues, no safety impact (upgrade all processes before using positive priorities; `0` and negative priorities are compatible: older clients keep them behind their own requests and never prune them, the upgraded processes drop the dead ones); strict priority
392
+ can starve lower priorities (bounded by their timeout/retry limits); the handoff is polled (`retry_delay`); dead requests are
393
+ pruned with the pruner's `queue_ttl` (existing behavior: a request that waits longer than the shorter `queue_ttl` of the
394
+ requests of its band or of the requests it blocks is dropped by their attempts and comes back at its old position on its own
395
+ ones (its own `queue_ttl` keeps the position): it loses its place and the advantage of its priority to the requests behind it
396
+ with a shorter `queue_ttl`; README recommends one `queue_ttl` for the lock requests of a lock); dead requests of lower
397
+ priorities can stay in the queues until an attempt of their band (they never block; visible in `queue_info`; an accepted
398
+ trade-off of the head-driven cleanup: README recommends a periodic `clear_dead_requests` without the lock index and the swarm).
399
+
165
400
  ### Redis data layout (`resource.rb`)
166
401
 
167
402
  | Key | Type | Purpose |
168
403
  |---|---|---|
169
404
  | `rql:lock:<name>` | HASH | lock owner + metadata (e.g. `l_spc_ts`, `l_spc_ext_ts` for reentrant cases) |
170
- | `rql:lock_queue:<name>` | ZSET | acquirer queue scored by enqueue time |
405
+ | `rql:lock_queue:<name>` | ZSET | acquirer queue scored by enqueue time shifted by the priority band (`ts - priority * 2**32`) |
171
406
  | `rql:lock_readers:<name>` | ZSET | read locks: acquirer id => expiration (Redis `TIME`, ms) |
172
- | `rql:lock_read_queue:<name>` | ZSET | read lock requests (write requests use `rql:lock_queue:<name>`) |
407
+ | `rql:lock_read_queue:<name>` | ZSET | read lock requests (write requests use `rql:lock_queue:<name>`), same scores |
173
408
  | `rql:lock_reader:<name>:<acq_id>` | HASH | read lock data (same fields as `rql:lock:<name>` incl. meta, without `acq_id`/`hst_id`; data carrier only) |
174
409
  | `rql:swarm:hsts` | HASH | swarm host heartbeats |
175
410
  | `rql:index:locks` | ZSET | lock index (the enabled lock index only): lock name => upper bound of the lifetime of all its keys (client time ms + ttl + grace), see "Lock index" |
@@ -178,7 +413,7 @@ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option),
178
413
 
179
414
  - Lock lookup modes of all lock-iterating operations (`locks`/`locks_info`, `queues`/`queues_info`, `keys`, `clear_locks`, `clear_locks_of`/`clear_current_locks`, `clear_dead_requests`, `flush_zombies` and the zombie flusher, `zombie_locks`/`zombie_acquirers`/`zombies_info`), switched by `config['lock_index_enabled']` (client-wide, `false` by default; `Client`/`Swarm` read it and pass it to the modules, the zombie flusher Ractor gets a copy at spawn). Each module keeps both lookups side by side (shared processing in one copy):
180
415
  - keyspace (disabled lock index, default): one SCAN pass over `KEY_PATTERN` (or `ALL_LOCK_QUEUES_PATTERN` for queues), keys classified by `Resource.lock_key_family`, per-key commands in pipelined batches collected across SCAN pages (`scan_locks`, `scan_queues`, `each_lock_queue`, `flush_keyspace_zombies`, `each_keyspace_zombie_lock`, inline SCAN branches in `keys`/`clear_locks`/`clear_locks_of`); no lock index commands at all;
181
- - lock index (enabled): one `ZSCAN` pass over `rql:index:locks` (lock names; names with an expired score are skipped) in batches of `scan_size`/`batch_size` lock names collected across ZSCAN pages, the lock keys are derived from the names (`prepare_lock_key`, `prepare_lock_readers`, both queues, reader data via registry members), per-key commands in pipelined batches; the full keyspace is never scanned. Each module has its own copy of the iteration (`each_lock_name_batch`, duplicated on purpose: module independence; `index_locks`, `index_queues`, `flush_lock_index_zombies`, `each_lock_index_zombie_lock`). Info queries check key existence (`EXISTS`); `clear_locks` deletes the keys of a batch with one `DEL` in a `MULTI` with the `ZREM` of the batch names; `clear_locks`/`clear_dead_requests`/`flush_zombies` prune expired names first.
416
+ - lock index (enabled): one `ZSCAN` pass over `rql:index:locks` (lock names; names with an expired score are skipped: `Resource.lock_index_score_expired?`, ZSCAN scores are Strings, the `+inf` score of a key without TTL (`"inf"`) never expires) in batches of `scan_size`/`batch_size` lock names collected across ZSCAN pages, the lock keys are derived from the names (`prepare_lock_key`, `prepare_lock_readers`, both queues, reader data via registry members), per-key commands in pipelined batches; the full keyspace is never scanned. Each module has its own copy of the iteration (`each_lock_name_batch`, duplicated on purpose: module independence; `index_locks`, `index_queues`, `flush_lock_index_zombies`, `each_lock_index_zombie_lock`). Info queries check key existence (`EXISTS`); `clear_locks` deletes the keys of a batch with one `DEL` in a `MULTI` with the `ZREM` of the batch names; `clear_locks`/`clear_dead_requests`/`flush_zombies` prune expired names first.
182
417
  - `reindex_locks`: one SCAN pass over `KEY_PATTERN` (keys classified by `Resource.lock_key_family`) that adds keys without index entries to the lock index; it works regardless of the mode (it is needed after enabling the lock index).
183
418
  - Acquirer ID: `rql:acq:<pid>/<thread>/<fiber>/<ractor>/<identity>`
184
419
  - Host ID: `rql:hst:<pid>/<thread>/<ractor>/<identity>` (no fiber)
@@ -208,7 +443,7 @@ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option),
208
443
  - the score is never decreased (`GT`; ttl decreases and releases do not touch the index);
209
444
  - lock request queues have TTL (`PEXPIRE NX` + `GT` = queue ttl on each queueing): the lifetime of every indexed key is bounded;
210
445
  - pruning: `ZREMRANGEBYSCORE <index> -inf <now - 1>` in every queueing pipeline (amortized, no background jobs) and in `flush_zombies`,
211
- `clear_locks`, `clear_dead_requests`; info operations skip names with an expired score;
446
+ `clear_locks`, `clear_dead_requests`; lock-iterating operations skip names with an expired score (`+inf`: never);
212
447
  - removal: only together with the deletion of all indexed keys of the name in one `MULTI` (`unlock`: `ZREM` in the release transaction;
213
448
  `clear_locks`: `DEL` of the batch keys + `ZREM` of the batch names); never remove a name while some of its keys can stay (`clear_locks_of`,
214
449
  `yield_expire` releases, `flush_zombies` keep the names).
@@ -223,7 +458,8 @@ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option),
223
458
 
224
459
  `redis_queued_locks.` + `lock_obtained`, `reentrant_lock_obtained`,
225
460
  `extendable_reentrant_lock_obtained`, `lock_hold_and_release`, `reentrant_lock_hold_completes`,
226
- `lock_obtaining_failed`, `lock_series_obtained`, `lock_series_hold_and_release`, `explicit_lock_release`,
461
+ `lock_obtaining_failed`, `lock_series_intermediate_lock_obtained`, `lock_series_lock_obtained`, `lock_series_lock_obtaining_failed`, `lock_series_obtained`,
462
+ `lock_series_hold_and_release`, `lock_series_obtaining_failed`, `explicit_lock_release`,
227
463
  `explicit_all_locks_release`, `release_locks_of`, `explicit_read_lock_release`, `extend_lock_ttl`, `clear_dead_requests`,
228
464
  `reindex_locks`. Observability rule (`logic.md#16`): logic that changes keys is logged and instrumented, the lock
229
465
  acquirement per meaningful outcome (visitors; `lock_obtaining_failed` with the failure `result` also before a raised
@@ -231,9 +467,17 @@ acquirement error), each mutation operation with one log line + one event of the
231
467
  `instrument` last in the payload, inline in the module); lock info queries are neither; the swarm is known debt.
232
468
  Lock events (`lock_obtained`, `reentrant_lock_obtained`, `extendable_reentrant_lock_obtained`,
233
469
  `lock_hold_and_release`, `reentrant_lock_hold_completes`) carry `rw_mode` (requested mode) in the payload;
234
- lock series events (`lock_series_obtained`, `lock_series_hold_and_release`) carry the mode of the series;
470
+ lock series events (`lock_series_*`) carry the mode of the series (`lock_series_obtained` and `lock_series_hold_and_release` also `reentrant_lock_keys`,
471
+ `lock_series_obtaining_failed` `result` and `failed_lock_key`, the events of each lock `lock_key` and `lock_keys`, `lock_series_lock_obtaining_failed` `result`),
472
+ the `AcquireLock` events are not raised for the locks of a series;
473
+ all lock acquirement events (the lock events above, `lock_obtaining_failed` and the lock series events) carry `priority`
474
+ after `rw_mode` (the Integer lock request priority: a priority name is resolved before; the series events: the priority of
475
+ the series); all lock acquirement log lines of `AcquireLock` and `LockSeries` (not the release logs of `YieldExpire`)
476
+ log `priority => N` after `rw_mode`, `try_lock.acq_added_to_queue` too; the try-lock logs that name another request log
477
+ its priority (`first_priority_in_queue` after `first_acq_id_in_queue`, `ahead_priority` after `ahead_acq_id`);
235
478
  lock log lines (incl. lock series logs) carry `rw_mode => '...'`. RW try-lock log events: `exit__write_request_ahead`,
236
- `exit__read_request_ahead`, `exit__read_lock_still_obtained`, `single_process_lock_conflict__lock_upgrade`.
479
+ `exit__read_request_ahead`, `exit__read_lock_still_obtained`, `single_process_lock_conflict__lock_upgrade`. Priorities try-lock
480
+ log event: `dead_request_removed` (`dead_acq_id`, `dead_priority`: a dead blocking request of another band was dropped).
237
481
 
238
482
  ### Errors (`errors.rb`)
239
483
 
@@ -246,7 +490,7 @@ hierarchy: `LockAlreadyObtainedError`, `LockAcquirementTimeoutError`, `LockAcqui
246
490
  `rescue => e`/`rescue StandardError`/`rescue Timeout::Error`/`rescue nil` of the interrupted code (the user
247
491
  block, logger and instrumenter calls) can not swallow them; the library converts them to the public errors/results).
248
492
  redis-client sends `UNWATCH` on an interrupted `multi(watch:)` only for `StandardError`s, so every `with` block
249
- around a `multi(watch:)` (`TryToLock`, `ExtendLockTTL` read extensions, `LockSeriesPoC` write release) rescues
493
+ around a `multi(watch:)` (`TryToLock`, `TryToLockSeries`, `ExtendLockTTL` read extensions, `ReleaseLockSeries` write release) rescues
250
494
  both internal timeouts and `Timeout::ExitException` (the user's `Timeout.timeout` without an exception class), closes
251
495
  the connection (drops the WATCH state, reconnected on the next use) and re-raises.
252
496
 
@@ -258,7 +502,7 @@ the connection (drops the WATCH state, reconnected on the next use) and re-raise
258
502
  |---|---|
259
503
  | `Client` | Facade; constructor injection (caller-supplied `RedisClient`); `x` returns a result hash, `x!` raises |
260
504
  | `Acquirer::*` | One module per command; stateless `class << self` functions; uniform `{ ok:, result: }` returns (public API contract; swarm element internals use bare scalars/primitives) |
261
- | `AcquireLock` | Composition via `extend` mixins; optimistic concurrency (WATCH/MULTI; asymmetric WATCH for read/write locks) + Lua only for write TTL extension/decrease; strategy options (`access_strategy`, `conflict_strategy`, `read_write_mode`) |
505
+ | `AcquireLock` | Composition via `extend` mixins; optimistic concurrency (WATCH/MULTI; asymmetric WATCH for read/write locks) + Lua only for write TTL extension/decrease; strategy options (`access_strategy`, `conflict_strategy`, `rw_mode`) |
262
506
  | Log/Instr visitors | Visitor-style event hooks keep observability out of the algorithm; percent sampling via `sampling_happened?(percent)` |
263
507
  | `Logging` / `Instrument` | Null Object defaults (`VoidLogger`, `VoidNotifier`); Adapter (`instrument/active_support.rb`); duck typing (`::Logger` API, `#notify(event, payload)`) |
264
508
  | `Config` | Declarative DSL: `setting(key, default)` and `validate(key) { }` registries; access as `config['a.b']`; runtime `Client#configure` |
@@ -274,7 +518,7 @@ the connection (drops the WATCH state, reconnected on the next use) and re-raise
274
518
  |---|---|
275
519
  | Retry/timeouts | `retry_count` (3), `retry_delay` (200 ms), `retry_jitter` (25 ms), `try_to_lock_timeout` (10 s), `is_timed_by_default` (false) |
276
520
  | TTLs | `default_lock_ttl` (5000 ms), `default_queue_ttl` (15 s), `dead_request_ttl` (1 day, ms) |
277
- | Strategies | `default_conflict_strategy` (`:wait_for_lock`), `default_access_strategy` (`:queued`) |
521
+ | Strategies | `default_conflict_strategy` (`:wait_for_lock`), `default_access_strategy` (`:queued`), `default_lock_priority` (0, from -100 to 100 or a priority name), `lock_priorities` (priority names => Integer priorities) |
278
522
  | Batching | `lock_release_batch_size` (100), `clear_locks_of__lock_scan_size` / `__queue_scan_size` (300), `key_extraction_batch_size` (500) |
279
523
  | Lock index | `lock_index_enabled` (false; client-wide mode, config only), `lock_index_grace_period` (60_000 ms; also a `lock`/`lock!`/`lock_series`/`lock_series!`/`reindex_locks` option) |
280
524
  | Identity | `uniq_identifier` (lambda → `Resource.calc_uniq_identity`) |
@@ -294,19 +538,18 @@ lib/redis_queued_locks/
294
538
  config.rb, config/dsl.rb settings DSL + defaults
295
539
  acquirer/*.rb one operation per file
296
540
  acquirer/acquire_lock/* lock algorithm mixins + visitors
541
+ acquirer/lock_series/* lock series attempt and release mixins + visitors
297
542
  swarm.rb, swarm/* supervisor, swarm elements, zombie logic, redis client builder
298
543
  logging/, instrument/ void defaults, samplers, ActiveSupport adapter
299
544
  resource.rb keys and identities
300
545
  data.rb, errors.rb, utilities.rb, utilities/lock.rb, debugger/, version.rb
301
546
  sig/ RBS mirror of lib/ + sig/vendor stubs (redis_client, active_support, semantic_logger)
302
- spec/ redis_queued_locks_spec.rb (~5.6k lines, integration: shared examples run with both lock lookup modes), spec_helper.rb, setup_simplecov.rb
547
+ spec/ redis_queued_locks_spec.rb (~5.6k lines, integration: shared examples run with both lock lookup modes), redis_queued_locks/config_spec.rb (Redis-free: `Config#[]` RBS table == `setting` registry), spec_helper.rb, setup_simplecov.rb, support/parallel_runner.rb (the `rake rspec:parallel` runner, not a spec)
303
548
  .github/workflows/ tests, lint, typecheck-static, typecheck-runtime
304
549
  bin/console, bin/setup dev scripts
305
550
  Rakefile, Steepfile, rbs_collection.yaml
306
551
  ```
307
552
 
308
- Known quirk: `sig/redis_queued_locks/acquier.rbs` is misspelled (should be `acquirer.rbs`).
309
-
310
553
  ---
311
554
 
312
555
  ## 5. Technology stack
@@ -329,27 +572,27 @@ Known quirk: `sig/redis_queued_locks/acquier.rbs` is misspelled (should be `acqu
329
572
  - rake, bundler, pry, pry-doc, reline, activesupport 8.1
330
573
 
331
574
  ### Rake tasks
332
- `rspec` (default), `rubocop` (runs `rubocop:ruby` and `rubocop:rbs`, fails if either fails), `steep:check`, plus bundler gem tasks (`build`, `release`, ...).
575
+ `rspec` (default), `rspec:parallel` (`spec/support/parallel_runner.rb`, the `Rakefile` only declares the task: the specs in `WORKERS` processes, ~1 min instead of ~6: each worker gets its own pair of Redis databases via `RQL_SPEC_DB_OFFSET` = worker index * 2, prepended to `RedisClient::Config::Common#initialize` in `spec_helper.rb` (the library, swarm and Ractor clients included; after the URL is resolved: the database of a `url:` gets the offset too); examples listed by `rspec --dry-run` and distributed by the durations of the previous runs (`tmp/rspec_parallel/durations.json`, the longest first to the least loaded worker); per-worker logs/JSON in `tmp/rspec_parallel/` (the worker files of the previous run are deleted before the workers start: a worker that dies before RSpec writes its JSON output is reported as broken, never with stale results); coverage: each worker stores only its SimpleCov result in `coverage/parallel/worker-<n>` (`rspec:worker:<n>` command names, no report), the task collates one report into `coverage/` after all workers (`SimpleCov.collate`) and drops the collated resultset (later `rspec` runs don't merge it); the environment is passed to the workers: the RBS runtime checks and `RSPEC_RETRY_RETRY_COUNT` work as with `rspec`), `rubocop` (runs `rubocop:ruby` and `rubocop:rbs`, fails if either fails), `steep:check`, plus bundler gem tasks (`build`, `release`, ...).
333
576
 
334
577
  ### CI (GitHub Actions, ubuntu-latest, Ruby 4.0, on every push)
335
578
 
336
579
  | Workflow | Command |
337
580
  |---|---|
338
- | tests | `bundle exec rake rspec` (with a Redis service) |
339
- | lint | `bundle exec rake rubocop` |
581
+ | tests | `bundle exec rake rspec:parallel` (with a Redis service; 4 workers on the 4-core runner; `tmp/rspec_parallel/durations.json` restored/saved with `actions/cache` (a new key per run, `restore-keys` prefix) to balance the workers; the worker logs are uploaded as an artifact on failure) |
582
+ | lint | `bundle exec rake rubocop` (precompiled native gems: no `BUNDLE_FORCE_RUBY_PLATFORM`, a rubydex 0.4.1 source build on the runner flags constants as `@deprecated` and breaks `Lint/DeprecatedReference`) |
340
583
  | typecheck-static | `bundle exec rbs collection install && bundle exec rake steep:check -j 10` |
341
- | typecheck-runtime | `RBS_TEST_RAISE=true RUBYOPT='-rrbs/test/setup' RBS_TEST_OPT='-I sig' RBS_TEST_TARGET='RedisQueuedLocks::*' bundle exec rspec --failure-exit-code=0` (never fails the build) |
584
+ | typecheck-runtime | `RBS_TEST_RAISE=true RUBYOPT='-rbundler/setup -rrbs/test/setup' RBS_TEST_OPT='-I sig' RBS_TEST_TARGET='RedisQueuedLocks::*' RBS_TEST_SKIP='::RedisQueuedLocks::Swarm::FlushZombies,::RedisQueuedLocks::Swarm::SwarmElement::Isolated,::RedisQueuedLocks::Swarm::RedisClientBuilder' bundle exec rspec` (blocking; serial on purpose: the RBS hooks keep a worker on a CPU core, and parallel workers on the runner (4 vCPU = 2 physical cores) starved the timing-sensitive examples: Redis connect timeouts with 4 workers, swarm flush waits with 2; locally `rake rspec:parallel` works for the RBS checks; Ractor-executed methods are skipped: `RBS_TEST_SKIP` + `%a{rbs:test:skip}` in `resource.rbs`/`utilities.rbs`) |
342
585
 
343
586
  ---
344
587
 
345
588
  ## 6. Conventions
346
589
 
347
590
  - `# frozen_string_literal: true` in every Ruby file.
348
- - YARD tags on every class/method: `@api public|private`, `@since`, `@version` (methods: bump on a behavior/signature change; classes/modules: only on an interface change: signatures, added/removed methods, `extend`/`include`/`prepend` or the superclass; `logic.md#2`).
591
+ - YARD tags on every class/method: `@api public|private`, `@since`, `@version` (methods: bump on a behavior/signature change; classes/modules: only on an interface change: signatures, added/removed methods, `extend`/`include`/`prepend` or the superclass; `logic.md#2`); `# @since` right above each `include`/`extend`/`prepend` line (the version that added the mixin).
349
592
  - Every `lib/` change gets a matching `sig/` RBS update; use `# steep:ignore` only where Steep can't infer types.
350
593
  - New operation: new `Acquirer::*` module, thin `Client` method (+ `!` variant if it should raise), RBS, spec.
351
- - New config option: `setting` (+ `validate`) in `config.rb`, read via `config['key']`, document defaults.
594
+ - New config option: `setting` (+ `validate`) in `config.rb` + the typed `Config#[]` overload line in `sig/redis_queued_locks/config.rbs`, read via `config['key']`, document defaults.
352
595
  - Development gems go in `Gemfile` (`Gemspec/DevelopmentDependencies: Gemfile`), not the gemspec.
353
- - This overview and `.claude/rules/*.md` are updated in the same change as the code (keys, options, results, events, algorithm steps, limitations); README and CHANGELOG `[Unreleased]` for user-visible behavior (log keys and instrumentation payloads: README `## Logging` / `### Instrumentation Events`).
354
- - Temporary debt: rspec-retry, disabled coverage minimum, runtime type-check job that can't fail.
596
+ - This overview and `.claude/rules/*.md` are updated in the same change as the code (keys, options, results, events, algorithm steps, limitations); README and CHANGELOG `[Unreleased]` for user-visible behavior (log keys and instrumentation payloads: README `## Logging` / `### Instrumentation Events`); CHANGELOG entries are compact, subject-level changes only (`.claude/rules/changelog.md`).
597
+ - Temporary debt: rspec-retry, disabled coverage minimum.
355
598
  - Specs (`spec/**/*_spec.rb`) are not ideal and will be totally reworked (header `NOTE` of `spec/redis_queued_locks_spec.rb`): until then any coding principle may be broken there; code review checks only that the library behavior is covered and that changed expectations are correct (`.claude/skills/rql-code-review`).