redis_queued_locks 1.19.0 → 1.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/project-overview.md +96 -23
  3. data/.claude/rules/acquirer.md +39 -20
  4. data/.claude/rules/arguments.md +3 -2
  5. data/.claude/rules/git.md +3 -2
  6. data/.claude/rules/logic.md +32 -8
  7. data/.claude/rules/swarm.md +14 -9
  8. data/.claude/rules/tests.md +12 -3
  9. data/.claude/rules/type-checking.md +2 -1
  10. data/.claude/rules/visitors.md +10 -11
  11. data/.claude/skills/rql-code-review/SKILL.md +386 -0
  12. data/.claude/skills/rql-code-review/templates/report.en.md +31 -0
  13. data/.claude/skills/rql-code-review/templates/report.ru.md +31 -0
  14. data/CHANGELOG.md +194 -0
  15. data/CLAUDE.md +6 -3
  16. data/README.md +502 -92
  17. data/lib/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue/log_visitor.rb +1 -0
  18. data/lib/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rb +42 -0
  19. data/lib/redis_queued_locks/acquirer/acquire_lock/log_visitor.rb +45 -0
  20. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rb +1 -0
  21. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +695 -128
  22. data/lib/redis_queued_locks/acquirer/acquire_lock/with_acq_timeout.rb +2 -2
  23. data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire/log_visitor.rb +1 -0
  24. data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire.rb +8 -6
  25. data/lib/redis_queued_locks/acquirer/acquire_lock.rb +134 -26
  26. data/lib/redis_queued_locks/acquirer/clear_dead_requests.rb +229 -24
  27. data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +189 -18
  28. data/lib/redis_queued_locks/acquirer/is_locked.rb +0 -1
  29. data/lib/redis_queued_locks/acquirer/is_queued.rb +0 -1
  30. data/lib/redis_queued_locks/acquirer/keys.rb +151 -7
  31. data/lib/redis_queued_locks/acquirer/lock_info.rb +39 -41
  32. data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +3 -2
  33. data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +5 -4
  34. data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +79 -20
  35. data/lib/redis_queued_locks/acquirer/locks.rb +212 -83
  36. data/lib/redis_queued_locks/acquirer/queue_info.rb +11 -18
  37. data/lib/redis_queued_locks/acquirer/queues.rb +123 -50
  38. data/lib/redis_queued_locks/acquirer/reindex_locks.rb +253 -0
  39. data/lib/redis_queued_locks/acquirer/release_all_locks.rb +177 -61
  40. data/lib/redis_queued_locks/acquirer/release_lock.rb +84 -25
  41. data/lib/redis_queued_locks/acquirer/release_locks_of.rb +274 -70
  42. data/lib/redis_queued_locks/acquirer/release_read_lock.rb +22 -3
  43. data/lib/redis_queued_locks/acquirer.rb +1 -0
  44. data/lib/redis_queued_locks/client.rb +164 -36
  45. data/lib/redis_queued_locks/config.rb +16 -13
  46. data/lib/redis_queued_locks/errors.rb +34 -4
  47. data/lib/redis_queued_locks/instrument/sampler.rb +4 -7
  48. data/lib/redis_queued_locks/logging/sampler.rb +4 -8
  49. data/lib/redis_queued_locks/resource.rb +113 -6
  50. data/lib/redis_queued_locks/swarm/flush_zombies.rb +333 -61
  51. data/lib/redis_queued_locks/swarm/redis_client_builder.rb +8 -4
  52. data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +3 -2
  53. data/lib/redis_queued_locks/swarm/zombie_info.rb +203 -53
  54. data/lib/redis_queued_locks/swarm.rb +97 -5
  55. data/lib/redis_queued_locks/utilities.rb +1 -1
  56. data/lib/redis_queued_locks/version.rb +2 -2
  57. data/sig/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rbs +14 -0
  58. data/sig/redis_queued_locks/acquirer/acquire_lock/log_visitor.rbs +13 -0
  59. data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +107 -1
  60. data/sig/redis_queued_locks/acquirer/acquire_lock/yield_expire.rbs +3 -3
  61. data/sig/redis_queued_locks/acquirer/acquire_lock.rbs +3 -1
  62. data/sig/redis_queued_locks/acquirer/clear_dead_requests.rbs +19 -1
  63. data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +20 -4
  64. data/sig/redis_queued_locks/acquirer/keys.rbs +11 -1
  65. data/sig/redis_queued_locks/acquirer/locks.rbs +15 -6
  66. data/sig/redis_queued_locks/acquirer/queues.rbs +9 -2
  67. data/sig/redis_queued_locks/acquirer/reindex_locks.rbs +52 -0
  68. data/sig/redis_queued_locks/acquirer/release_all_locks.rbs +8 -1
  69. data/sig/redis_queued_locks/acquirer/release_lock.rbs +6 -3
  70. data/sig/redis_queued_locks/acquirer/release_locks_of.rbs +20 -1
  71. data/sig/redis_queued_locks/acquirer/release_read_lock.rbs +1 -1
  72. data/sig/redis_queued_locks/client.rbs +21 -2
  73. data/sig/redis_queued_locks/errors.rbs +5 -2
  74. data/sig/redis_queued_locks/instrument/sampler.rbs +0 -2
  75. data/sig/redis_queued_locks/logging/sampler.rbs +0 -2
  76. data/sig/redis_queued_locks/resource.rbs +7 -1
  77. data/sig/redis_queued_locks/swarm/flush_zombies.rbs +51 -1
  78. data/sig/redis_queued_locks/swarm/zombie_info.rbs +58 -7
  79. data/sig/vendor/redis_client.rbs +2 -0
  80. metadata +6 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e33f744b3798258657c7b699b078ff4017f0d21341a1103437734a97cc5a46e8
4
- data.tar.gz: dacf22eae42a663ddf9984da0cda311f3b44f06b4a9e22d015eedc941b77ddbb
3
+ metadata.gz: e7aeb9d0b5ea57783c70a285b753616abf277955029184236ed7746f127148e4
4
+ data.tar.gz: 9a5771fd5f0df1edad7eb6270a211576fbcd33f4049a5fb040a65e334f92a32f
5
5
  SHA512:
6
- metadata.gz: f7b7c4b1b38e633f7ee1fd3dbccd8846da716fa6756cf5a8279488cc75ebb93244adbccbd315844ad6a8590250450beb1487fd7b4b7d3c3441833a0f6d6b6d0f
7
- data.tar.gz: 34189139e943bdae9e15d8642a8f2d504984547dd08a62efd92d27d786043e74808122c72dc24f25ccacc385862fe65e87249071f6e07e2925a72166ac3b2845
6
+ metadata.gz: 2c2328b3187ce087a694d3169aaaf9362ffb48557e626ab32933ffa4b70e015121edababddaeb3d2e24df7bfcb2c77980c54f1ed315c7a08d2d41e23e631351e
7
+ data.tar.gz: 4a380d09471c48bc00ef7454e5755420738c5f1233c576d10d115c6ca973ecb505ce77b037d805f0299964da1eae31f02dd5999e2e1945cfef6aa5508e3b7116
@@ -53,10 +53,11 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
53
53
  | `extend_lock_ttl` | `acquirer/extend_lock_ttl.rb` (`read_write_mode: :read` extends own read lock, `+ all_read_locks: true` all live read locks) |
54
54
  | `locked?` / `queued?` | `acquirer/is_locked.rb` / `acquirer/is_queued.rb` |
55
55
  | `lock_info` / `queue_info` | `acquirer/lock_info.rb` / `acquirer/queue_info.rb` |
56
- | `locks`, `locks_info`, `queues`, `queues_info`, `keys` | `acquirer/locks.rb`, `queues.rb`, `keys.rb` (SCAN-based) |
56
+ | `locks`, `locks_info`, `queues`, `queues_info`, `keys` | `acquirer/locks.rb`, `queues.rb`, `keys.rb` (keyspace SCAN or lock index: `lock_index_enabled`) |
57
57
  | `clear_locks` | `acquirer/release_all_locks.rb` |
58
58
  | `clear_locks_of`, `clear_current_locks` | `acquirer/release_locks_of.rb` |
59
59
  | `clear_dead_requests` | `acquirer/clear_dead_requests.rb` |
60
+ | `reindex_locks` | `acquirer/reindex_locks.rb` (adds keys without lock index entries to the lock index: one SCAN pass; works in both modes) |
60
61
  | `current_acquirer_id`, `current_host_id`, `possible_host_ids` | `resource.rb` |
61
62
  | `swarmize!`, `deswarmize!`, `swarm_status`, `swarm_info`, `probe_hosts`, `flush_zombies`, `zombie_locks`, `zombie_acquirers`, `zombie_hosts`, `zombies_info` | `swarm.rb` + `swarm/*` |
62
63
 
@@ -66,16 +67,17 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
66
67
 
67
68
  | Mixin | Role |
68
69
  |---|---|
69
- | `TryToLock` (`try_to_lock.rb`) | One attempt: `multi(watch:)` (write: lock key + readers registry, read: lock key only), one pipelined read of the lock state (`HGET acq_id`, `TIME`, own reader score, longest reader), `ZADD NX` into the queue of the requested mode (timestamp score), prune both queues, take the lock if allowed (see Read/Write locks). Handles reentrant conflicts (incl. read/write ones); write TTL extension via inline Lua (`PTTL` + `PEXPIRE`). |
70
+ | `TryToLock` (`try_to_lock.rb`) | One attempt: `multi(watch:)` (write: lock key + readers registry, read: lock key only), one pipelined read of the lock state (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only), the lock request queueing in one pipelined round trip (`queue_lock_request`: `ZADD NX` into the queue of the requested mode (timestamp score), prune both queues, both queue heads, then (the enabled lock index only) the lock index commands: request queue `PEXPIRE NX`+`GT` (queue ttl), `ZADD GT` of the lock name into the lock index (queue ttl), `ZREMRANGEBYSCORE` of expired lock names), take the lock if allowed (see Read/Write locks; with the enabled lock index the obtaining `MULTI` ends with the lock index `ZADD GT` (lock ttl), the `:extendable_work_through` extension `MULTI` with `ZADD XX INCR`). Eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends the queueing commands in the lock state round trip: `lock {}` = 4 round trips (`WATCH`, state + queueing, `MULTI/EXEC`, release), a failed attempt = 3. Pre-check (`precheck_lock`, retries and `fail_fast` attempts): the lock state (+ the queueing for retries) without `WATCH` in one round trip; an attempt that can not obtain the lock fails there (same results and log events: `lock_obstacle`/`log_lock_obstacle`), otherwise the attempt continues with `WATCH`/`MULTI` (a failed retry = 1 round trip, a successful one = 4; `fail_fast` on an obtained lock = 1); same-process conflicts are never decided by the pre-check. Handles reentrant conflicts (incl. read/write ones); write TTL extension via inline Lua (`PTTL` + `PEXPIRE`). |
70
71
  | `WithAcqTimeout` | Global acquisition timeout (`timeout`). |
71
72
  | `DelayExecution` | Retry delay + jitter between attempts. |
72
73
  | `DequeueFromLockQueue` | Removes the acquirer from the queue of the requested mode on timeout/failure. |
73
- | `YieldExpire` | Runs the user block (optionally `timed`), then releases/expires the lock of the held mode (write: `EXPIRE 0`; read: `ZREM` own reader + `DEL` own read lock data; extendable reentrant: decrease). |
74
+ | `YieldExpire` | Runs the user block (optionally `timed`), then releases/expires the lock of the held mode (write: `EXPIRE 0`; read: `ZREM` own reader + `DEL` own read lock data; extendable reentrant: decrease). Known limitation (kept by the user's decision, documented in README): the write release and the write ttl decrease have no owner check, so a block longer than the ttl can delete/shorten the lock of another acquirer. |
74
75
  | `LogVisitor` / `InstrVisitor` | One method per lifecycle event for logs and instrumentation. |
75
76
 
76
77
  ### Main `lock` options (defaults from config)
77
78
 
78
- `ttl`, `queue_ttl`, `timeout`, `timed`, `retry_count`, `retry_delay`, `retry_jitter`,
79
+ `ttl`, `queue_ttl`, `lock_index_grace_period` (used with the enabled lock index only; the mode itself, `config['lock_index_enabled']`,
80
+ is client-wide: `Client` passes it to the acquirers, it is not a `lock` option), `timeout`, `timed`, `retry_count`, `retry_delay`, `retry_jitter`,
79
81
  `raise_errors`, `fail_fast`, `conflict_strategy`, `access_strategy`, `read_write_mode`
80
82
  (`:write` (default, exclusive) | `:read` (shared)), `identity`, `meta`, `logger`, `log_lock_try`,
81
83
  `instrumenter`, `instrument`, log/instr sampling options, `log_sample_this`, `instr_sample_this`.
@@ -99,8 +101,9 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
99
101
  write must be the head of the write queue and waits for earlier read requests. Every attempt prunes
100
102
  dead requests in both queues. `:random` ignores queues (writers can starve). Ordering is best effort
101
103
  (client clocks, `unlock` clears queues); it never affects mutual exclusion.
102
- - Read lock data: `rql:lock_reader:<name>:<acq_id>` HASH with the write-lock format (`acq_id`, `hst_id`,
103
- `ts`, `ini_ttl`, `meta`, `spc_*` reentrant counters), TTL = read lock TTL; data carrier only (the
104
+ - Read lock data: `rql:lock_reader:<name>:<acq_id>` HASH with the write-lock format without `acq_id`/`hst_id`
105
+ (`ts`, `ini_ttl`, `meta`, `spc_*` reentrant counters; acquirer and host are derived from the acquirer id and are
106
+ added by `lock_info`/`locks_info` as the first reader fields), TTL = read lock TTL; data carrier only (the
104
107
  registry decides existence). Recreated (DEL + HSET) on each read acquisition.
105
108
  - Reentrancy (same acquirer): read under own write/read lock -> `conflict_strategy` as usual
106
109
  (`held_rw_mode` decides which lock is extended/decreased/released); read->write upgrade ->
@@ -110,21 +113,35 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
110
113
  - Try results (internal symbols): `:write_request_is_ahead`, `:read_request_is_ahead`,
111
114
  `:read_lock_is_still_acquired`, `:conflict_lock_upgrade`, `:read_lock_is_expired_during_extension`;
112
115
  try success result carries internal `rw_mode` (held mode), not exposed in the public `lock` result.
113
- - Validation (`AcquireLock`): `read_write_mode` must be `:read`/`:write`; read `ttl` must be a positive
114
- Integer (`RedisQueuedLocks::ArgumentError`).
115
- - RW-aware operations: `unlock` (write lock + readers + reader data + both queues, same result shape),
116
+ - Validation (`AcquireLock`): `read_write_mode` must be `:read`/`:write`; `ttl` must be a positive
117
+ Integer in both modes (`RedisQueuedLocks::ArgumentError`; write: `PEXPIRE` with a non-positive ttl would
118
+ delete the lock key in the obtaining `MULTI`, read: the reader expiration is computed from it);
119
+ `LockSeriesPoC` checks the series `ttl` the same way before computing the lock ttls (they include
120
+ the retry delay and jitter); `config['default_lock_ttl']` must be positive too; `extend_lock_ttl`
121
+ `milliseconds` must be a positive Integer (a non-positive one would decrease/delete the lock);
122
+ `meta` keys must not be reserved lock data keys, String or Symbol (Symbol keys are written as
123
+ Strings: `Symbol#name`; the same check in `AcquireLock` and `LockSeriesPoC`); the other per-call
124
+ numeric options are validated with the ranges of their config keys (`logic.md#7`): `queue_ttl` > 0,
125
+ `timeout`/`retry_count` nil or >= 0, `retry_delay`/`retry_jitter`/`lock_index_grace_period` >= 0
126
+ (`AcquireLock`; `LockSeriesPoC` checks the retry delay/jitter before the lock ttl calculation),
127
+ scan/batch sizes, `dead_ttl`, reindexing `queue_ttl` > 0 (operation modules), `zombie_ttl` and the
128
+ zombie scan sizes > 0 (`Swarm` facade: `SwarmArgumentError`).
129
+ - RW-aware operations: `unlock` (write lock + readers + reader data + both queues + the lock index entry (the enabled lock index), same result shape; one release
130
+ `MULTI` that also reads the readers registry, then one `DEL` of the released readers' data only when readers existed:
131
+ 1 round trip without readers; trade-off: a released reader that obtains a new read lock in between loses its new data),
116
132
  `clear_locks`, `clear_locks_of`/`clear_current_locks` (host derived from acq id via
117
133
  `Resource.host_identifier_from_acquirer`), `clear_dead_requests`, `flush_zombies`/`zombie_*`
118
134
  (zombie readers reported as their registry key), `locked?`, `queued?`, `lock_info` (read info:
119
135
  `'rw_mode' => 'read'`, `'rem_ttl'`, `'readers' => [reader data + rem_ttl]` when no write lock),
120
136
  `queue_info` (`'read_lock_queue'`/`'read_queue'` when present), `locks`/`locks_info`, `queues`/`queues_info`;
121
- write lock info in `lock_info`/`locks_info` has `'rw_mode' => 'write'` (computed on formatting, not stored),
122
- so `'rw_mode'` is a reserved `meta` key (validated in `AcquireLock` and `LockSeriesPoC`);
137
+ write lock info in `lock_info`/`locks_info` has `'lock_key'` and `'rw_mode' => 'write'` (computed on formatting,
138
+ not stored), so `'rw_mode'` is a reserved `meta` key (validated in `AcquireLock` and `LockSeriesPoC`);
123
139
  every request in `queue_info`/`queues_info` is `{ 'acq_id', 'score', 'rw_mode' => 'write'|'read' }`
124
140
  (`queues_info` derives the mode from the queue key via `Resource.lock_queue_rw_mode`).
125
141
  - `lock_series` (both modes): one mode per series; `release_lock_series` releases only locks the series
126
142
  obtained itself (`process == :lock_obtaining`, reentrant ones are kept) and only the current acquirer's:
127
- write = compare-and-delete (`WATCH` + `HGET acq_id` + `DEL`), read = `ZREM` + `DEL` reader data. It runs
143
+ write = compare-and-delete in one transaction per series (`WATCH` the own lock keys + pipelined `HGET acq_id` + `DEL`,
144
+ retried on abort with the keys still owned at the last check), read = `ZREM` + `DEL` reader data. It runs
128
145
  after the block and on any failure (exception or `raise_errors: false`), once per failed series.
129
146
  - Own read lock operations (current acquirer via `current_acquirer_id(identity:)`): `unlock_read` /
130
147
  `release_read_lock` (`ZREM` + `DEL` reader data, event `explicit_read_lock_release`, result
@@ -153,18 +170,65 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
153
170
  | `rql:lock_queue:<name>` | ZSET | acquirer queue scored by enqueue time |
154
171
  | `rql:lock_readers:<name>` | ZSET | read locks: acquirer id => expiration (Redis `TIME`, ms) |
155
172
  | `rql:lock_read_queue:<name>` | ZSET | read lock requests (write requests use `rql:lock_queue:<name>`) |
156
- | `rql:lock_reader:<name>:<acq_id>` | HASH | read lock data (same fields as `rql:lock:<name>` incl. meta; data carrier only) |
173
+ | `rql:lock_reader:<name>:<acq_id>` | HASH | read lock data (same fields as `rql:lock:<name>` incl. meta, without `acq_id`/`hst_id`; data carrier only) |
157
174
  | `rql:swarm:hsts` | HASH | swarm host heartbeats |
175
+ | `rql:index:locks` | ZSET | lock index (the enabled lock index only): lock name => upper bound of the lifetime of all its keys (client time ms + ttl + grace), see "Lock index" |
158
176
 
177
+ - Lock request queues have TTL only with the enabled lock index: `PEXPIRE NX` + `GT` with the queue ttl on each queueing (the queue lives as long as its latest request).
178
+
179
+ - Lock lookup modes of all lock-iterating operations (`locks`/`locks_info`, `queues`/`queues_info`, `keys`, `clear_locks`, `clear_locks_of`/`clear_current_locks`, `clear_dead_requests`, `flush_zombies` and the zombie flusher, `zombie_locks`/`zombie_acquirers`/`zombies_info`), switched by `config['lock_index_enabled']` (client-wide, `false` by default; `Client`/`Swarm` read it and pass it to the modules, the zombie flusher Ractor gets a copy at spawn). Each module keeps both lookups side by side (shared processing in one copy):
180
+ - keyspace (disabled lock index, default): one SCAN pass over `KEY_PATTERN` (or `ALL_LOCK_QUEUES_PATTERN` for queues), keys classified by `Resource.lock_key_family`, per-key commands in pipelined batches collected across SCAN pages (`scan_locks`, `scan_queues`, `each_lock_queue`, `flush_keyspace_zombies`, `each_keyspace_zombie_lock`, inline SCAN branches in `keys`/`clear_locks`/`clear_locks_of`); no lock index commands at all;
181
+ - lock index (enabled): one `ZSCAN` pass over `rql:index:locks` (lock names; names with an expired score are skipped) in batches of `scan_size`/`batch_size` lock names collected across ZSCAN pages, the lock keys are derived from the names (`prepare_lock_key`, `prepare_lock_readers`, both queues, reader data via registry members), per-key commands in pipelined batches; the full keyspace is never scanned. Each module has its own copy of the iteration (`each_lock_name_batch`, duplicated on purpose: module independence; `index_locks`, `index_queues`, `flush_lock_index_zombies`, `each_lock_index_zombie_lock`). Info queries check key existence (`EXISTS`); `clear_locks` deletes the keys of a batch with one `DEL` in a `MULTI` with the `ZREM` of the batch names; `clear_locks`/`clear_dead_requests`/`flush_zombies` prune expired names first.
182
+ - `reindex_locks`: one SCAN pass over `KEY_PATTERN` (keys classified by `Resource.lock_key_family`) that adds keys without index entries to the lock index; it works regardless of the mode (it is needed after enabling the lock index).
159
183
  - Acquirer ID: `rql:acq:<pid>/<thread>/<fiber>/<ractor>/<identity>`
160
184
  - Host ID: `rql:hst:<pid>/<thread>/<ractor>/<identity>` (no fiber)
161
185
 
186
+ ### Lock index (`Resource::LOCK_INDEX_KEY` = `rql:index:locks`)
187
+
188
+ - Opt-in: `config['lock_index_enabled']` (`false` by default), a client-wide mode (not a per-call option: one call without the index
189
+ would break the superset invariant). With the disabled lock index nothing below is sent or read (the pre-index behavior: keyspace
190
+ `SCAN` lookups, queues without TTL, `extend_lock_ttl` write = one `EVAL`); the hot path passes `lock_index_enabled` down
191
+ (`AcquireLock` keyword → `TryToLock` positional before `lock_index_grace_period`) and the lock index commands are conditional
192
+ tails of their pipelines/transactions (their replies are never read: the reply positions do not depend on the mode, except the
193
+ reentrant write extension check `result.size == (lock_index_enabled ? 5 : 4)`). All processes sharing a Redis database must use
194
+ the same mode: locks of processes without the index are invisible to the index lookups (enable everywhere, then `reindex_locks`).
195
+ - ZSET: member = lock name, score = upper bound of the lifetime of all keys of the lock (write lock, readers registry, both queues;
196
+ reader data is found via the registry) in client time: `Resource.lock_index_time` (epoch ms, `CLOCK_REALTIME`) + key ttl +
197
+ `lock_index_grace_period` (config `lock_index_grace_period`, 60_000 ms; tolerance of client clock skew + network latency).
198
+ - Invariant: the index is a superset of the existing lock names (stale names are harmless, a missing name hides a lock from
199
+ lock-iterating operations).
200
+ - every command that creates a lock key is followed by `ZADD GT <now + ttl + grace> <name>` in the same pipeline/transaction,
201
+ after the key command: the request queue in the queueing pipeline (`queue_lock_request`, queue ttl), the lock (write lock /
202
+ registry + reader data) at the end of the obtaining `MULTI` (lock ttl);
203
+ - the obtaining `MULTI` must index the lock itself (atomically): `unlock`/`clear_locks` can drop the name between the queueing and
204
+ the obtaining (`EXPIRE`/`DEL` of an absent lock key does not abort the attempt's `WATCH`); spec `the lock obtaining transaction
205
+ indexes the lock` drops the entry right before the obtaining `MULTI`;
206
+ - every extension of a lock key: `ZADD XX INCR <ms> <name>` in the same transaction (reentrant `:extendable_work_through`, `extend_lock_ttl`
207
+ write (`MULTI { EVAL; ZADD }`), own read, all read locks); `XX`: names without an entry (older versions) are not indexed with a wrong score;
208
+ - the score is never decreased (`GT`; ttl decreases and releases do not touch the index);
209
+ - lock request queues have TTL (`PEXPIRE NX` + `GT` = queue ttl on each queueing): the lifetime of every indexed key is bounded;
210
+ - pruning: `ZREMRANGEBYSCORE <index> -inf <now - 1>` in every queueing pipeline (amortized, no background jobs) and in `flush_zombies`,
211
+ `clear_locks`, `clear_dead_requests`; info operations skip names with an expired score;
212
+ - removal: only together with the deletion of all indexed keys of the name in one `MULTI` (`unlock`: `ZREM` in the release transaction;
213
+ `clear_locks`: `DEL` of the batch keys + `ZREM` of the batch names); never remove a name while some of its keys can stay (`clear_locks_of`,
214
+ `yield_expire` releases, `flush_zombies` keep the names).
215
+ - Migration: keys of older RQL versions and keys created with the disabled lock index have no entries: `reindex_locks` (one SCAN pass:
216
+ `PEXPIRE NX` queue ttl for queues, `PTTL` of locks/registries/queues, `ZADD GT <now + pttl + grace>`; `-1` => `+inf`; idempotent)
217
+ once after the lock index is enabled on all processes. Disabling: the index key is neither used nor maintained (stale, can be deleted).
218
+ - Limitations: completeness needs `grace > clock skew + network latency`; one hot key (fine for standalone/Sentinel; Cluster unsupported anyway);
219
+ hot path cost: 4 more commands per attempt in the existing queueing pipeline + 1 in the obtaining `MULTI` (no new round trips);
220
+ index size ~ distinct lock names per second x (max(queue ttl, lock ttl) + grace): lock-iterating operations visit dead names of this window too.
221
+
162
222
  ### Instrumentation events
163
223
 
164
224
  `redis_queued_locks.` + `lock_obtained`, `reentrant_lock_obtained`,
165
225
  `extendable_reentrant_lock_obtained`, `lock_hold_and_release`, `reentrant_lock_hold_completes`,
166
- `lock_series_obtained`, `lock_series_hold_and_release`, `explicit_lock_release`,
167
- `explicit_all_locks_release`, `release_locks_of`, `explicit_read_lock_release`.
226
+ `lock_obtaining_failed`, `lock_series_obtained`, `lock_series_hold_and_release`, `explicit_lock_release`,
227
+ `explicit_all_locks_release`, `release_locks_of`, `explicit_read_lock_release`, `extend_lock_ttl`, `clear_dead_requests`,
228
+ `reindex_locks`. Observability rule (`logic.md#16`): logic that changes keys is logged and instrumented, the lock
229
+ acquirement per meaningful outcome (visitors; `lock_obtaining_failed` with the failure `result` also before a raised
230
+ acquirement error), each mutation operation with one log line + one event of the whole operation (the same name and data,
231
+ `instrument` last in the payload, inline in the module); lock info queries are neither; the swarm is known debt.
168
232
  Lock events (`lock_obtained`, `reentrant_lock_obtained`, `extendable_reentrant_lock_obtained`,
169
233
  `lock_hold_and_release`, `reentrant_lock_hold_completes`) carry `rw_mode` (requested mode) in the payload;
170
234
  lock series events (`lock_series_obtained`, `lock_series_hold_and_release`) carry the mode of the series;
@@ -173,11 +237,18 @@ lock log lines (incl. lock series logs) carry `rw_mode => '...'`. RW try-lock lo
173
237
 
174
238
  ### Errors (`errors.rb`)
175
239
 
176
- Base `RedisQueuedLocks::Error < StandardError`, plus `ArgumentError`, `LockAlreadyObtainedError`,
177
- `LockAcquirementTimeoutError`, `LockAcquirementRetryLimitError`, `TimedLockTimeoutError`,
178
- `ConflictLockObtainError`, `SwarmError`, `SwarmArgumentError`, `ConfigError`
179
- (`ConfigNotFoundError`, `ConfigValidationError`). Internal `*IntermediateTimeoutError`s
180
- inherit from `Timeout::Error`.
240
+ Global base errors per branch (`logic.md#15`): `RedisQueuedLocks::Error < StandardError` (the main
241
+ hierarchy: `LockAlreadyObtainedError`, `LockAcquirementTimeoutError`, `LockAcquirementRetryLimitError`,
242
+ `TimedLockTimeoutError`, `ConflictLockObtainError`, `SwarmError`, `ConfigError` (`ConfigNotFoundError`,
243
+ `ConfigValidationError`)), `RedisQueuedLocks::ArgumentError < ::ArgumentError` (`SwarmArgumentError`),
244
+ `RedisQueuedLocks::Exception < ::Exception` (`@api private`, super exceptional cases only: the internal
245
+ `*IntermediateTimeoutError`s of `Timeout.timeout` for `timeout` and `timed`; like `Timeout::ExitException`,
246
+ `rescue => e`/`rescue StandardError`/`rescue Timeout::Error`/`rescue nil` of the interrupted code (the user
247
+ block, logger and instrumenter calls) can not swallow them; the library converts them to the public errors/results).
248
+ redis-client sends `UNWATCH` on an interrupted `multi(watch:)` only for `StandardError`s, so every `with` block
249
+ around a `multi(watch:)` (`TryToLock`, `ExtendLockTTL` read extensions, `LockSeriesPoC` write release) rescues
250
+ both internal timeouts and `Timeout::ExitException` (the user's `Timeout.timeout` without an exception class), closes
251
+ the connection (drops the WATCH state, reconnected on the next use) and re-raises.
181
252
 
182
253
  ---
183
254
 
@@ -205,6 +276,7 @@ inherit from `Timeout::Error`.
205
276
  | TTLs | `default_lock_ttl` (5000 ms), `default_queue_ttl` (15 s), `dead_request_ttl` (1 day, ms) |
206
277
  | Strategies | `default_conflict_strategy` (`:wait_for_lock`), `default_access_strategy` (`:queued`) |
207
278
  | Batching | `lock_release_batch_size` (100), `clear_locks_of__lock_scan_size` / `__queue_scan_size` (300), `key_extraction_batch_size` (500) |
279
+ | Lock index | `lock_index_enabled` (false; client-wide mode, config only), `lock_index_grace_period` (60_000 ms; also a `lock`/`lock!`/`lock_series`/`lock_series!`/`reindex_locks` option) |
208
280
  | Identity | `uniq_identifier` (lambda → `Resource.calc_uniq_identity`) |
209
281
  | Logging | `logger` (VoidLogger), `log_lock_try`, `log_sampling_enabled`, `log_sampling_percent` (15), `log_sampler` |
210
282
  | Instrumentation | `instrumenter` (VoidNotifier), `instr_sampling_enabled`, `instr_sampling_percent` (15), `instr_sampler` |
@@ -227,7 +299,7 @@ lib/redis_queued_locks/
227
299
  resource.rb keys and identities
228
300
  data.rb, errors.rb, utilities.rb, utilities/lock.rb, debugger/, version.rb
229
301
  sig/ RBS mirror of lib/ + sig/vendor stubs (redis_client, active_support, semantic_logger)
230
- spec/ redis_queued_locks_spec.rb (~3.4k lines, integration), spec_helper.rb, setup_simplecov.rb
302
+ spec/ redis_queued_locks_spec.rb (~5.6k lines, integration: shared examples run with both lock lookup modes), spec_helper.rb, setup_simplecov.rb
231
303
  .github/workflows/ tests, lint, typecheck-static, typecheck-runtime
232
304
  bin/console, bin/setup dev scripts
233
305
  Rakefile, Steepfile, rbs_collection.yaml
@@ -241,7 +313,7 @@ Known quirk: `sig/redis_queued_locks/acquier.rbs` is misspelled (should be `acqu
241
313
 
242
314
  ### Runtime
243
315
  - Ruby >= 4.0; stdlib `timeout`, `securerandom`, `logger`; Thread, Fiber, Ractor, Mutex
244
- - Redis server: WATCH/MULTI, EVAL (Lua), ZSET, HASH, PTTL/PEXPIRE, SCAN
316
+ - Redis server: WATCH/MULTI, EVAL (Lua), ZSET, HASH, PTTL/PEXPIRE (NX/GT: Redis >= 7), SCAN/ZSCAN
245
317
  - `redis-client ~> 0.20`
246
318
  - Optional, duck-typed: any `::Logger`-compatible logger; any `#notify(event, payload)` instrumenter (ActiveSupport adapter included; SemanticLogger has only an RBS stub)
247
319
 
@@ -273,10 +345,11 @@ Known quirk: `sig/redis_queued_locks/acquier.rbs` is misspelled (should be `acqu
273
345
  ## 6. Conventions
274
346
 
275
347
  - `# frozen_string_literal: true` in every Ruby file.
276
- - YARD tags on every class/method: `@api public|private`, `@since`, `@version` (bump `@version` when changing behavior).
348
+ - YARD tags on every class/method: `@api public|private`, `@since`, `@version` (methods: bump on a behavior/signature change; classes/modules: only on an interface change: signatures, added/removed methods, `extend`/`include`/`prepend` or the superclass; `logic.md#2`).
277
349
  - Every `lib/` change gets a matching `sig/` RBS update; use `# steep:ignore` only where Steep can't infer types.
278
350
  - New operation: new `Acquirer::*` module, thin `Client` method (+ `!` variant if it should raise), RBS, spec.
279
351
  - New config option: `setting` (+ `validate`) in `config.rb`, read via `config['key']`, document defaults.
280
352
  - Development gems go in `Gemfile` (`Gemspec/DevelopmentDependencies: Gemfile`), not the gemspec.
281
353
  - This overview and `.claude/rules/*.md` are updated in the same change as the code (keys, options, results, events, algorithm steps, limitations); README and CHANGELOG `[Unreleased]` for user-visible behavior (log keys and instrumentation payloads: README `## Logging` / `### Instrumentation Events`).
282
354
  - Temporary debt: rspec-retry, disabled coverage minimum, runtime type-check job that can't fail.
355
+ - Specs (`spec/**/*_spec.rb`) are not ideal and will be totally reworked (header `NOTE` of `spec/redis_queued_locks_spec.rb`): until then any coding principle may be broken there; code review checks only that the library behavior is covered and that changed expectations are correct (`.claude/skills/rql-code-review`).
@@ -9,45 +9,62 @@ paths:
9
9
  ## Observed style and patterns
10
10
  - **Module shape**: `module RedisQueuedLocks::Acquirer::<CamelName>` in `acquirer/<snake_name>.rb`, `# @api private`, a single public entry function inside `class << self` named after the file (`ReleaseLock.release_lock`, `IsLocked.locked?`), helpers under `private` in the same `class << self`.
11
11
  - **Utilities**: modules that time or instrument `extend RedisQueuedLocks::Utilities` (gives `clock_gettime`, `run_non_critical`).
12
- - **Independence of operation modules** (core principle): the modules behind `Client` public methods that don't acquire locks (`ClearDeadRequests`, `ExtendLockTTL`, `IsLocked`, `IsQueued`, `Keys`, `LockInfo`, `Locks`, `QueueInfo`, `Queues`, `ReleaseAllLocks`, `ReleaseLock`, `ReleaseLocksOf`, `ReleaseReadLock`) never call each other and never reference each other's RBS types. They share logic only through `Resource`, `Utilities` and other non-operation helpers; similar logic is duplicated on purpose (independence wins over DRY):
13
- - `Locks.extract_locks_info` / `Locks.read_lock_info` duplicate the write/read lock formatting of `LockInfo.lock_info` / `LockInfo.read_lock_info`;
12
+ - **Independence of operation modules** (core principle): the modules behind `Client` public methods that don't acquire locks (`ClearDeadRequests`, `ExtendLockTTL`, `IsLocked`, `IsQueued`, `Keys`, `LockInfo`, `Locks`, `QueueInfo`, `Queues`, `ReindexLocks`, `ReleaseAllLocks`, `ReleaseLock`, `ReleaseLocksOf`, `ReleaseReadLock`) never call each other and never reference each other's RBS types. They share logic only through `Resource`, `Utilities` and other non-operation helpers; similar logic is duplicated on purpose (independence wins over DRY):
13
+ - `Locks.extract_locks_info` / `Locks.read_lock_info` duplicate the write/read lock formatting of `LockInfo.lock_info` / `LockInfo.read_lock_info` (`Locks` fetches the data of a batch of locks in two round trips and `Locks.read_lock_info` formats the prefetched read lock data);
14
14
  - `Queues.extract_queues_info` duplicates the request formatting of `QueueInfo.queue_info`;
15
15
  - each module declares its own RBS aliases (`Locks::lockInfo` / `Locks::readerInfo` mirror `LockInfo::lockInfo` / `LockInfo::readerInfo`); only `Client` signatures reference them.
16
16
  - `AcquireLock` is the lock acquisition core, not an operation module of this list.
17
17
  - PoC modules (`*PoC`, e.g. `LockSeriesPoC`) are not covered by this principle: they may reuse operation modules and the `AcquireLock` core (`LockSeriesPoC` uses `AcquireLock.acquire_lock` and its `YieldExpire` mixin).
18
18
  - Known debt (don't copy): `AcquireLock::WithAcqTimeout` calls `LockInfo.lock_info` / `QueueInfo.queue_info` for detailed timeout errors (its TODO asks to make `AcquireLock` independent of them).
19
+ - Known limitation (kept by the user's decision): `ReleaseLocksOf` (and `FlushZombies`) check the lock owner (`HMGET`) and delete the lock (`DEL`) in two round trips without `WATCH`: a lock re-acquired in between by another acquirer can be deleted.
20
+ - Known limitation (kept by the user's decision, documented in README `#lock`): `YieldExpire` releases the write lock (`EXPIRE 0`) and decreases its ttl (Lua) without an owner check: a block (without `timed`) longer than the lock ttl can delete/shorten the lock of another acquirer. Read lock release/decrease is per acquirer (owner-safe).
19
21
  - **Signatures** (see `arguments.md` for the design rationale):
20
22
  - Read-only queries: few positional args `(redis_client, lock_name)`; collection queries use keywords `(redis_client, scan_size:, with_info:)`.
21
23
  - Mutating operations: long positional lists ending with the fixed observability tail
22
24
  `logger, instrumenter, instrument, log_sampling_enabled, log_sampling_percent, log_sampler, log_sample_this, instr_sampling_enabled, instr_sampling_percent, instr_sampler, instr_sample_this`
23
- (order of `logger`/`instrumenter` varies between modules; check the existing signature).
25
+ (the same order in all mutating modules).
24
26
  - Only `AcquireLock.acquire_lock` and `LockSeriesPoC.lock_series_poc` use keyword args (`process_id:`, `thread_id:`, ...).
25
- - **Two-layer mutating operations** (`release_lock`, `release_read_lock`, `release_all_locks`, `release_locks_of`):
26
- 1. `rel_start_time = clock_gettime`
27
- 2. call a private `fully_*` helper returning `{ ok:, result: }` and destructure it: `fully_x(...) => { ok:, result: }`
27
+ - **Two-layer mutating operations** (`release_lock`, `release_read_lock`, `release_all_locks`, `release_locks_of`, `extend_lock_ttl`, `clear_dead_requests`, `reindex_locks`; observability rule: `logic.md#16`):
28
+ 1. argument validations (`RedisQueuedLocks::ArgumentError`) first, then `rel_start_time = clock_gettime` (`ext_`/`idx_` prefixes for the extension/reindexing)
29
+ 2. call a private `fully_*` helper with the Redis work (returns `{ ok:, result: }`, destructured with `fully_x(...) => { ok:, result: }`, or the raw data: a `Set`/`Integer`)
28
30
  3. `time_at = Time.now.to_f`; `rel_time = ((rel_end_time - rel_start_time) / 1_000.0).ceil(2)` (microseconds → ms)
29
- 4. `instr_sampled = RedisQueuedLocks::Instrument.should_instrument?(...)`
30
- 5. `run_non_critical { instrumenter.notify('redis_queued_locks.<event>', { at:, rel_time:, ... }) } if instr_sampled`
31
- 6. return `{ ok: true, result: { ..., rel_time: } }`
31
+ 4. `log_sampled = RedisQueuedLocks::Logging.should_log?(...)`; `instr_sampled = RedisQueuedLocks::Instrument.should_instrument?(...)`
32
+ 5. `run_non_critical { logger.debug { "[redis_queued_locks.<event>] key => value ..." } } if log_sampled`
33
+ 6. `run_non_critical { instrumenter.notify('redis_queued_locks.<event>', { ..., instrument: }) } if instr_sampled` (the same data as the log line, `instrument` last)
34
+ 7. return `{ ok: true, result: { ..., rel_time: } }` (or the result of `fully_*`)
32
35
  - **Results**: always `{ ok: Boolean, result: ... }` for operations; `result` is a Symbol status (`:async_expire_or_no_lock`, `:released`, `:nothing_to_release`) or a Symbol-keyed Hash (`rel_key_cnt`, `tch_queue_cnt`, `rel_time`, `extended_locks_count`). Info queries return a String-keyed Hash / Set or `nil` when absent.
33
36
  - **Redis access**:
34
- - Keys only via `Resource.prepare_lock_key` / `prepare_lock_queue` / `prepare_read_lock_queue` / `prepare_lock_readers` / `prepare_read_lock_key(lock_name, acquirer_id)` and `Resource::*_PATTERN` (`LOCK_PATTERN`, `LOCK_QUEUE_PATTERN`, `READ_LOCK_QUEUE_PATTERN`, `LOCK_READERS_PATTERN`, `READ_LOCK_PATTERN`); `Resource.lock_name_from_readers` / `lock_key_from_readers` map registry keys back.
37
+ - Keys only via `Resource.prepare_lock_key` / `prepare_lock_queue` / `prepare_read_lock_queue` / `prepare_lock_readers` / `prepare_read_lock_key(lock_name, acquirer_id)` and `Resource::*_PATTERN` (`KEY_PATTERN`, `LOCK_PATTERN`, `LOCK_QUEUE_PATTERN`, `READ_LOCK_QUEUE_PATTERN`, `LOCK_READERS_PATTERN`, `READ_LOCK_PATTERN`, `ALL_LOCK_QUEUES_PATTERN`) and `Resource::LOCK_INDEX_KEY`; `Resource.lock_key_family` classifies scanned keys (keyspace lookups, `ReindexLocks`); `Resource.lock_name_from_readers` / `lock_key_from_readers` map registry keys back.
35
38
  - Several independent reads inside one attempt/operation go into one `pipelined` round trip (it works inside `multi(watch:)` too: reads are sent after WATCH).
36
39
  - Raw commands: `redis.call('CMD', ...)` with uppercase string command names and string args (`'0'`, `'-inf'`, `'+inf'`).
37
40
  - Pooled connection: wrap multi-command work in `redis.with do |rconn| ... end`.
38
41
  - Atomic writes: `rconn.multi do |transact| ... end` (or `multi(watch: [lock_key])` in `TryToLock`); batch reads: `pipelined do |pipeline| ... end` then index `result[0]`, `result[1]` into named vars (`hget_cmd_res`, `pttl_cmd_res`).
39
- - Iteration: `scan('MATCH', PATTERN, count: scan_size) { |key| ... }`, collecting into `Set.new.tap { |set| ... }`; deletes are batched by scan size.
42
+ - Internal timeouts (`LockAcquirementIntermediateTimeoutError`, `TimedLockIntermediateTimeoutError`) inherit from `RedisQueuedLocks::Exception` (`::Exception`, `logic.md#15`): redis-client sends `UNWATCH` on an interrupted `multi(watch:)` only for `StandardError`s, so a connection interrupted inside a WATCH block would return to the pool with the active WATCH (the next `EXEC` on it is aborted when a watched key changes). Every `redis.with do |rconn| ... end` block with a `multi(watch:)` (`TryToLock`, `ExtendLockTTL` read extensions, `LockSeriesPoC.release_write_lock_series`) ends with `rescue <both internal timeouts>, ::Timeout::ExitException` (the user's `Timeout.timeout` without an exception class interrupts with it the same way) → `rconn.close` → `raise`.
43
+ - Iteration: two lock lookup modes, switched by `config['lock_index_enabled']` (client-wide, `false` by default): `Client` (and the `Swarm` facade, the zombie flusher Ractor via a copied value) passes `lock_index_enabled` to every lock-iterating module (`Locks`, `Queues`, `Keys`, `ReleaseAllLocks`, `ReleaseLocksOf`, `ClearDeadRequests`, `FlushZombies`, `ZombieInfo`). Each module keeps both lookups side by side and shares the processing (lock info formatting, flush/release helpers, instrumentation) in one copy; the lookup is chosen in one place (`lock_index_enabled ? index_x : scan_x` or one `if lock_index_enabled ... else ... end` with `(lock index)` / `(keyspace)` step comments):
44
+ - keyspace (the disabled lock index, the pre-index code): one SCAN pass over `KEY_PATTERN` (or `ALL_LOCK_QUEUES_PATTERN` for queues) per operation, keys classified by `Resource.lock_key_family`, per-key commands in pipelined batches collected across SCAN pages (`scan_locks`, `scan_queues`, `each_lock_queue`, `flush_keyspace_zombies`, `each_keyspace_zombie_lock`); no lock index command is sent;
45
+ - lock index (enabled): one `ZSCAN` pass over the lock index per operation (`each_lock_name_batch(rconn, scan_size) { |lock_names| ... }`: `zscan(Resource::LOCK_INDEX_KEY, count: scan_size)`, names with an expired score are skipped, batches of `scan_size` names collected across ZSCAN pages, the rest yielded after the pass); keys are derived from the names (`Resource.prepare_*`, reader data via registry members) and per-key commands are sent in one pipeline (or one variadic command / `MULTI`) per batch; info queries check existence (`EXISTS`, empty `ZRANGE` = missing registry/queue); the keyspace is never scanned (only `ReindexLocks` scans `KEY_PATTERN` with `Resource.lock_key_family` to adopt keys without index entries, in both modes). Each operation module has its own `each_lock_name_batch` copy (same name and shape, duplicated on purpose; the swarm modules iterate inline: `flush_lock_index_zombies`, `each_lock_index_zombie_lock`).
46
+ - Never one round trip per key in either mode.
40
47
  - Lua: frozen heredoc constant (`<<~LUA_SCRIPT.strip.tr("\n", '').freeze`) + `call('EVAL', SCRIPT, 1, key, arg)`.
41
48
  - Release = `EXPIRE key 0` / `ZREMRANGEBYSCORE queue -inf +inf`; a read lock is released with `ZREM <readers> <acq_id>` + `DEL <reader data>` (owner-safe, other readers untouched); Redis TTL sentinels handled explicitly (`PTTL` `-2` = missing, `-1` = no expiry → `Float::INFINITY`).
49
+ - `ReleaseLock` (`unlock`): one `MULTI` with `ZRANGE <readers>` first (the snapshot of the released read locks, read by position) and the release commands, then one `DEL` of the snapshot readers' data only when it is not empty (1 round trip without readers). Trade-off kept by the user's decision: a snapshot reader that obtains a new read lock between the two round trips loses its new read lock data (the read lock itself stays: the registry is the source of truth).
42
50
  - Readers registry members are live only while `score > Redis TIME (ms)` (`Resource.redis_time_ms`); expired members can stay till cleanup, so info/checks must filter by score, never by `ZCARD`/`EXISTS` alone.
51
+ - **Lock index** (`Resource::LOCK_INDEX_KEY` = `rql:index:locks`, ZSET lock name => upper bound of the lifetime of all lock keys in client ms: `Resource.lock_index_time + ttl + lock_index_grace_period`), only with the enabled lock index (`lock_index_enabled`; with the disabled one no command below is sent: the pre-index behavior): the index must stay a superset of the existing lock names (a missing name hides a lock from all lock-iterating operations):
52
+ - the mode is client-wide (`config['lock_index_enabled']` read by `Client`, never a per-call keyword): the hot path gets it as `lock_index_enabled` (`AcquireLock` required keyword → `TryToLock` positional before `lock_index_grace_period`; `ExtendLockTTL`/`ReleaseLock` positional after the lock data); every lock index command is a conditional **tail** of its pipeline/transaction (`return unless lock_index_enabled` at the end of `queue_lock_request`, `if lock_index_enabled` around the last `MULTI` command), so reply positions never depend on the mode (only the reentrant write extension checks `result.size == (lock_index_enabled ? 5 : 4)`);
53
+ - a command that **creates** a lock key is followed by `ZADD <index> GT <now + ttl + grace> <lock_name>` in the same pipeline/transaction, **after** the key command: the request queue `ZADD NX` in the queueing pipeline (queue ttl), the write lock `HSET` / registry `ZADD` + reader data at the end of the obtaining `MULTI` (lock ttl); the obtaining `MULTI` must index the lock itself: `unlock`/`clear_locks` can drop the name between the queueing and the obtaining (`EXPIRE`/`DEL` of an absent key does not abort the attempt's `WATCH`);
54
+ - a name is **removed** only together with the deletion of all its indexed keys in one `MULTI` (`ReleaseLock`: `ZREM` in the release transaction; `ReleaseAllLocks`: `DEL` of the batch keys + `ZREM` of the batch names); operations that can leave keys of a name (`ReleaseLocksOf`, `YieldExpire` releases, `FlushZombies`) never remove names;
55
+ - a command that **extends** a lock key is followed by `ZADD <index> XX INCR <ms> <lock_name>` in the same transaction (`XX`: names without an entry are not indexed with a wrong score); ttl decreases and releases never touch the index (the bound stays an upper bound);
56
+ - lock request queues get `PEXPIRE NX` + `PEXPIRE GT` with the queue ttl (ms, `(queue_ttl * 1_000).ceil`) on each queueing: every indexed key has a bounded lifetime;
57
+ - pruning is `ZREMRANGEBYSCORE <index> -inf <lock_index_time - 1>` (each queueing, `flush_zombies`, `clear_locks`, `clear_dead_requests`); keys written without the index (older versions, raw writes) are adopted by `ReindexLocks` (SCAN pass, `PEXPIRE NX` for queues, `PTTL` => `ZADD GT`).
58
+ - Redis never keeps empty sorted sets/hashes: an empty `ZRANGE`/`HGETALL` result already means a missing key, so don't pair such reads with `EXISTS` (`QueueInfo`, `Queues`).
43
59
  - **Data normalization**: Redis hash strings are converted with `Float(...)` / `Integer(...)` inside `hget_cmd_res.tap do |lock_data| ... end`, optional fields guarded with `if lock_data['x']`.
44
60
  - **AcquireLock**:
45
61
  - Main module `require_relative`s its parts, then `extend`s the mixins (`TryToLock`, `DelayExecution`, `YieldExpire`, `WithAcqTimeout`, `DequeueFromLockQueue`); mixins are plain modules with instance methods (`def try_to_lock(...)`), visitors are `class << self` modules called explicitly (see `visitors.md`).
46
62
  - The algorithm is a numbered step script (`# Step 0`, `# Step 2.1`, `# Step 2.2.a`) driven by a mutable `acq_process` hash (`:should_try`, `:tries`, `:acquired`, `:result`, `:lock_info`, timings).
47
63
  - Failure modes are Symbols (`:fail_fast_no_try`, `:fail_fast_after_try`, `:conflict_dead_lock`, `:conflict_lock_upgrade`, ...); exceptions are raised only when `raise_errors` is true, with a message naming the lock key / acquirer id.
48
- - `TryToLock` attempt (both modes): `multi(watch:)` (write: `[lock_key, lock_readers_key]`, read: `[lock_key]`) → one pipelined state read (`HGET acq_id`, `TIME`, own reader score, longest reader) → same-process conflict detection (`sp_conflict_rw_mode` = held `:write`/`:read`) → `ZADD NX` into the queue of the requested mode → prune dead requests in **both** queues → queue heads of both queues (`:queued`: write must be first in its queue; any mode waits for an earlier `(score, acq_id)` request of the opposite mode) → lock state (write held → `:lock_is_still_acquired`; live readers for write / own live read for read → `:read_lock_is_still_acquired`) → MULTI (write: `ZREM` + `HSET` + `PEXPIRE`; read: `ZREM` + prune registry + `ZADD expiry` + registry `PEXPIRE NX`+`GT` + reader data `DEL`+`HSET`+`PEXPIRE`).
64
+ - `TryToLock` pre-check (retries: `acquire_lock` passes `retry_attempt = tries > 0`, and `fail_fast` attempts): `precheck_lock` reads the lock state with the same pipeline (`pipeline_lock_state`; with the queueing for retries, without it for `fail_fast`) without `WATCH`; it returns the failure result (`lock_obstacle` / `:fail_fast_no_try`) and logs the same events as the failed attempt, or `nil` to continue the attempt (lock obtainable or a same-process conflict). The pre-check never obtains the lock: mutual exclusion stays with `WATCH`/`MULTI`.
65
+ - `TryToLock` attempt (both modes): `multi(watch:)` (write: `[lock_key, lock_readers_key]`, read: `[lock_key]`) → one pipelined state read (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only, the write-only tail of the pipeline) → same-process conflict detection (`sp_conflict_rw_mode` = held `:write`/`:read`) → the lock request queueing in one pipelined round trip (`queue_lock_request`: `ZADD NX` into the queue of the requested mode → prune dead requests in **both** queues → queue heads of both queues → (the enabled lock index only) request queue `PEXPIRE NX`+`GT` → lock index `ZADD GT` → lock index pruning (5 replies, 9 with the lock index; new commands go to the end, replies after the queue heads are not read); eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends these commands in the state pipeline right after its head, so `lock {}` = 4 round trips; the queueing log events are still emitted in the classic branch in the same order) (`:queued`: write must be first in its queue; any mode waits for an earlier `(score, acq_id)` request of the opposite mode) → lock state (write held → `:lock_is_still_acquired`; live readers for write / own live read for read → `:read_lock_is_still_acquired`) → MULTI (write: `ZREM` + `HSET` + `PEXPIRE` (+ lock index `ZADD GT`); read: `ZREM` + prune registry + `ZADD expiry` + registry `PEXPIRE NX`+`GT` + reader data `DEL`+`HSET`+`PEXPIRE` (+ lock index `ZADD GT`)). With the enabled lock index the `:extendable_work_through` extension `MULTI` ends with the lock index `ZADD XX INCR` (write: 4 replies, 5 with the lock index: `result.size == (lock_index_enabled ? 5 : 4)`).
49
66
  - The try result carries internal `rw_mode` (the **held** lock mode); `acquire_lock` stores it as `acq_process[:held_rw_mode]` and passes it to `yield_expire`, which releases/decreases the held lock. The public `lock` result shape does not include it.
50
- - Read lock data (`rql:lock_reader:<name>:<acq_id>`) uses the write-lock field format (`acq_id`, `hst_id`, `ts`, `ini_ttl`, meta, `spc_*`) and the read lock TTL; it is a data carrier only (writers never read it).
67
+ - Read lock data (`rql:lock_reader:<name>:<acq_id>`) uses the write-lock field format without `acq_id`/`hst_id` (`ts`, `ini_ttl`, meta, `spc_*`: both ids are derived from the acquirer id, `LockInfo.read_lock_info`/`Locks.read_lock_info` put them first in each reader and still accept older data that contains them) and the read lock TTL; it is a data carrier only (writers never read it). Its key (`read_lock_key`) is built only for read lock requests (`nil` for write requests: they never hold read locks).
51
68
  - Timing via monotonic `clock_gettime` (microseconds); wall time only for `at:`/`ts` fields.
52
69
  - **Inline typing**: `# @type var result: Symbol`, `{} #: Hash[String,String|Float|Integer]`, and `# steep:ignore` on pattern-matching destructures and splats (`rconn.call('DEL', *keys) # steep:ignore`).
53
70
 
@@ -56,26 +73,28 @@ paths:
56
73
  2. Read-only query → small positional args or `scan_size:` / `with_info:` keywords, return data or `nil`. Mutating operation → follow the two-layer pattern (public timing/instrumentation wrapper + private `fully_*` helper).
57
74
  3. Keep the observability tail parameters in the established order of the closest sibling module; pass them through unchanged from `Client`.
58
75
  4. Return `{ ok:, result: }`; use Symbol statuses for simple outcomes and abbreviated Symbol keys (`rel_*`, `*_cnt`, `*_time`) consistent with existing results. Don't raise from acquirer modules except through the `raise_errors` path in `AcquireLock`.
59
- 5. Use `redis.with { |rconn| ... }` for any multi-command work; use `multi` for writes that must be atomic, `pipelined` for independent reads, `multi(watch:)` or Lua when a write depends on a read.
76
+ 5. Use `redis.with { |rconn| ... }` for any multi-command work; use `multi` for writes that must be atomic, `pipelined` for independent reads, `multi(watch:)` or Lua when a write depends on a read. A new `multi(watch:)` gets the timeouts `rescue` (both internal timeouts and `Timeout::ExitException`: close the connection, re-raise) in its `with` block (see "Internal timeouts" above); never make the internal timeouts `StandardError`s again (the interrupted code would swallow them).
60
77
  6. Use uppercase string Redis commands and string numeric args; handle `PTTL`/`TTL` sentinel values explicitly.
61
- 7. Iterate keys with `scan('MATCH', Resource::*_PATTERN, count:)`, never `KEYS`; batch deletes by scan size.
78
+ 7. Iterate locks in one pass with both lookups (see "Iteration"): a new lock-iterating operation takes `lock_index_enabled` (from `Client`: `config['lock_index_enabled']`) and implements the keyspace lookup (one SCAN pass over `KEY_PATTERN`/`ALL_LOCK_QUEUES_PATTERN`, `Resource.lock_key_family`) and the lock index lookup (one `ZSCAN` pass, `each_lock_name_batch`, keys derived from the lock names) next to each other, sharing the rest of the logic in one copy; never `KEYS`, never a keyspace `SCAN` with the enabled lock index (except `ReindexLocks`); send per-key commands in pipelined batches of `scan_size` keys/lock names (plus the rest after the pass), never one round trip per key; cover the remainder batches of both modes in specs (`lock-iterating operations: one pass (keyspace or lock index) with small batches` runs in both modes) and assert the absence of `SCAN` (lock index) / `ZSCAN` and lock index commands (keyspace) in recorded-command specs (`lock-iterating operations use the lock index`, `the disabled lock index (default): ...`).
62
79
  8. Measure durations with `clock_gettime` and report ms with `/ 1_000.0).ceil(2)`; use `Time.now.to_f` only for event timestamps.
63
80
  9. Wrap every `instrumenter.notify` / logger call in `run_non_critical` (or a visitor) and gate it with `Instrument.should_instrument?` / `Logging.should_log?`; observability must never break locking. Payload changes of inline `notify` calls are documented in README `### Instrumentation Events` in the same change (see `visitors.md` rule 14).
64
81
  10. In `AcquireLock`, add behavior as a new step or mixin rather than growing `acquire_lock`; keep the `# Step N.x` comment numbering, update `acq_process` keys consistently, and add a matching `LogVisitor`/`InstrVisitor` method for each new lifecycle event.
65
- 11. When normalizing lock hash fields, follow the `Float()` / `Integer()` conversion pattern; if a new lock field is added, update every copy of the formatting: write locks in `LockInfo.lock_info` and `Locks.extract_locks_info`, readers in `LockInfo.read_lock_info` and `Locks.read_lock_info` (reader data uses the same fields).
82
+ 11. When normalizing lock hash fields, follow the `Float()` / `Integer()` conversion pattern; if a new lock field is added, update every copy of the formatting: write locks in `LockInfo.lock_info` and `Locks.extract_locks_info`, readers in `LockInfo.read_lock_info` and `Locks.read_lock_info` (reader data uses the same fields except `acq_id`/`hst_id`, which are derived from the acquirer id). Don't store derivable values in lock data.
66
83
  12. Read/write checklist for any change in this directory:
67
84
  - lock state checks: write requires no write lock and no **live** readers; read requires no write lock; same-acquirer reader/writer cases go through `conflict_strategy` (read→write upgrade never "works through");
68
85
  - WATCH: writers watch the write key + readers registry, readers watch the write key only (never the registry: readers must not abort each other); don't modify a watched key outside MULTI after WATCH (it aborts your own EXEC);
69
86
  - every attempt prunes dead requests in both queues; dequeue/cleanup removes the acquirer from the queue of its mode;
70
87
  - every release/cleanup/info/zombie path covers write lock, readers registry, reader data and both queues;
71
88
  - registry/reader-data TTLs: `PEXPIRE NX` + `PEXPIRE GT` pair to keep max TTL (`GT` alone never sets TTL on a key without one);
72
- - keep write-only result shapes unchanged; add read keys/fields only when read data exists.
73
- 13. Values used only for logs (e.g. `HGETALL` of lock data) are fetched only when the log is enabled (`(log_sampled && log_lock_try) ? ... : {}`): visitor arguments are evaluated before the visitor's guard.
89
+ - keep write-only result shapes unchanged; add read keys/fields only when read data exists;
90
+ - lock index (the enabled lock index only, `if lock_index_enabled`): every key-creating command is followed by the index `ZADD GT` (the obtaining `MULTI` too), every extension by `ZADD XX INCR`, names are removed only with all their keys in one `MULTI` (see "Lock index" above); the disabled lock index sends no lock index command.
91
+ 13. Values used only for logs (e.g. `HGETALL` of lock data) are fetched only when the log is enabled (`(log_sampled && log_lock_try) ? ... : NO_LOCK_DATA`, a shared frozen hash instead of a new `{}` per attempt): visitor arguments are evaluated before the visitor's guard.
74
92
  14. Keep operation modules independent (see "Independence of operation modules"; PoC modules such as `LockSeriesPoC` are exempt): never call another operation module's function (public or private) or reference its RBS types from an operation module. When similar logic is needed, duplicate it inside the module (same method name and shape as the original copy, with a `NOTE` that it is duplicated on purpose) or move a pure, data-only helper (key names, id parsing, time conversion) to `Resource` / `Utilities`. Helpers used by one module stay `private`. When changing logic that has copies, update every copy in the same change.
93
+ 15. Pipelines/transactions whose replies are read by position (`lock_state[N]`, `result[N]`): document the reply positions next to the commands (`NOTE: (pipeline layout)`). Conditionally sent commands (e.g. `... unless read_mode`) form a tail after all unconditional commands; read their replies only under the same condition (a command appended after a conditional tail gets a mode-dependent position and its reply is read as another command's reply). Insert new commands before the conditional tail and update the positions, the reply reading, the position constants and the layout spec in the same change. `TryToLock` lock state pipeline (`pipeline_lock_state`, shared by the attempt and the pre-check): head `[LOCK_STATE_OBTAINER_REPLY, LOCK_STATE_REDIS_TIME_REPLY, LOCK_STATE_OWN_READ_LOCK_REPLY]`, queueing commands at `LOCK_STATE_QUEUE_REPLIES_AT` (`queue_lock_request` replies, used ones at `QUEUE_HEAD_REPLY`/`OPPOSITE_QUEUE_HEAD_REPLY`), the write-only tail last (`lock_state.last`); read replies only by these constants (both reply readers: `try_to_lock` Step 0.1 and `precheck_lock`); the obstacle decision and its exit logs are shared too (`find_opposite_request_ahead`, `lock_obstacle`, `log_lock_obstacle`): change them in one place; spec `lock state pipeline layout` records the pipelines of both modes with and without eager queueing and fails on a broken layout or stale position constants.
94
+ 16. Lock index (see "Lock index" above): a new command that creates a lock key must be followed (with the enabled lock index: `if lock_index_enabled`, as a tail of the pipeline/transaction) by the index `ZADD <LOCK_INDEX_KEY> GT <Resource.lock_index_time + lifetime + lock_index_grace_period> <lock_name>` in the same pipeline/transaction (after the key command), a new extension by `ZADD XX INCR <ms> <lock_name>`; a new name removal is allowed only in the same `MULTI` that deletes all indexed keys of the name; a new key family with its own lifetime must get a bounded TTL (the enabled lock index) and be covered by the index score and by `ReindexLocks`; never decrease index scores and never derive mutual exclusion from the index; never send lock index commands with the disabled lock index. Cover new index writes in `describe 'lock index'` specs (they enable the lock index explicitly).
75
95
 
76
96
  ## Recommendations (proposed, not yet project policy)
77
97
  Apply to new or touched code; don't refactor existing code for these unless asked.
78
98
  1. Load Lua scripts once (`SCRIPT LOAD` + `EVALSHA`, falling back to `EVAL` on `NOSCRIPT`), as the TODO in `extend_lock_ttl.rb` asks.
79
- 2. Prefer indexed lookups over full `SCAN` loops for new features (see TODOs in `release_locks_of.rb`, `locks.rb`, `queues.rb`).
99
+ 2. Measure lock-iterating operations by the lock index size (distinct lock names x (max(queue ttl, lock ttl) + grace)): most of the visited names are dead; prefer one cheap existence check per name (`EXISTS` with several keys) before reading the details when adding new iterating features.
80
100
  3. Treat `lock_series_poc.rb` as experimental: don't build new features on it without asking. Its locks are released only through `release_lock_series` (obtained-by-series locks only, owner-checked, on success and on every failure path).
81
- 4. Unify the observability tail order (`logger, instrumenter` vs `instrumenter, logger` in `release_lock`) when those signatures are next changed.
@@ -50,11 +50,12 @@ This is a deliberate framework-level decision, **not** something to refactor int
50
50
  2. New public option: add it as an explicit keyword to **every** affected `Client` method (`x` and `x!`, e.g. `lock` and `lock!`), with the default taken from `config['...']` (add the `setting` + `validate` first) or a literal for per-call-only flags (`raise_errors: false`, `meta: nil`).
51
51
  3. Forward it explicitly by name through every layer using shorthand (`new_option:`); never forward via `**kwargs`, `...`, or `method(__method__).parameters` tricks.
52
52
  4. Keep defaults only at the public boundary (`Client`); internal functions take required keywords/positionals with no defaults so a missed forward fails loudly.
53
- 5. Internal positional lists: append new params in the established order (subject → domain data → observability tail `logger, instrumenter, instrument, log_sampling_*, log_sample_this, instr_sampling_*, instr_sample_this`) and update every call site and the RBS signature in the same change. Lock key params keep this order: `lock_key, read_write_mode, lock_key_queue, read_lock_key_queue, lock_readers_key, read_lock_key, acquirer_id, host_id, ...` (`yield_expire`: `lock_key, lock_readers_key, read_lock_key, rw_mode, acquirer_id, ...`); compute keys once in `acquire_lock` and pass them down.
53
+ 5. Internal positional lists: append new params in the established order (subject → domain data → observability tail `logger, instrumenter, instrument, log_sampling_*, log_sample_this, instr_sampling_*, instr_sample_this`) and update every call site and the RBS signature in the same change. Lock key params keep this order: `lock_key, read_write_mode, lock_key_queue, read_lock_key_queue, lock_readers_key, read_lock_key, lock_name, acquirer_id, host_id, ...` (`lock_name` is the lock index member; `yield_expire`: `lock_key, lock_readers_key, read_lock_key, rw_mode, acquirer_id, ...`); TTL params: `ttl, queue_ttl, lock_index_enabled, lock_index_grace_period`; compute keys once in `acquire_lock` and pass them down (`read_lock_key` is `nil` for write lock requests). Queueing helpers (`queue_lock_request`, `pipeline_lock_state`) take the lock index params last: `lock_name, queue_ttl, lock_index_enabled, lock_index_grace_period`. Lock-iterating modules take `lock_index_enabled` after their domain data (`scan_size:`/`with_info:` keywords + `lock_index_enabled:`; positional lists: right after the batch sizes / `dead_ttl`, before the observability tail; swarm functions: last).
54
54
  6. Document each new option with its own `@option`/`@param` YARD line (type, meaning, default source) and add it to the RBS method signature as a named param/keyword.
55
55
  7. Keep hot-path code allocation-light: no per-call wrapper objects, no hash merging, no splats to build args, no `tap`/closures purely for argument plumbing; prefer passing existing locals.
56
56
  8. Silence length cops locally (`# rubocop:disable Metrics/MethodLength`) rather than shortening signatures; `Metrics/ParameterLists` is disabled project-wide on purpose.
57
- 9. `lock`-family options exist in four `Client` methods: `lock`, `lock!`, `lock_series`, `lock_series!` (and `LockSeriesPoC.lock_series_poc` forwards them to `AcquireLock.acquire_lock`). A new lock option is added to all of them; `lock_series` keeps one value for the whole series (e.g. `read_write_mode`).
57
+ 9. Client-wide modes are not per-call options: `config['lock_index_enabled']` is read inside the `Client` (and `Swarm`) method bodies and forwarded by name to the acquirers (`lock_index_enabled: config['lock_index_enabled']` / positional `config['lock_index_enabled']`), never added as a keyword of `lock`/`locks`/`clear_locks`/...: one call in another mode would break the lock index invariant (all lock acquirements of a client must maintain the index when its operations read it).
58
+ 10. `lock`-family options exist in four `Client` methods: `lock`, `lock!`, `lock_series`, `lock_series!` (and `LockSeriesPoC.lock_series_poc` forwards them to `AcquireLock.acquire_lock`). A new lock option is added to all of them; `lock_series` keeps one value for the whole series (e.g. `read_write_mode`).
58
59
 
59
60
  ## Recommendations (proposed, not yet project policy)
60
61
  1. For new internal functions with many same-typed params (e.g. several Integers/Booleans in a row), consider required keywords instead of positionals to prevent misordering; keep the list flat and explicit.
data/.claude/rules/git.md CHANGED
@@ -29,9 +29,10 @@ Applies to every commit Claude makes in this repository (no `paths:` scope: alwa
29
29
  **When to commit**
30
30
  7. Never commit on your own. When a step with changes reaches completion, ask the user for permission to commit and wait for the answer:
31
31
  - completion = the feature/functionality required by the prompt is fully implemented (code, RBS, specs, README/CHANGELOG, `.claude/project-overview.md` and rules kept in sync) and checks are green: the touched examples and the full suite (`bundle exec rake rspec`), `bundle exec rake rubocop`, `bundle exec rake steep:check` (no `ERROR`/`FATAL` lines); changes without code (docs, `.claude` rules) need no test run;
32
- - the question contains: the proposed commit subject (following the format rules above), a short recap of what was done in the changes (features/fixes, touched areas, check results) and the list of files that will be committed;
32
+ - pre-commit review: once the step is complete, and only then, run the `rql-code-review` skill on the step's changes (scope `uncommitted`) before the question; never during the implementation or for incomplete/red states. Blocker/Major findings in the step's own changes mean the step is not complete yet: fix them, re-run the affected checks and review the fixes; findings that need the user's decision are reported instead of the question (rule 9);
33
+ - the question contains: the proposed commit subject (following the format rules above), a short recap of what was done in the changes (features/fixes, touched areas, check results), the review verdict with the remaining Minor/Nit findings and the list of files that will be committed;
33
34
  - commit only after an explicit "yes" (or with the subject the user corrects); on "no" leave the changes uncommitted (staging also only on request).
34
- 8. An explicit user request to commit ("commit", "закоммить") is the permission itself: commit right away with the given subject (or a subject built by the rules) without asking again.
35
+ 8. An explicit user request to commit ("commit", "закоммить") is the permission itself: commit right away with the given subject (or a subject built by the rules) without asking again and without a review. A commit authorized in advance for a task ("implement X and commit", "commit and continue") still goes through the pre-commit review of rule 7 when the step is complete.
35
36
  9. Don't propose a commit for incomplete or red states (failing specs, lint/type errors, half-done requirements); report the blocker instead.
36
37
  10. One commit per completed step. Stage only the files of that step (`git status` first); never commit environment artifacts: `rbs_collection.lock.yaml` rewritten by `rbs collection install` (restore it with `git checkout --`), `coverage/`, `.gem_rbs_collection/`, local settings.
37
38
  11. Never commit to `master`: create a feature branch first (named after the feature, e.g. `read-write-locks-realisation`). Never push, amend, rebase, reset or force anything unless the user asks.
@@ -12,7 +12,7 @@ Related rule files (loaded for narrower paths):
12
12
  - `swarm.md`: swarm architecture, element lifecycle, Ractor/Thread coding rules
13
13
 
14
14
  ## Swarm overview (details in `swarm.md`)
15
- - Purpose: zombie-lock elimination. `ProbeHosts` periodically `HSET`s every host id (`rql:hst:<pid>/<thread>/<ractor>/<identity>`) with `Time.now.to_f` into `rql:swarm:hsts`; `FlushZombies` treats hosts older than `zombie_ttl` (ms) as zombies and deletes their write locks, their read locks (registry members + reader data), their entries in both request queues and the hosts themselves.
15
+ - Purpose: zombie-lock elimination. `ProbeHosts` periodically `HSET`s every host id (`rql:hst:<pid>/<thread>/<ractor>/<identity>`) with `Time.now.to_f` into `rql:swarm:hsts`; `FlushZombies` treats hosts older than `zombie_ttl` (ms) as zombies and deletes their write locks, their read locks (registry members + reader data), their entries in both request queues and the hosts themselves; locks are found via one keyspace `SCAN` pass (default) or via the lock index (`rql:index:locks`, `ZSCAN`) with the enabled `config['lock_index_enabled']`.
16
16
  - `Client#swarm` → `Swarm` facade (one per client) owns a `Supervisor` and the swarm elements; started by `swarmize!` / `swarm.auto_swarm`, stopped by `deswarmize!`.
17
17
  - Elements are independent background units: a control unit (`SwarmElement::Threaded` = Thread + `SizedQueue` command channel; `SwarmElement::Isolated` = Ractor driven via its own pair of `Ractor::Port`s: results port created in the main Ractor, where the Swarm and Supervisor live; command port created inside the element Ractor) that spawns, stops and reports on a main-loop Thread with its own Redis connection (`Swarm::RedisClientBuilder`).
18
18
  - `ProbeHosts` is Threaded (it must see the client's ractor threads); `FlushZombies` is Isolated (copied config values only).
@@ -24,35 +24,59 @@ Related rule files (loaded for narrower paths):
24
24
  - Constants are defined compactly: `class RedisQueuedLocks::Acquirer::IsLocked` / `module ...`, never nested `module A; module B`.
25
25
  - Namespace files (`acquirer.rb`, `swarm.rb`, `logging.rb`, `swarm/swarm_element.rb`) only `require_relative` their children; the root `lib/redis_queued_locks.rb` requires the namespaces in dependency order.
26
26
  - YARD doc block on every class, module, constant, attr and method:
27
- `@param name [Type]`, `@option`, `@return [Type]`, blank `#` line, then `@api public|private`, `@since X.Y.Z`, optional `@version X.Y.Z` (latest behavior change).
27
+ `@param name [Type]`, `@option`, `@return [Type]`, blank `#` line, then `@api public|private`, `@since X.Y.Z`, optional `@version X.Y.Z` (methods: the latest behavior/signature change; classes/modules: the latest interface change, see rule 2).
28
28
  - Operations are stateless modules with `class << self` functions; dependencies (`redis_client`, logger, instrumenter, sampling options) are passed as explicit args, never read from globals.
29
29
  - Operation modules (except PoC modules such as `LockSeriesPoC`) are independent of each other: no cross-calls and no shared RBS types between them; common logic lives in `Resource`/`Utilities`, similar logic is duplicated on purpose (see `acquirer.md`, "Independence of operation modules").
30
30
  - Public API results are hashes `{ ok: Boolean, result: Symbol|Hash }` (`Client`, `Acquirer::*`, `Swarm` facade actions); `Client` `!` methods raise `RedisQueuedLocks::*Error`. Internal swarm element APIs return bare scalars/primitives (see `swarm.md`).
31
31
  - `Client` methods only fill defaults from `config['...']` and delegate to `Acquirer::*` / `Swarm`.
32
- - Redis keys come only from `RedisQueuedLocks::Resource.prepare_*` helpers and its `*_PATTERN` / `SWARM_KEY` constants. Key families: write lock `rql:lock:` (`LOCK_PATTERN`), write requests `rql:lock_queue:` (`LOCK_QUEUE_PATTERN`), readers registry `rql:lock_readers:` (`LOCK_READERS_PATTERN`), read requests `rql:lock_read_queue:` (`READ_LOCK_QUEUE_PATTERN`), reader data `rql:lock_reader:<name>:<acq_id>` (`READ_LOCK_PATTERN`); all of them match `KEY_PATTERN` (`rql:lock*`).
32
+ - Redis keys come only from `RedisQueuedLocks::Resource.prepare_*` helpers and its `*_PATTERN` / `SWARM_KEY` constants. Key families: write lock `rql:lock:` (`LOCK_PATTERN`), write requests `rql:lock_queue:` (`LOCK_QUEUE_PATTERN`), readers registry `rql:lock_readers:` (`LOCK_READERS_PATTERN`), read requests `rql:lock_read_queue:` (`READ_LOCK_QUEUE_PATTERN`), reader data `rql:lock_reader:<name>:<acq_id>` (`READ_LOCK_PATTERN`); all of them match `KEY_PATTERN` (`rql:lock*`); `Resource.lock_key_family(key)` classifies a key by its prefix (`:lock`, `:lock_queue`, `:lock_readers`, `:read_lock_queue`, `:read_lock`, `nil`).
33
33
  - New key families use a distinct **prefix** (`rql:lock_<kind>:<name>`), never a suffix of the lock name: the lock name is an arbitrary string, so `rql:lock_queue:<name>:read` would collide with the queue of a lock named `<name>:read`.
34
- - Time: lock expirations that affect safety use Redis time (`PEXPIRE`, or `TIME` via `Resource.redis_time_ms` for reader scores); client `Time.now.to_f` is used only for queue positions, `ts` fields and zombie probes.
34
+ - Lock index `rql:index:locks` (`Resource::LOCK_INDEX_KEY`), opt-in via the client-wide `config['lock_index_enabled']` (`false` by default: lock-iterating operations scan the keyspace, no lock index commands, queues without TTL): a global ZSET of lock names (score = upper bound of the lifetime of all lock keys, client ms + ttl + `lock_index_grace_period`); not matched by `KEY_PATTERN`. With the enabled lock index every key-creating command is followed by the index `ZADD GT` (queueing pipeline, obtaining `MULTI`), every extension by `ZADD XX INCR`, names are removed only with all their keys (`unlock`, `clear_locks`), lock request queues have TTL; all lock-iterating operations use it (`ZSCAN`), only `reindex_locks` scans the keyspace. Every lock-iterating module keeps both lookups (keyspace and lock index) side by side with the shared processing in one copy (details in `acquirer.md`, "Iteration" and "Lock index").
35
+ - Time: lock expirations that affect safety use Redis time (`PEXPIRE`, or `TIME` via `Resource.redis_time_ms` for reader scores); client `Time.now.to_f` is used only for queue positions, `ts` fields and zombie probes; lock index scores use client ms (`Resource.lock_index_time`) with the `lock_index_grace_period` margin (clock skew tolerance; the index never affects mutual exclusion).
35
36
  - Config: `setting('key', default)` + `validate('key') { |val| ... }` in `config.rb`; dotted keys for nested groups (`swarm.flush_zombies.zombie_ttl`); units in a trailing `# NOTE: in milliseconds` comment.
36
- - Errors: subclasses in `errors.rb`, written as `class XError < Error; end` (not `Class.new`) so RBS/Steep can see the superclass.
37
+ - Errors: subclasses in `errors.rb`, written as `class XError < Error; end` (not `Class.new`) so RBS/Steep can see the superclass. Each error inherits from the global base error of its branch: `Error < ::StandardError` (the main hierarchy), `ArgumentError < ::ArgumentError` (`SwarmArgumentError`), `Exception < ::Exception` (the internal timeouts `LockAcquirementIntermediateTimeoutError`, `TimedLockIntermediateTimeoutError`); inside `module RedisQueuedLocks` the bare `Error`/`ArgumentError`/`Exception` are these bases.
37
38
  - Comments use `# NOTE:`, `# TODO:` and `# @type var x: T` / `#: T` for inline type hints.
38
39
  - Rubocop is suppressed locally with `# rubocop:disable Metrics/MethodLength` etc. (paired `enable`), mostly for large algorithm methods.
39
40
  - Shared mutable state is guarded by `RedisQueuedLocks::Utilities::Lock#synchronize`; Ractor code must not capture non-shareable objects (build a fresh Redis client via `Swarm::RedisClientBuilder`).
40
41
 
41
42
  ## Claude rules
42
43
  1. Start new files with `# frozen_string_literal: true` and use compact constant paths.
43
- 2. Add a full YARD block to every new public/private method; new code gets `@since <next version>`; when changing existing behavior, add/bump `@version`.
44
+ 2. Add a full YARD block to every new public/private method; new code gets `@since <next version>`. `@version <next version>`:
45
+ - a method: when its behavior or its signature changes;
46
+ - a class/module header: only when its interface changes:
47
+ - a method signature changes: its parameters (their kinds and names); the default values and the return values are behavior;
48
+ - a method is added or removed (public or private, `attr_*` and aliases too);
49
+ - the ancestors change: `extend`/`include`/`prepend` of a module or a changed superclass (e.g. the internal timeout errors in 1.21.0);
50
+ - behavior changes inside the methods, constants, config settings (`setting`/`validate`: `Config` has no interface changes since 1.13.0) and comments bump only the method tags, never the class/module header;
51
+ - the header keeps the version of the latest interface change; no `@version` when the interface never changed since `@since` (the headers of the versions >= 1.17.0 are aligned with this rule).
44
52
  3. New operation = new `Acquirer::*` module + thin `Client` method delegating to it (+ `!` variant only if it must raise); details in `acquirer.md`.
45
53
  4. Return `{ ok:, result: }` hashes from public API operations (`Client`, `Acquirer::*`, `Swarm` facade actions); raise only in `!` methods or on invalid arguments (`RedisQueuedLocks::ArgumentError`). Internal APIs, and swarm element internals in particular, use bare scalars/primitives (`true`/`false`, String, Symbol, Integer, `nil`, a flat Hash of scalars) without the wrapper.
46
54
  5. Never hardcode `rql:` strings; add a `Resource.prepare_*` helper or constant instead.
47
55
  6. Use WATCH/MULTI or a Lua constant for any read-modify-write on lock/queue keys; never do check-then-set in Ruby without a transaction.
48
- 7. New config option: `setting` with default + `validate` with a type check, both in `config.rb`; read it in `Client` as `config['key']`.
56
+ 7. New config option: `setting` with default + `validate` with a type check and the valid range (TTLs, periods, batch/scan sizes are positive Integers: `val.is_a?(Integer) && val > 0`), both in `config.rb`; read it in `Client` as `config['key']`. A per-call numeric option (a `Client` keyword with a `config` default) is validated with the same type and range at the start of the operation module (the `Swarm` facade for the swarm actions), before any Redis command: `RedisQueuedLocks::ArgumentError` (`SwarmArgumentError` in `Swarm`) with the option name and the value (`` "`:scan_size` argument should be a positive Integer, got 0." ``).
49
57
  8. Logs and instrumentation events go through visitor modules (see `visitors.md`); event names follow `redis_queued_locks.<snake_case>`.
50
58
  9. Keep thread/Ractor safety: guard shared state with `Utilities::Lock`, never share a `RedisClient` across Ractors.
51
- 10. Disable rubocop cops only locally with a matching `rubocop:enable`, and only for Metrics/Layout on large methods.
59
+ 10. Disable rubocop cops only locally with a matching `rubocop:enable`, and only for Metrics/Layout on large methods (the only exception: the line-level `Lint/InheritException` of the base `RedisQueuedLocks::Exception < ::Exception` in `errors.rb`, see rule 15).
52
60
  11. After any change here, update the mirrored `sig/` file (see `type-checking.md`) and add/adjust a spec (see `tests.md`).
53
61
  12. Keep `.claude/project-overview.md` and the affected `.claude/rules/*.md` in sync in the same change (keys, options, results, events, algorithm steps, limitations); read the relevant overview sections before implementing and follow their invariants.
54
62
  13. Read/write awareness: every new or changed operation over locks/queues (release, cleanup, info, zombies, series) must handle the read key family too (see the key families above) and keep the result shape for write-only usage unchanged (add keys/branches only when read data exists).
55
63
  14. Never derive safety from queues or client clocks: mutual exclusion is decided by the lock state checked under WATCH (writers WATCH the write key + readers registry, readers WATCH the write key only), queues and positions only order requests.
64
+ 15. Error hierarchy (`errors.rb`, mirrored in `sig/redis_queued_locks/errors.rbs`):
65
+ - every `RedisQueuedLocks` error inherits from the global base error of its branch, never directly from a Ruby class; the main base is `Error < ::StandardError`;
66
+ - a branch that can not be a part of the main hierarchy (a Ruby core error that our error must stay, e.g. `ArgumentError < ::ArgumentError`) gets its own base error named after the Ruby class (`class TypeError < ::TypeError` for a new one), listed in the `errors.rb` header and in the overview "Errors";
67
+ - errors are built per case on the `StandardError` hierarchy or on the Ruby core errors (through the base of their branch), not on `::Exception`;
68
+ - `Exception < ::Exception` is for super exceptional cases only (internal interrupts that the interrupted user/observability code must not rescue: `rescue => e`, `rescue StandardError`, `rescue nil`); each error of this branch states in its comment why it is not a `StandardError` and where the library rescues it (it never reaches the public API: it is converted to a public error/result); only the base disables `Lint/InheritException` (subclasses use the relative `< Exception`);
69
+ - new public errors are `@api public` and documented in README/CHANGELOG; the `Exception` branch is `@api private`.
70
+ 16. Observability of the logic (logs and instrumentation):
71
+ - every logic that creates, changes or deletes Redis keys has both logs and instrumentation; logic that only reads the lock data (`locked?`, `queued?`, `lock_info`, `queue_info`, `locks`, `queues`, `keys`, `*_info`, zombie queries) has neither;
72
+ - lock acquirement (`AcquireLock`, `LockSeriesPoC`): each meaningful step/outcome is logged and instrumented through the visitors (`visitors.md`): obtained (classic and reentrant), held and released, not obtained (`lock_obtaining_failed`, also before a raised acquirement error); the per-attempt `try_lock.*` steps are logs only (gated by `log_lock_try`), never instrumented (no `notify` per attempt);
73
+ - lock mutation operations (release, extension, cleanup, reindexing): exactly one log line and one instrumentation event per operation with the time of the whole work of the root operation module called by the public `Client` method (the two-layer pattern of `acquirer.md`: the timer wraps the `fully_*` helper), never of its internal parts; the log line and the event have the same name (`redis_queued_locks.<event>`) and the same data in the same order (the log has no `instrument`), the payload ends with the user's `instrument`; they are written inline in the module (`run_non_critical`, gated by `Logging.should_log?` / `Instrument.should_instrument?`);
74
+ - every such public `Client` method takes `logger:`, `instrumenter:`, `instrument:` and the log/instrumentation sampling keywords (defaults from `config`) and forwards them by name to the operation module;
75
+ - known debt: the swarm elements and the manual swarm actions (`ProbeHosts`, `FlushZombies`: `#probe_hosts`, `#flush_zombies`) are neither logged nor instrumented yet (the flusher Ractor can not get the user's logger/instrumenter).
76
+ 17. Debug methods of `Resource`: `Resource` may keep methods that the library does not use when they are justified for debugging: they let you check a subtle part of the logic by hand at runtime (in a console), e.g. `lock_key_from_queue` maps a lock request queue of any mode to its lock. Each of them:
77
+ - has a `# NOTE: (debug)` comment in its YARD block: that it is not used by the library and what it checks (with a console example);
78
+ - is correct for every key family it can get (write and read keys), has the full YARD block, the RBS signature and a spec;
79
+ - is not a dead code finding; an unused method without the `(debug)` note is dead code (remove it); a debug method that the library starts to use loses the note.
56
80
 
57
81
  ## Recommendations (proposed, not yet project policy)
58
82
  Apply to new or touched code; don't refactor existing code for these unless asked.