redis_queued_locks 1.18.0 → 1.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/project-overview.md +65 -13
- data/.claude/rules/acquirer.md +29 -11
- data/.claude/rules/arguments.md +3 -2
- data/.claude/rules/git.md +3 -2
- data/.claude/rules/logic.md +4 -3
- data/.claude/rules/swarm.md +12 -7
- data/.claude/rules/tests.md +6 -0
- data/.claude/rules/type-checking.md +2 -1
- data/.claude/rules/visitors.md +1 -1
- data/.claude/skills/rql-code-review/SKILL.md +372 -0
- data/.claude/skills/rql-code-review/templates/report.en.md +31 -0
- data/.claude/skills/rql-code-review/templates/report.ru.md +31 -0
- data/CHANGELOG.md +74 -0
- data/CLAUDE.md +4 -2
- data/README.md +338 -24
- data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +684 -128
- data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire.rb +3 -1
- data/lib/redis_queued_locks/acquirer/acquire_lock.rb +24 -3
- data/lib/redis_queued_locks/acquirer/clear_dead_requests.rb +145 -18
- data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +71 -10
- data/lib/redis_queued_locks/acquirer/keys.rb +142 -7
- data/lib/redis_queued_locks/acquirer/lock_info.rb +10 -4
- data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +26 -7
- data/lib/redis_queued_locks/acquirer/locks.rb +195 -83
- data/lib/redis_queued_locks/acquirer/queue_info.rb +12 -18
- data/lib/redis_queued_locks/acquirer/queues.rb +114 -50
- data/lib/redis_queued_locks/acquirer/reindex_locks.rb +140 -0
- data/lib/redis_queued_locks/acquirer/release_all_locks.rb +147 -58
- data/lib/redis_queued_locks/acquirer/release_lock.rb +52 -19
- data/lib/redis_queued_locks/acquirer/release_locks_of.rb +235 -66
- data/lib/redis_queued_locks/acquirer.rb +1 -0
- data/lib/redis_queued_locks/client.rb +116 -16
- data/lib/redis_queued_locks/config.rb +5 -1
- data/lib/redis_queued_locks/resource.rb +99 -2
- data/lib/redis_queued_locks/swarm/flush_zombies.rb +332 -61
- data/lib/redis_queued_locks/swarm/zombie_info.rb +203 -53
- data/lib/redis_queued_locks/swarm.rb +15 -4
- data/lib/redis_queued_locks/version.rb +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +107 -1
- data/sig/redis_queued_locks/acquirer/acquire_lock/yield_expire.rbs +1 -1
- data/sig/redis_queued_locks/acquirer/acquire_lock.rbs +2 -0
- data/sig/redis_queued_locks/acquirer/clear_dead_requests.rbs +10 -1
- data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +5 -2
- data/sig/redis_queued_locks/acquirer/keys.rbs +11 -1
- data/sig/redis_queued_locks/acquirer/locks.rbs +15 -6
- data/sig/redis_queued_locks/acquirer/queues.rbs +9 -2
- data/sig/redis_queued_locks/acquirer/reindex_locks.rbs +31 -0
- data/sig/redis_queued_locks/acquirer/release_all_locks.rbs +8 -1
- data/sig/redis_queued_locks/acquirer/release_lock.rbs +3 -1
- data/sig/redis_queued_locks/acquirer/release_locks_of.rbs +20 -1
- data/sig/redis_queued_locks/client.rbs +9 -0
- data/sig/redis_queued_locks/resource.rbs +6 -0
- data/sig/redis_queued_locks/swarm/flush_zombies.rbs +51 -1
- data/sig/redis_queued_locks/swarm/zombie_info.rbs +58 -7
- data/sig/vendor/redis_client.rbs +1 -0
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b84159ef72c995d09109b9a653637c399638329e8b02cda9a271a8eda0856795
|
|
4
|
+
data.tar.gz: 45d47d23c4167b241fe80bc23cc698fbbceda4a6fd67a777f2cb379538d6e91d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '068d7991fc00ffc8118f9e00a816916158fe4f2a962598ef41ffd96b7a4202ebe5cff51c9c6b515ace936cebb0736173b61527c03c774aea5f3794e173a5353d'
|
|
7
|
+
data.tar.gz: bb7541f8c08b1852239366cf43d3de16817a06b36207b2a4c1d2e14a9b6723e1bb7c5d3c8067629b225026d18a83adb9086a41ef590675e26f78888666c66333
|
data/.claude/project-overview.md
CHANGED
|
@@ -49,14 +49,15 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
|
|
|
49
49
|
| `lock` / `lock!` | `acquirer/acquire_lock.rb` + `acquirer/acquire_lock/*` |
|
|
50
50
|
| `lock_series` / `lock_series!` | `acquirer/lock_series_poc.rb` (proof of concept; one `read_write_mode` for the whole series) |
|
|
51
51
|
| `unlock` | `acquirer/release_lock.rb` |
|
|
52
|
-
| `unlock_read` / `release_read_lock` | `acquirer/release_read_lock.rb` (own read lock only) |
|
|
52
|
+
| `unlock_read` / `release_read_lock` | `acquirer/release_read_lock.rb` (own read lock or `acquirer_id:` read lock only) |
|
|
53
53
|
| `extend_lock_ttl` | `acquirer/extend_lock_ttl.rb` (`read_write_mode: :read` extends own read lock, `+ all_read_locks: true` all live read locks) |
|
|
54
54
|
| `locked?` / `queued?` | `acquirer/is_locked.rb` / `acquirer/is_queued.rb` |
|
|
55
55
|
| `lock_info` / `queue_info` | `acquirer/lock_info.rb` / `acquirer/queue_info.rb` |
|
|
56
|
-
| `locks`, `locks_info`, `queues`, `queues_info`, `keys` | `acquirer/locks.rb`, `queues.rb`, `keys.rb` (SCAN
|
|
56
|
+
| `locks`, `locks_info`, `queues`, `queues_info`, `keys` | `acquirer/locks.rb`, `queues.rb`, `keys.rb` (keyspace SCAN or lock index: `lock_index_enabled`) |
|
|
57
57
|
| `clear_locks` | `acquirer/release_all_locks.rb` |
|
|
58
58
|
| `clear_locks_of`, `clear_current_locks` | `acquirer/release_locks_of.rb` |
|
|
59
59
|
| `clear_dead_requests` | `acquirer/clear_dead_requests.rb` |
|
|
60
|
+
| `reindex_locks` | `acquirer/reindex_locks.rb` (adds keys without lock index entries to the lock index: one SCAN pass; works in both modes) |
|
|
60
61
|
| `current_acquirer_id`, `current_host_id`, `possible_host_ids` | `resource.rb` |
|
|
61
62
|
| `swarmize!`, `deswarmize!`, `swarm_status`, `swarm_info`, `probe_hosts`, `flush_zombies`, `zombie_locks`, `zombie_acquirers`, `zombie_hosts`, `zombies_info` | `swarm.rb` + `swarm/*` |
|
|
62
63
|
|
|
@@ -66,16 +67,17 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
|
|
|
66
67
|
|
|
67
68
|
| Mixin | Role |
|
|
68
69
|
|---|---|
|
|
69
|
-
| `TryToLock` (`try_to_lock.rb`) | One attempt: `multi(watch:)` (write: lock key + readers registry, read: lock key only), one pipelined read of the lock state (`HGET acq_id`, `TIME`, own reader score, longest reader), `ZADD NX` into the queue of the requested mode (timestamp score), prune both queues, take the lock if allowed (see Read/Write locks). Handles reentrant conflicts (incl. read/write ones); write TTL extension via inline Lua (`PTTL` + `PEXPIRE`). |
|
|
70
|
+
| `TryToLock` (`try_to_lock.rb`) | One attempt: `multi(watch:)` (write: lock key + readers registry, read: lock key only), one pipelined read of the lock state (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only), the lock request queueing in one pipelined round trip (`queue_lock_request`: `ZADD NX` into the queue of the requested mode (timestamp score), prune both queues, both queue heads, then (the enabled lock index only) the lock index commands: request queue `PEXPIRE NX`+`GT` (queue ttl), `ZADD GT` of the lock name into the lock index (queue ttl), `ZREMRANGEBYSCORE` of expired lock names), take the lock if allowed (see Read/Write locks; with the enabled lock index the obtaining `MULTI` ends with the lock index `ZADD GT` (lock ttl), the `:extendable_work_through` extension `MULTI` with `ZADD XX INCR`). Eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends the queueing commands in the lock state round trip: `lock {}` = 4 round trips (`WATCH`, state + queueing, `MULTI/EXEC`, release), a failed attempt = 3. Pre-check (`precheck_lock`, retries and `fail_fast` attempts): the lock state (+ the queueing for retries) without `WATCH` in one round trip; an attempt that can not obtain the lock fails there (same results and log events: `lock_obstacle`/`log_lock_obstacle`), otherwise the attempt continues with `WATCH`/`MULTI` (a failed retry = 1 round trip, a successful one = 4; `fail_fast` on an obtained lock = 1); same-process conflicts are never decided by the pre-check. Handles reentrant conflicts (incl. read/write ones); write TTL extension via inline Lua (`PTTL` + `PEXPIRE`). |
|
|
70
71
|
| `WithAcqTimeout` | Global acquisition timeout (`timeout`). |
|
|
71
72
|
| `DelayExecution` | Retry delay + jitter between attempts. |
|
|
72
73
|
| `DequeueFromLockQueue` | Removes the acquirer from the queue of the requested mode on timeout/failure. |
|
|
73
|
-
| `YieldExpire` | Runs the user block (optionally `timed`), then releases/expires the lock of the held mode (write: `EXPIRE 0`; read: `ZREM` own reader + `DEL` own read lock data; extendable reentrant: decrease). |
|
|
74
|
+
| `YieldExpire` | Runs the user block (optionally `timed`), then releases/expires the lock of the held mode (write: `EXPIRE 0`; read: `ZREM` own reader + `DEL` own read lock data; extendable reentrant: decrease). Known limitation (kept by the user's decision, documented in README): the write release and the write ttl decrease have no owner check, so a block longer than the ttl can delete/shorten the lock of another acquirer. |
|
|
74
75
|
| `LogVisitor` / `InstrVisitor` | One method per lifecycle event for logs and instrumentation. |
|
|
75
76
|
|
|
76
77
|
### Main `lock` options (defaults from config)
|
|
77
78
|
|
|
78
|
-
`ttl`, `queue_ttl`, `
|
|
79
|
+
`ttl`, `queue_ttl`, `lock_index_grace_period` (used with the enabled lock index only; the mode itself, `config['lock_index_enabled']`,
|
|
80
|
+
is client-wide: `Client` passes it to the acquirers, it is not a `lock` option), `timeout`, `timed`, `retry_count`, `retry_delay`, `retry_jitter`,
|
|
79
81
|
`raise_errors`, `fail_fast`, `conflict_strategy`, `access_strategy`, `read_write_mode`
|
|
80
82
|
(`:write` (default, exclusive) | `:read` (shared)), `identity`, `meta`, `logger`, `log_lock_try`,
|
|
81
83
|
`instrumenter`, `instrument`, log/instr sampling options, `log_sample_this`, `instr_sample_this`.
|
|
@@ -99,8 +101,9 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
|
|
|
99
101
|
write must be the head of the write queue and waits for earlier read requests. Every attempt prunes
|
|
100
102
|
dead requests in both queues. `:random` ignores queues (writers can starve). Ordering is best effort
|
|
101
103
|
(client clocks, `unlock` clears queues); it never affects mutual exclusion.
|
|
102
|
-
- Read lock data: `rql:lock_reader:<name>:<acq_id>` HASH with the write-lock format
|
|
103
|
-
`ts`, `ini_ttl`, `meta`, `spc_*` reentrant counters
|
|
104
|
+
- Read lock data: `rql:lock_reader:<name>:<acq_id>` HASH with the write-lock format without `acq_id`/`hst_id`
|
|
105
|
+
(`ts`, `ini_ttl`, `meta`, `spc_*` reentrant counters; acquirer and host are derived from the acquirer id and are
|
|
106
|
+
added by `lock_info`/`locks_info` as the first reader fields), TTL = read lock TTL; data carrier only (the
|
|
104
107
|
registry decides existence). Recreated (DEL + HSET) on each read acquisition.
|
|
105
108
|
- Reentrancy (same acquirer): read under own write/read lock -> `conflict_strategy` as usual
|
|
106
109
|
(`held_rw_mode` decides which lock is extended/decreased/released); read->write upgrade ->
|
|
@@ -112,7 +115,9 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
|
|
|
112
115
|
try success result carries internal `rw_mode` (held mode), not exposed in the public `lock` result.
|
|
113
116
|
- Validation (`AcquireLock`): `read_write_mode` must be `:read`/`:write`; read `ttl` must be a positive
|
|
114
117
|
Integer (`RedisQueuedLocks::ArgumentError`).
|
|
115
|
-
- RW-aware operations: `unlock` (write lock + readers + reader data + both queues, same result shape
|
|
118
|
+
- RW-aware operations: `unlock` (write lock + readers + reader data + both queues + the lock index entry (the enabled lock index), same result shape; one release
|
|
119
|
+
`MULTI` that also reads the readers registry, then one `DEL` of the released readers' data only when readers existed:
|
|
120
|
+
1 round trip without readers; trade-off: a released reader that obtains a new read lock in between loses its new data),
|
|
116
121
|
`clear_locks`, `clear_locks_of`/`clear_current_locks` (host derived from acq id via
|
|
117
122
|
`Resource.host_identifier_from_acquirer`), `clear_dead_requests`, `flush_zombies`/`zombie_*`
|
|
118
123
|
(zombie readers reported as their registry key), `locked?`, `queued?`, `lock_info` (read info:
|
|
@@ -124,11 +129,13 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
|
|
|
124
129
|
(`queues_info` derives the mode from the queue key via `Resource.lock_queue_rw_mode`).
|
|
125
130
|
- `lock_series` (both modes): one mode per series; `release_lock_series` releases only locks the series
|
|
126
131
|
obtained itself (`process == :lock_obtaining`, reentrant ones are kept) and only the current acquirer's:
|
|
127
|
-
write = compare-and-delete (`WATCH`
|
|
132
|
+
write = compare-and-delete in one transaction per series (`WATCH` the own lock keys + pipelined `HGET acq_id` + `DEL`,
|
|
133
|
+
retried on abort with the keys still owned at the last check), read = `ZREM` + `DEL` reader data. It runs
|
|
128
134
|
after the block and on any failure (exception or `raise_errors: false`), once per failed series.
|
|
129
135
|
- Own read lock operations (current acquirer via `current_acquirer_id(identity:)`): `unlock_read` /
|
|
130
136
|
`release_read_lock` (`ZREM` + `DEL` reader data, event `explicit_read_lock_release`, result
|
|
131
|
-
`{ rel_time:, rel_key:, rel_acq_id:, lock_res: }`
|
|
137
|
+
`{ rel_time:, rel_key:, rel_acq_id:, lock_res: }`; `acquirer_id:` (default `nil` => current acquirer;
|
|
138
|
+
when passed, `identity:` is ignored: the acquirer id already contains its identity) releases the read lock of a different acquirer) and `extend_lock_ttl(..., read_write_mode: :read)`
|
|
132
139
|
(`multi(watch: [lock_key])`: extends only a live read lock while no write lock exists, never revives
|
|
133
140
|
an expired one; registry and reader data TTLs via `PEXPIRE NX`+`GT`). `unlock` releases everything.
|
|
134
141
|
- `extend_lock_ttl(..., read_write_mode: :read, all_read_locks: true)` (`false` by default, ignored for
|
|
@@ -152,12 +159,55 @@ duplicated on purpose (`Locks` duplicates the `LockInfo` write/read lock formatt
|
|
|
152
159
|
| `rql:lock_queue:<name>` | ZSET | acquirer queue scored by enqueue time |
|
|
153
160
|
| `rql:lock_readers:<name>` | ZSET | read locks: acquirer id => expiration (Redis `TIME`, ms) |
|
|
154
161
|
| `rql:lock_read_queue:<name>` | ZSET | read lock requests (write requests use `rql:lock_queue:<name>`) |
|
|
155
|
-
| `rql:lock_reader:<name>:<acq_id>` | HASH | read lock data (same fields as `rql:lock:<name>` incl. meta
|
|
162
|
+
| `rql:lock_reader:<name>:<acq_id>` | HASH | read lock data (same fields as `rql:lock:<name>` incl. meta, without `acq_id`/`hst_id`; data carrier only) |
|
|
156
163
|
| `rql:swarm:hsts` | HASH | swarm host heartbeats |
|
|
164
|
+
| `rql:index:locks` | ZSET | lock index (the enabled lock index only): lock name => upper bound of the lifetime of all its keys (client time ms + ttl + grace), see "Lock index" |
|
|
157
165
|
|
|
166
|
+
- Lock request queues have TTL only with the enabled lock index: `PEXPIRE NX` + `GT` with the queue ttl on each queueing (the queue lives as long as its latest request).
|
|
167
|
+
|
|
168
|
+
- Lock lookup modes of all lock-iterating operations (`locks`/`locks_info`, `queues`/`queues_info`, `keys`, `clear_locks`, `clear_locks_of`/`clear_current_locks`, `clear_dead_requests`, `flush_zombies` and the zombie flusher, `zombie_locks`/`zombie_acquirers`/`zombies_info`), switched by `config['lock_index_enabled']` (client-wide, `false` by default; `Client`/`Swarm` read it and pass it to the modules, the zombie flusher Ractor gets a copy at spawn). Each module keeps both lookups side by side (shared processing in one copy):
|
|
169
|
+
- keyspace (disabled lock index, default): one SCAN pass over `KEY_PATTERN` (or `ALL_LOCK_QUEUES_PATTERN` for queues), keys classified by `Resource.lock_key_family`, per-key commands in pipelined batches collected across SCAN pages (`scan_locks`, `scan_queues`, `each_lock_queue`, `flush_keyspace_zombies`, `each_keyspace_zombie_lock`, inline SCAN branches in `keys`/`clear_locks`/`clear_locks_of`); no lock index commands at all;
|
|
170
|
+
- lock index (enabled): one `ZSCAN` pass over `rql:index:locks` (lock names; names with an expired score are skipped) in batches of `scan_size`/`batch_size` lock names collected across ZSCAN pages, the lock keys are derived from the names (`prepare_lock_key`, `prepare_lock_readers`, both queues, reader data via registry members), per-key commands in pipelined batches; the full keyspace is never scanned. Each module has its own copy of the iteration (`each_lock_name_batch`, duplicated on purpose: module independence; `index_locks`, `index_queues`, `flush_lock_index_zombies`, `each_lock_index_zombie_lock`). Info queries check key existence (`EXISTS`); `clear_locks` deletes the keys of a batch with one `DEL` in a `MULTI` with the `ZREM` of the batch names; `clear_locks`/`clear_dead_requests`/`flush_zombies` prune expired names first.
|
|
171
|
+
- `reindex_locks`: one SCAN pass over `KEY_PATTERN` (keys classified by `Resource.lock_key_family`) that adds keys without index entries to the lock index; it works regardless of the mode (it is needed after enabling the lock index).
|
|
158
172
|
- Acquirer ID: `rql:acq:<pid>/<thread>/<fiber>/<ractor>/<identity>`
|
|
159
173
|
- Host ID: `rql:hst:<pid>/<thread>/<ractor>/<identity>` (no fiber)
|
|
160
174
|
|
|
175
|
+
### Lock index (`Resource::LOCK_INDEX_KEY` = `rql:index:locks`)
|
|
176
|
+
|
|
177
|
+
- Opt-in: `config['lock_index_enabled']` (`false` by default), a client-wide mode (not a per-call option: one call without the index
|
|
178
|
+
would break the superset invariant). With the disabled lock index nothing below is sent or read (the pre-index behavior: keyspace
|
|
179
|
+
`SCAN` lookups, queues without TTL, `extend_lock_ttl` write = one `EVAL`); the hot path passes `lock_index_enabled` down
|
|
180
|
+
(`AcquireLock` keyword → `TryToLock` positional before `lock_index_grace_period`) and the lock index commands are conditional
|
|
181
|
+
tails of their pipelines/transactions (their replies are never read: the reply positions do not depend on the mode, except the
|
|
182
|
+
reentrant write extension check `result.size == (lock_index_enabled ? 5 : 4)`). All processes sharing a Redis database must use
|
|
183
|
+
the same mode: locks of processes without the index are invisible to the index lookups (enable everywhere, then `reindex_locks`).
|
|
184
|
+
- ZSET: member = lock name, score = upper bound of the lifetime of all keys of the lock (write lock, readers registry, both queues;
|
|
185
|
+
reader data is found via the registry) in client time: `Resource.lock_index_time` (epoch ms, `CLOCK_REALTIME`) + key ttl +
|
|
186
|
+
`lock_index_grace_period` (config `lock_index_grace_period`, 60_000 ms; tolerance of client clock skew + network latency).
|
|
187
|
+
- Invariant: the index is a superset of the existing lock names (stale names are harmless, a missing name hides a lock from
|
|
188
|
+
lock-iterating operations).
|
|
189
|
+
- every command that creates a lock key is followed by `ZADD GT <now + ttl + grace> <name>` in the same pipeline/transaction,
|
|
190
|
+
after the key command: the request queue in the queueing pipeline (`queue_lock_request`, queue ttl), the lock (write lock /
|
|
191
|
+
registry + reader data) at the end of the obtaining `MULTI` (lock ttl);
|
|
192
|
+
- the obtaining `MULTI` must index the lock itself (atomically): `unlock`/`clear_locks` can drop the name between the queueing and
|
|
193
|
+
the obtaining (`EXPIRE`/`DEL` of an absent lock key does not abort the attempt's `WATCH`); spec `the lock obtaining transaction
|
|
194
|
+
indexes the lock` drops the entry right before the obtaining `MULTI`;
|
|
195
|
+
- every extension of a lock key: `ZADD XX INCR <ms> <name>` in the same transaction (reentrant `:extendable_work_through`, `extend_lock_ttl`
|
|
196
|
+
write (`MULTI { EVAL; ZADD }`), own read, all read locks); `XX`: names without an entry (older versions) are not indexed with a wrong score;
|
|
197
|
+
- the score is never decreased (`GT`; ttl decreases and releases do not touch the index);
|
|
198
|
+
- lock request queues have TTL (`PEXPIRE NX` + `GT` = queue ttl on each queueing): the lifetime of every indexed key is bounded;
|
|
199
|
+
- pruning: `ZREMRANGEBYSCORE <index> -inf <now - 1>` in every queueing pipeline (amortized, no background jobs) and in `flush_zombies`,
|
|
200
|
+
`clear_locks`, `clear_dead_requests`; info operations skip names with an expired score;
|
|
201
|
+
- removal: only together with the deletion of all indexed keys of the name in one `MULTI` (`unlock`: `ZREM` in the release transaction;
|
|
202
|
+
`clear_locks`: `DEL` of the batch keys + `ZREM` of the batch names); never remove a name while some of its keys can stay (`clear_locks_of`,
|
|
203
|
+
`yield_expire` releases, `flush_zombies` keep the names).
|
|
204
|
+
- Migration: keys of older RQL versions and keys created with the disabled lock index have no entries: `reindex_locks` (one SCAN pass:
|
|
205
|
+
`PEXPIRE NX` queue ttl for queues, `PTTL` of locks/registries/queues, `ZADD GT <now + pttl + grace>`; `-1` => `+inf`; idempotent)
|
|
206
|
+
once after the lock index is enabled on all processes. Disabling: the index key is neither used nor maintained (stale, can be deleted).
|
|
207
|
+
- Limitations: completeness needs `grace > clock skew + network latency`; one hot key (fine for standalone/Sentinel; Cluster unsupported anyway);
|
|
208
|
+
hot path cost: 4 more commands per attempt in the existing queueing pipeline + 1 in the obtaining `MULTI` (no new round trips);
|
|
209
|
+
index size ~ distinct lock names per second x (max(queue ttl, lock ttl) + grace): lock-iterating operations visit dead names of this window too.
|
|
210
|
+
|
|
161
211
|
### Instrumentation events
|
|
162
212
|
|
|
163
213
|
`redis_queued_locks.` + `lock_obtained`, `reentrant_lock_obtained`,
|
|
@@ -204,6 +254,7 @@ inherit from `Timeout::Error`.
|
|
|
204
254
|
| TTLs | `default_lock_ttl` (5000 ms), `default_queue_ttl` (15 s), `dead_request_ttl` (1 day, ms) |
|
|
205
255
|
| Strategies | `default_conflict_strategy` (`:wait_for_lock`), `default_access_strategy` (`:queued`) |
|
|
206
256
|
| Batching | `lock_release_batch_size` (100), `clear_locks_of__lock_scan_size` / `__queue_scan_size` (300), `key_extraction_batch_size` (500) |
|
|
257
|
+
| Lock index | `lock_index_enabled` (false; client-wide mode, config only), `lock_index_grace_period` (60_000 ms; also a `lock`/`lock!`/`lock_series`/`lock_series!`/`reindex_locks` option) |
|
|
207
258
|
| Identity | `uniq_identifier` (lambda → `Resource.calc_uniq_identity`) |
|
|
208
259
|
| Logging | `logger` (VoidLogger), `log_lock_try`, `log_sampling_enabled`, `log_sampling_percent` (15), `log_sampler` |
|
|
209
260
|
| Instrumentation | `instrumenter` (VoidNotifier), `instr_sampling_enabled`, `instr_sampling_percent` (15), `instr_sampler` |
|
|
@@ -226,7 +277,7 @@ lib/redis_queued_locks/
|
|
|
226
277
|
resource.rb keys and identities
|
|
227
278
|
data.rb, errors.rb, utilities.rb, utilities/lock.rb, debugger/, version.rb
|
|
228
279
|
sig/ RBS mirror of lib/ + sig/vendor stubs (redis_client, active_support, semantic_logger)
|
|
229
|
-
spec/ redis_queued_locks_spec.rb (~
|
|
280
|
+
spec/ redis_queued_locks_spec.rb (~4.9k lines, integration: shared examples run with both lock lookup modes), spec_helper.rb, setup_simplecov.rb
|
|
230
281
|
.github/workflows/ tests, lint, typecheck-static, typecheck-runtime
|
|
231
282
|
bin/console, bin/setup dev scripts
|
|
232
283
|
Rakefile, Steepfile, rbs_collection.yaml
|
|
@@ -240,7 +291,7 @@ Known quirk: `sig/redis_queued_locks/acquier.rbs` is misspelled (should be `acqu
|
|
|
240
291
|
|
|
241
292
|
### Runtime
|
|
242
293
|
- Ruby >= 4.0; stdlib `timeout`, `securerandom`, `logger`; Thread, Fiber, Ractor, Mutex
|
|
243
|
-
- Redis server: WATCH/MULTI, EVAL (Lua), ZSET, HASH, PTTL/PEXPIRE, SCAN
|
|
294
|
+
- Redis server: WATCH/MULTI, EVAL (Lua), ZSET, HASH, PTTL/PEXPIRE (NX/GT: Redis >= 7), SCAN/ZSCAN
|
|
244
295
|
- `redis-client ~> 0.20`
|
|
245
296
|
- Optional, duck-typed: any `::Logger`-compatible logger; any `#notify(event, payload)` instrumenter (ActiveSupport adapter included; SemanticLogger has only an RBS stub)
|
|
246
297
|
|
|
@@ -279,3 +330,4 @@ Known quirk: `sig/redis_queued_locks/acquier.rbs` is misspelled (should be `acqu
|
|
|
279
330
|
- Development gems go in `Gemfile` (`Gemspec/DevelopmentDependencies: Gemfile`), not the gemspec.
|
|
280
331
|
- This overview and `.claude/rules/*.md` are updated in the same change as the code (keys, options, results, events, algorithm steps, limitations); README and CHANGELOG `[Unreleased]` for user-visible behavior (log keys and instrumentation payloads: README `## Logging` / `### Instrumentation Events`).
|
|
281
332
|
- Temporary debt: rspec-retry, disabled coverage minimum, runtime type-check job that can't fail.
|
|
333
|
+
- Specs (`spec/**/*_spec.rb`) are not ideal and will be totally reworked (header `NOTE` of `spec/redis_queued_locks_spec.rb`): until then any coding principle may be broken there; code review checks only that the library behavior is covered and that changed expectations are correct (`.claude/skills/rql-code-review`).
|
data/.claude/rules/acquirer.md
CHANGED
|
@@ -9,13 +9,15 @@ paths:
|
|
|
9
9
|
## Observed style and patterns
|
|
10
10
|
- **Module shape**: `module RedisQueuedLocks::Acquirer::<CamelName>` in `acquirer/<snake_name>.rb`, `# @api private`, a single public entry function inside `class << self` named after the file (`ReleaseLock.release_lock`, `IsLocked.locked?`), helpers under `private` in the same `class << self`.
|
|
11
11
|
- **Utilities**: modules that time or instrument `extend RedisQueuedLocks::Utilities` (gives `clock_gettime`, `run_non_critical`).
|
|
12
|
-
- **Independence of operation modules** (core principle): the modules behind `Client` public methods that don't acquire locks (`ClearDeadRequests`, `ExtendLockTTL`, `IsLocked`, `IsQueued`, `Keys`, `LockInfo`, `Locks`, `QueueInfo`, `Queues`, `ReleaseAllLocks`, `ReleaseLock`, `ReleaseLocksOf`, `ReleaseReadLock`) never call each other and never reference each other's RBS types. They share logic only through `Resource`, `Utilities` and other non-operation helpers; similar logic is duplicated on purpose (independence wins over DRY):
|
|
13
|
-
- `Locks.extract_locks_info` / `Locks.read_lock_info` duplicate the write/read lock formatting of `LockInfo.lock_info` / `LockInfo.read_lock_info
|
|
12
|
+
- **Independence of operation modules** (core principle): the modules behind `Client` public methods that don't acquire locks (`ClearDeadRequests`, `ExtendLockTTL`, `IsLocked`, `IsQueued`, `Keys`, `LockInfo`, `Locks`, `QueueInfo`, `Queues`, `ReindexLocks`, `ReleaseAllLocks`, `ReleaseLock`, `ReleaseLocksOf`, `ReleaseReadLock`) never call each other and never reference each other's RBS types. They share logic only through `Resource`, `Utilities` and other non-operation helpers; similar logic is duplicated on purpose (independence wins over DRY):
|
|
13
|
+
- `Locks.extract_locks_info` / `Locks.read_lock_info` duplicate the write/read lock formatting of `LockInfo.lock_info` / `LockInfo.read_lock_info` (`Locks` fetches the data of a batch of locks in two round trips and `Locks.read_lock_info` formats the prefetched read lock data);
|
|
14
14
|
- `Queues.extract_queues_info` duplicates the request formatting of `QueueInfo.queue_info`;
|
|
15
15
|
- each module declares its own RBS aliases (`Locks::lockInfo` / `Locks::readerInfo` mirror `LockInfo::lockInfo` / `LockInfo::readerInfo`); only `Client` signatures reference them.
|
|
16
16
|
- `AcquireLock` is the lock acquisition core, not an operation module of this list.
|
|
17
17
|
- PoC modules (`*PoC`, e.g. `LockSeriesPoC`) are not covered by this principle: they may reuse operation modules and the `AcquireLock` core (`LockSeriesPoC` uses `AcquireLock.acquire_lock` and its `YieldExpire` mixin).
|
|
18
18
|
- Known debt (don't copy): `AcquireLock::WithAcqTimeout` calls `LockInfo.lock_info` / `QueueInfo.queue_info` for detailed timeout errors (its TODO asks to make `AcquireLock` independent of them).
|
|
19
|
+
- Known limitation (kept by the user's decision): `ReleaseLocksOf` (and `FlushZombies`) check the lock owner (`HMGET`) and delete the lock (`DEL`) in two round trips without `WATCH`: a lock re-acquired in between by another acquirer can be deleted.
|
|
20
|
+
- Known limitation (kept by the user's decision, documented in README `#lock`): `YieldExpire` releases the write lock (`EXPIRE 0`) and decreases its ttl (Lua) without an owner check: a block (without `timed`) longer than the lock ttl can delete/shorten the lock of another acquirer. Read lock release/decrease is per acquirer (owner-safe).
|
|
19
21
|
- **Signatures** (see `arguments.md` for the design rationale):
|
|
20
22
|
- Read-only queries: few positional args `(redis_client, lock_name)`; collection queries use keywords `(redis_client, scan_size:, with_info:)`.
|
|
21
23
|
- Mutating operations: long positional lists ending with the fixed observability tail
|
|
@@ -31,23 +33,36 @@ paths:
|
|
|
31
33
|
6. return `{ ok: true, result: { ..., rel_time: } }`
|
|
32
34
|
- **Results**: always `{ ok: Boolean, result: ... }` for operations; `result` is a Symbol status (`:async_expire_or_no_lock`, `:released`, `:nothing_to_release`) or a Symbol-keyed Hash (`rel_key_cnt`, `tch_queue_cnt`, `rel_time`, `extended_locks_count`). Info queries return a String-keyed Hash / Set or `nil` when absent.
|
|
33
35
|
- **Redis access**:
|
|
34
|
-
- Keys only via `Resource.prepare_lock_key` / `prepare_lock_queue` / `prepare_read_lock_queue` / `prepare_lock_readers` / `prepare_read_lock_key(lock_name, acquirer_id)` and `Resource::*_PATTERN` (`LOCK_PATTERN`, `LOCK_QUEUE_PATTERN`, `READ_LOCK_QUEUE_PATTERN`, `LOCK_READERS_PATTERN`, `READ_LOCK_PATTERN`); `Resource.lock_name_from_readers` / `lock_key_from_readers` map registry keys back.
|
|
36
|
+
- Keys only via `Resource.prepare_lock_key` / `prepare_lock_queue` / `prepare_read_lock_queue` / `prepare_lock_readers` / `prepare_read_lock_key(lock_name, acquirer_id)` and `Resource::*_PATTERN` (`KEY_PATTERN`, `LOCK_PATTERN`, `LOCK_QUEUE_PATTERN`, `READ_LOCK_QUEUE_PATTERN`, `LOCK_READERS_PATTERN`, `READ_LOCK_PATTERN`, `ALL_LOCK_QUEUES_PATTERN`) and `Resource::LOCK_INDEX_KEY`; `Resource.lock_key_family` classifies scanned keys (keyspace lookups, `ReindexLocks`); `Resource.lock_name_from_readers` / `lock_key_from_readers` map registry keys back.
|
|
35
37
|
- Several independent reads inside one attempt/operation go into one `pipelined` round trip (it works inside `multi(watch:)` too: reads are sent after WATCH).
|
|
36
38
|
- Raw commands: `redis.call('CMD', ...)` with uppercase string command names and string args (`'0'`, `'-inf'`, `'+inf'`).
|
|
37
39
|
- Pooled connection: wrap multi-command work in `redis.with do |rconn| ... end`.
|
|
38
40
|
- Atomic writes: `rconn.multi do |transact| ... end` (or `multi(watch: [lock_key])` in `TryToLock`); batch reads: `pipelined do |pipeline| ... end` then index `result[0]`, `result[1]` into named vars (`hget_cmd_res`, `pttl_cmd_res`).
|
|
39
|
-
- Iteration: `
|
|
41
|
+
- Iteration: two lock lookup modes, switched by `config['lock_index_enabled']` (client-wide, `false` by default): `Client` (and the `Swarm` facade, the zombie flusher Ractor via a copied value) passes `lock_index_enabled` to every lock-iterating module (`Locks`, `Queues`, `Keys`, `ReleaseAllLocks`, `ReleaseLocksOf`, `ClearDeadRequests`, `FlushZombies`, `ZombieInfo`). Each module keeps both lookups side by side and shares the processing (lock info formatting, flush/release helpers, instrumentation) in one copy; the lookup is chosen in one place (`lock_index_enabled ? index_x : scan_x` or one `if lock_index_enabled ... else ... end` with `(lock index)` / `(keyspace)` step comments):
|
|
42
|
+
- keyspace (the disabled lock index, the pre-index code): one SCAN pass over `KEY_PATTERN` (or `ALL_LOCK_QUEUES_PATTERN` for queues) per operation, keys classified by `Resource.lock_key_family`, per-key commands in pipelined batches collected across SCAN pages (`scan_locks`, `scan_queues`, `each_lock_queue`, `flush_keyspace_zombies`, `each_keyspace_zombie_lock`); no lock index command is sent;
|
|
43
|
+
- lock index (enabled): one `ZSCAN` pass over the lock index per operation (`each_lock_name_batch(rconn, scan_size) { |lock_names| ... }`: `zscan(Resource::LOCK_INDEX_KEY, count: scan_size)`, names with an expired score are skipped, batches of `scan_size` names collected across ZSCAN pages, the rest yielded after the pass); keys are derived from the names (`Resource.prepare_*`, reader data via registry members) and per-key commands are sent in one pipeline (or one variadic command / `MULTI`) per batch; info queries check existence (`EXISTS`, empty `ZRANGE` = missing registry/queue); the keyspace is never scanned (only `ReindexLocks` scans `KEY_PATTERN` with `Resource.lock_key_family` to adopt keys without index entries, in both modes). Each operation module has its own `each_lock_name_batch` copy (same name and shape, duplicated on purpose; the swarm modules iterate inline: `flush_lock_index_zombies`, `each_lock_index_zombie_lock`).
|
|
44
|
+
- Never one round trip per key in either mode.
|
|
40
45
|
- Lua: frozen heredoc constant (`<<~LUA_SCRIPT.strip.tr("\n", '').freeze`) + `call('EVAL', SCRIPT, 1, key, arg)`.
|
|
41
46
|
- Release = `EXPIRE key 0` / `ZREMRANGEBYSCORE queue -inf +inf`; a read lock is released with `ZREM <readers> <acq_id>` + `DEL <reader data>` (owner-safe, other readers untouched); Redis TTL sentinels handled explicitly (`PTTL` `-2` = missing, `-1` = no expiry → `Float::INFINITY`).
|
|
47
|
+
- `ReleaseLock` (`unlock`): one `MULTI` with `ZRANGE <readers>` first (the snapshot of the released read locks, read by position) and the release commands, then one `DEL` of the snapshot readers' data only when it is not empty (1 round trip without readers). Trade-off kept by the user's decision: a snapshot reader that obtains a new read lock between the two round trips loses its new read lock data (the read lock itself stays: the registry is the source of truth).
|
|
42
48
|
- Readers registry members are live only while `score > Redis TIME (ms)` (`Resource.redis_time_ms`); expired members can stay till cleanup, so info/checks must filter by score, never by `ZCARD`/`EXISTS` alone.
|
|
49
|
+
- **Lock index** (`Resource::LOCK_INDEX_KEY` = `rql:index:locks`, ZSET lock name => upper bound of the lifetime of all lock keys in client ms: `Resource.lock_index_time + ttl + lock_index_grace_period`), only with the enabled lock index (`lock_index_enabled`; with the disabled one no command below is sent: the pre-index behavior): the index must stay a superset of the existing lock names (a missing name hides a lock from all lock-iterating operations):
|
|
50
|
+
- the mode is client-wide (`config['lock_index_enabled']` read by `Client`, never a per-call keyword): the hot path gets it as `lock_index_enabled` (`AcquireLock` required keyword → `TryToLock` positional before `lock_index_grace_period`; `ExtendLockTTL`/`ReleaseLock` positional after the lock data); every lock index command is a conditional **tail** of its pipeline/transaction (`return unless lock_index_enabled` at the end of `queue_lock_request`, `if lock_index_enabled` around the last `MULTI` command), so reply positions never depend on the mode (only the reentrant write extension checks `result.size == (lock_index_enabled ? 5 : 4)`);
|
|
51
|
+
- a command that **creates** a lock key is followed by `ZADD <index> GT <now + ttl + grace> <lock_name>` in the same pipeline/transaction, **after** the key command: the request queue `ZADD NX` in the queueing pipeline (queue ttl), the write lock `HSET` / registry `ZADD` + reader data at the end of the obtaining `MULTI` (lock ttl); the obtaining `MULTI` must index the lock itself: `unlock`/`clear_locks` can drop the name between the queueing and the obtaining (`EXPIRE`/`DEL` of an absent key does not abort the attempt's `WATCH`);
|
|
52
|
+
- a name is **removed** only together with the deletion of all its indexed keys in one `MULTI` (`ReleaseLock`: `ZREM` in the release transaction; `ReleaseAllLocks`: `DEL` of the batch keys + `ZREM` of the batch names); operations that can leave keys of a name (`ReleaseLocksOf`, `YieldExpire` releases, `FlushZombies`) never remove names;
|
|
53
|
+
- a command that **extends** a lock key is followed by `ZADD <index> XX INCR <ms> <lock_name>` in the same transaction (`XX`: names without an entry are not indexed with a wrong score); ttl decreases and releases never touch the index (the bound stays an upper bound);
|
|
54
|
+
- lock request queues get `PEXPIRE NX` + `PEXPIRE GT` with the queue ttl (ms, `(queue_ttl * 1_000).ceil`) on each queueing: every indexed key has a bounded lifetime;
|
|
55
|
+
- pruning is `ZREMRANGEBYSCORE <index> -inf <lock_index_time - 1>` (each queueing, `flush_zombies`, `clear_locks`, `clear_dead_requests`); keys written without the index (older versions, raw writes) are adopted by `ReindexLocks` (SCAN pass, `PEXPIRE NX` for queues, `PTTL` => `ZADD GT`).
|
|
56
|
+
- Redis never keeps empty sorted sets/hashes: an empty `ZRANGE`/`HGETALL` result already means a missing key, so don't pair such reads with `EXISTS` (`QueueInfo`, `Queues`).
|
|
43
57
|
- **Data normalization**: Redis hash strings are converted with `Float(...)` / `Integer(...)` inside `hget_cmd_res.tap do |lock_data| ... end`, optional fields guarded with `if lock_data['x']`.
|
|
44
58
|
- **AcquireLock**:
|
|
45
59
|
- Main module `require_relative`s its parts, then `extend`s the mixins (`TryToLock`, `DelayExecution`, `YieldExpire`, `WithAcqTimeout`, `DequeueFromLockQueue`); mixins are plain modules with instance methods (`def try_to_lock(...)`), visitors are `class << self` modules called explicitly (see `visitors.md`).
|
|
46
60
|
- The algorithm is a numbered step script (`# Step 0`, `# Step 2.1`, `# Step 2.2.a`) driven by a mutable `acq_process` hash (`:should_try`, `:tries`, `:acquired`, `:result`, `:lock_info`, timings).
|
|
47
61
|
- Failure modes are Symbols (`:fail_fast_no_try`, `:fail_fast_after_try`, `:conflict_dead_lock`, `:conflict_lock_upgrade`, ...); exceptions are raised only when `raise_errors` is true, with a message naming the lock key / acquirer id.
|
|
48
|
-
- `TryToLock`
|
|
62
|
+
- `TryToLock` pre-check (retries: `acquire_lock` passes `retry_attempt = tries > 0`, and `fail_fast` attempts): `precheck_lock` reads the lock state with the same pipeline (`pipeline_lock_state`; with the queueing for retries, without it for `fail_fast`) without `WATCH`; it returns the failure result (`lock_obstacle` / `:fail_fast_no_try`) and logs the same events as the failed attempt, or `nil` to continue the attempt (lock obtainable or a same-process conflict). The pre-check never obtains the lock: mutual exclusion stays with `WATCH`/`MULTI`.
|
|
63
|
+
- `TryToLock` attempt (both modes): `multi(watch:)` (write: `[lock_key, lock_readers_key]`, read: `[lock_key]`) → one pipelined state read (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only, the write-only tail of the pipeline) → same-process conflict detection (`sp_conflict_rw_mode` = held `:write`/`:read`) → the lock request queueing in one pipelined round trip (`queue_lock_request`: `ZADD NX` into the queue of the requested mode → prune dead requests in **both** queues → queue heads of both queues → (the enabled lock index only) request queue `PEXPIRE NX`+`GT` → lock index `ZADD GT` → lock index pruning (5 replies, 9 with the lock index; new commands go to the end, replies after the queue heads are not read); eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends these commands in the state pipeline right after its head, so `lock {}` = 4 round trips; the queueing log events are still emitted in the classic branch in the same order) (`:queued`: write must be first in its queue; any mode waits for an earlier `(score, acq_id)` request of the opposite mode) → lock state (write held → `:lock_is_still_acquired`; live readers for write / own live read for read → `:read_lock_is_still_acquired`) → MULTI (write: `ZREM` + `HSET` + `PEXPIRE` (+ lock index `ZADD GT`); read: `ZREM` + prune registry + `ZADD expiry` + registry `PEXPIRE NX`+`GT` + reader data `DEL`+`HSET`+`PEXPIRE` (+ lock index `ZADD GT`)). With the enabled lock index the `:extendable_work_through` extension `MULTI` ends with the lock index `ZADD XX INCR` (write: 4 replies, 5 with the lock index: `result.size == (lock_index_enabled ? 5 : 4)`).
|
|
49
64
|
- The try result carries internal `rw_mode` (the **held** lock mode); `acquire_lock` stores it as `acq_process[:held_rw_mode]` and passes it to `yield_expire`, which releases/decreases the held lock. The public `lock` result shape does not include it.
|
|
50
|
-
- Read lock data (`rql:lock_reader:<name>:<acq_id>`) uses the write-lock field format
|
|
65
|
+
- Read lock data (`rql:lock_reader:<name>:<acq_id>`) uses the write-lock field format without `acq_id`/`hst_id` (`ts`, `ini_ttl`, meta, `spc_*`: both ids are derived from the acquirer id, `LockInfo.read_lock_info`/`Locks.read_lock_info` put them first in each reader and still accept older data that contains them) and the read lock TTL; it is a data carrier only (writers never read it). Its key (`read_lock_key`) is built only for read lock requests (`nil` for write requests: they never hold read locks).
|
|
51
66
|
- Timing via monotonic `clock_gettime` (microseconds); wall time only for `at:`/`ts` fields.
|
|
52
67
|
- **Inline typing**: `# @type var result: Symbol`, `{} #: Hash[String,String|Float|Integer]`, and `# steep:ignore` on pattern-matching destructures and splats (`rconn.call('DEL', *keys) # steep:ignore`).
|
|
53
68
|
|
|
@@ -58,24 +73,27 @@ paths:
|
|
|
58
73
|
4. Return `{ ok:, result: }`; use Symbol statuses for simple outcomes and abbreviated Symbol keys (`rel_*`, `*_cnt`, `*_time`) consistent with existing results. Don't raise from acquirer modules except through the `raise_errors` path in `AcquireLock`.
|
|
59
74
|
5. Use `redis.with { |rconn| ... }` for any multi-command work; use `multi` for writes that must be atomic, `pipelined` for independent reads, `multi(watch:)` or Lua when a write depends on a read.
|
|
60
75
|
6. Use uppercase string Redis commands and string numeric args; handle `PTTL`/`TTL` sentinel values explicitly.
|
|
61
|
-
7. Iterate
|
|
76
|
+
7. Iterate locks in one pass with both lookups (see "Iteration"): a new lock-iterating operation takes `lock_index_enabled` (from `Client`: `config['lock_index_enabled']`) and implements the keyspace lookup (one SCAN pass over `KEY_PATTERN`/`ALL_LOCK_QUEUES_PATTERN`, `Resource.lock_key_family`) and the lock index lookup (one `ZSCAN` pass, `each_lock_name_batch`, keys derived from the lock names) next to each other, sharing the rest of the logic in one copy; never `KEYS`, never a keyspace `SCAN` with the enabled lock index (except `ReindexLocks`); send per-key commands in pipelined batches of `scan_size` keys/lock names (plus the rest after the pass), never one round trip per key; cover the remainder batches of both modes in specs (`lock-iterating operations: one pass (keyspace or lock index) with small batches` runs in both modes) and assert the absence of `SCAN` (lock index) / `ZSCAN` and lock index commands (keyspace) in recorded-command specs (`lock-iterating operations use the lock index`, `the disabled lock index (default): ...`).
|
|
62
77
|
8. Measure durations with `clock_gettime` and report ms with `/ 1_000.0).ceil(2)`; use `Time.now.to_f` only for event timestamps.
|
|
63
78
|
9. Wrap every `instrumenter.notify` / logger call in `run_non_critical` (or a visitor) and gate it with `Instrument.should_instrument?` / `Logging.should_log?`; observability must never break locking. Payload changes of inline `notify` calls are documented in README `### Instrumentation Events` in the same change (see `visitors.md` rule 14).
|
|
64
79
|
10. In `AcquireLock`, add behavior as a new step or mixin rather than growing `acquire_lock`; keep the `# Step N.x` comment numbering, update `acq_process` keys consistently, and add a matching `LogVisitor`/`InstrVisitor` method for each new lifecycle event.
|
|
65
|
-
11. When normalizing lock hash fields, follow the `Float()` / `Integer()` conversion pattern; if a new lock field is added, update every copy of the formatting: write locks in `LockInfo.lock_info` and `Locks.extract_locks_info`, readers in `LockInfo.read_lock_info` and `Locks.read_lock_info` (reader data uses the same fields).
|
|
80
|
+
11. When normalizing lock hash fields, follow the `Float()` / `Integer()` conversion pattern; if a new lock field is added, update every copy of the formatting: write locks in `LockInfo.lock_info` and `Locks.extract_locks_info`, readers in `LockInfo.read_lock_info` and `Locks.read_lock_info` (reader data uses the same fields except `acq_id`/`hst_id`, which are derived from the acquirer id). Don't store derivable values in lock data.
|
|
66
81
|
12. Read/write checklist for any change in this directory:
|
|
67
82
|
- lock state checks: write requires no write lock and no **live** readers; read requires no write lock; same-acquirer reader/writer cases go through `conflict_strategy` (read→write upgrade never "works through");
|
|
68
83
|
- WATCH: writers watch the write key + readers registry, readers watch the write key only (never the registry: readers must not abort each other); don't modify a watched key outside MULTI after WATCH (it aborts your own EXEC);
|
|
69
84
|
- every attempt prunes dead requests in both queues; dequeue/cleanup removes the acquirer from the queue of its mode;
|
|
70
85
|
- every release/cleanup/info/zombie path covers write lock, readers registry, reader data and both queues;
|
|
71
86
|
- registry/reader-data TTLs: `PEXPIRE NX` + `PEXPIRE GT` pair to keep max TTL (`GT` alone never sets TTL on a key without one);
|
|
72
|
-
- keep write-only result shapes unchanged; add read keys/fields only when read data exists
|
|
73
|
-
|
|
87
|
+
- keep write-only result shapes unchanged; add read keys/fields only when read data exists;
|
|
88
|
+
- lock index (the enabled lock index only, `if lock_index_enabled`): every key-creating command is followed by the index `ZADD GT` (the obtaining `MULTI` too), every extension by `ZADD XX INCR`, names are removed only with all their keys in one `MULTI` (see "Lock index" above); the disabled lock index sends no lock index command.
|
|
89
|
+
13. Values used only for logs (e.g. `HGETALL` of lock data) are fetched only when the log is enabled (`(log_sampled && log_lock_try) ? ... : NO_LOCK_DATA`, a shared frozen hash instead of a new `{}` per attempt): visitor arguments are evaluated before the visitor's guard.
|
|
74
90
|
14. Keep operation modules independent (see "Independence of operation modules"; PoC modules such as `LockSeriesPoC` are exempt): never call another operation module's function (public or private) or reference its RBS types from an operation module. When similar logic is needed, duplicate it inside the module (same method name and shape as the original copy, with a `NOTE` that it is duplicated on purpose) or move a pure, data-only helper (key names, id parsing, time conversion) to `Resource` / `Utilities`. Helpers used by one module stay `private`. When changing logic that has copies, update every copy in the same change.
|
|
91
|
+
15. Pipelines/transactions whose replies are read by position (`lock_state[N]`, `result[N]`): document the reply positions next to the commands (`NOTE: (pipeline layout)`). Conditionally sent commands (e.g. `... unless read_mode`) form a tail after all unconditional commands; read their replies only under the same condition (a command appended after a conditional tail gets a mode-dependent position and its reply is read as another command's reply). Insert new commands before the conditional tail and update the positions, the reply reading, the position constants and the layout spec in the same change. `TryToLock` lock state pipeline (`pipeline_lock_state`, shared by the attempt and the pre-check): head `[LOCK_STATE_OBTAINER_REPLY, LOCK_STATE_REDIS_TIME_REPLY, LOCK_STATE_OWN_READ_LOCK_REPLY]`, queueing commands at `LOCK_STATE_QUEUE_REPLIES_AT` (`queue_lock_request` replies, used ones at `QUEUE_HEAD_REPLY`/`OPPOSITE_QUEUE_HEAD_REPLY`), the write-only tail last (`lock_state.last`); read replies only by these constants (both reply readers: `try_to_lock` Step 0.1 and `precheck_lock`); the obstacle decision and its exit logs are shared too (`find_opposite_request_ahead`, `lock_obstacle`, `log_lock_obstacle`): change them in one place; spec `lock state pipeline layout` records the pipelines of both modes with and without eager queueing and fails on a broken layout or stale position constants.
|
|
92
|
+
16. Lock index (see "Lock index" above): a new command that creates a lock key must be followed (with the enabled lock index: `if lock_index_enabled`, as a tail of the pipeline/transaction) by the index `ZADD <LOCK_INDEX_KEY> GT <Resource.lock_index_time + lifetime + lock_index_grace_period> <lock_name>` in the same pipeline/transaction (after the key command), a new extension by `ZADD XX INCR <ms> <lock_name>`; a new name removal is allowed only in the same `MULTI` that deletes all indexed keys of the name; a new key family with its own lifetime must get a bounded TTL (the enabled lock index) and be covered by the index score and by `ReindexLocks`; never decrease index scores and never derive mutual exclusion from the index; never send lock index commands with the disabled lock index. Cover new index writes in `describe 'lock index'` specs (they enable the lock index explicitly).
|
|
75
93
|
|
|
76
94
|
## Recommendations (proposed, not yet project policy)
|
|
77
95
|
Apply to new or touched code; don't refactor existing code for these unless asked.
|
|
78
96
|
1. Load Lua scripts once (`SCRIPT LOAD` + `EVALSHA`, falling back to `EVAL` on `NOSCRIPT`), as the TODO in `extend_lock_ttl.rb` asks.
|
|
79
|
-
2.
|
|
97
|
+
2. Measure lock-iterating operations by the lock index size (distinct lock names x (max(queue ttl, lock ttl) + grace)): most of the visited names are dead; prefer one cheap existence check per name (`EXISTS` with several keys) before reading the details when adding new iterating features.
|
|
80
98
|
3. Treat `lock_series_poc.rb` as experimental: don't build new features on it without asking. Its locks are released only through `release_lock_series` (obtained-by-series locks only, owner-checked, on success and on every failure path).
|
|
81
99
|
4. Unify the observability tail order (`logger, instrumenter` vs `instrumenter, logger` in `release_lock`) when those signatures are next changed.
|
data/.claude/rules/arguments.md
CHANGED
|
@@ -50,11 +50,12 @@ This is a deliberate framework-level decision, **not** something to refactor int
|
|
|
50
50
|
2. New public option: add it as an explicit keyword to **every** affected `Client` method (`x` and `x!`, e.g. `lock` and `lock!`), with the default taken from `config['...']` (add the `setting` + `validate` first) or a literal for per-call-only flags (`raise_errors: false`, `meta: nil`).
|
|
51
51
|
3. Forward it explicitly by name through every layer using shorthand (`new_option:`); never forward via `**kwargs`, `...`, or `method(__method__).parameters` tricks.
|
|
52
52
|
4. Keep defaults only at the public boundary (`Client`); internal functions take required keywords/positionals with no defaults so a missed forward fails loudly.
|
|
53
|
-
5. Internal positional lists: append new params in the established order (subject → domain data → observability tail `logger, instrumenter, instrument, log_sampling_*, log_sample_this, instr_sampling_*, instr_sample_this`) and update every call site and the RBS signature in the same change. Lock key params keep this order: `lock_key, read_write_mode, lock_key_queue, read_lock_key_queue, lock_readers_key, read_lock_key, acquirer_id, host_id, ...` (`yield_expire`: `lock_key, lock_readers_key, read_lock_key, rw_mode, acquirer_id, ...`); compute keys once in `acquire_lock` and pass them down.
|
|
53
|
+
5. Internal positional lists: append new params in the established order (subject → domain data → observability tail `logger, instrumenter, instrument, log_sampling_*, log_sample_this, instr_sampling_*, instr_sample_this`) and update every call site and the RBS signature in the same change. Lock key params keep this order: `lock_key, read_write_mode, lock_key_queue, read_lock_key_queue, lock_readers_key, read_lock_key, lock_name, acquirer_id, host_id, ...` (`lock_name` is the lock index member; `yield_expire`: `lock_key, lock_readers_key, read_lock_key, rw_mode, acquirer_id, ...`); TTL params: `ttl, queue_ttl, lock_index_enabled, lock_index_grace_period`; compute keys once in `acquire_lock` and pass them down (`read_lock_key` is `nil` for write lock requests). Queueing helpers (`queue_lock_request`, `pipeline_lock_state`) take the lock index params last: `lock_name, queue_ttl, lock_index_enabled, lock_index_grace_period`. Lock-iterating modules take `lock_index_enabled` after their domain data (`scan_size:`/`with_info:` keywords + `lock_index_enabled:`; positional lists: right after the batch sizes / `dead_ttl`, before the observability tail; swarm functions: last).
|
|
54
54
|
6. Document each new option with its own `@option`/`@param` YARD line (type, meaning, default source) and add it to the RBS method signature as a named param/keyword.
|
|
55
55
|
7. Keep hot-path code allocation-light: no per-call wrapper objects, no hash merging, no splats to build args, no `tap`/closures purely for argument plumbing; prefer passing existing locals.
|
|
56
56
|
8. Silence length cops locally (`# rubocop:disable Metrics/MethodLength`) rather than shortening signatures; `Metrics/ParameterLists` is disabled project-wide on purpose.
|
|
57
|
-
9.
|
|
57
|
+
9. Client-wide modes are not per-call options: `config['lock_index_enabled']` is read inside the `Client` (and `Swarm`) method bodies and forwarded by name to the acquirers (`lock_index_enabled: config['lock_index_enabled']` / positional `config['lock_index_enabled']`), never added as a keyword of `lock`/`locks`/`clear_locks`/...: one call in another mode would break the lock index invariant (all lock acquirements of a client must maintain the index when its operations read it).
|
|
58
|
+
10. `lock`-family options exist in four `Client` methods: `lock`, `lock!`, `lock_series`, `lock_series!` (and `LockSeriesPoC.lock_series_poc` forwards them to `AcquireLock.acquire_lock`). A new lock option is added to all of them; `lock_series` keeps one value for the whole series (e.g. `read_write_mode`).
|
|
58
59
|
|
|
59
60
|
## Recommendations (proposed, not yet project policy)
|
|
60
61
|
1. For new internal functions with many same-typed params (e.g. several Integers/Booleans in a row), consider required keywords instead of positionals to prevent misordering; keep the list flat and explicit.
|
data/.claude/rules/git.md
CHANGED
|
@@ -29,9 +29,10 @@ Applies to every commit Claude makes in this repository (no `paths:` scope: alwa
|
|
|
29
29
|
**When to commit**
|
|
30
30
|
7. Never commit on your own. When a step with changes reaches completion, ask the user for permission to commit and wait for the answer:
|
|
31
31
|
- completion = the feature/functionality required by the prompt is fully implemented (code, RBS, specs, README/CHANGELOG, `.claude/project-overview.md` and rules kept in sync) and checks are green: the touched examples and the full suite (`bundle exec rake rspec`), `bundle exec rake rubocop`, `bundle exec rake steep:check` (no `ERROR`/`FATAL` lines); changes without code (docs, `.claude` rules) need no test run;
|
|
32
|
-
- the
|
|
32
|
+
- pre-commit review: once the step is complete, and only then, run the `rql-code-review` skill on the step's changes (scope `uncommitted`) before the question; never during the implementation or for incomplete/red states. Blocker/Major findings in the step's own changes mean the step is not complete yet: fix them, re-run the affected checks and review the fixes; findings that need the user's decision are reported instead of the question (rule 9);
|
|
33
|
+
- the question contains: the proposed commit subject (following the format rules above), a short recap of what was done in the changes (features/fixes, touched areas, check results), the review verdict with the remaining Minor/Nit findings and the list of files that will be committed;
|
|
33
34
|
- commit only after an explicit "yes" (or with the subject the user corrects); on "no" leave the changes uncommitted (staging also only on request).
|
|
34
|
-
8. An explicit user request to commit ("commit", "закоммить") is the permission itself: commit right away with the given subject (or a subject built by the rules) without asking again.
|
|
35
|
+
8. An explicit user request to commit ("commit", "закоммить") is the permission itself: commit right away with the given subject (or a subject built by the rules) without asking again and without a review. A commit authorized in advance for a task ("implement X and commit", "commit and continue") still goes through the pre-commit review of rule 7 when the step is complete.
|
|
35
36
|
9. Don't propose a commit for incomplete or red states (failing specs, lint/type errors, half-done requirements); report the blocker instead.
|
|
36
37
|
10. One commit per completed step. Stage only the files of that step (`git status` first); never commit environment artifacts: `rbs_collection.lock.yaml` rewritten by `rbs collection install` (restore it with `git checkout --`), `coverage/`, `.gem_rbs_collection/`, local settings.
|
|
37
38
|
11. Never commit to `master`: create a feature branch first (named after the feature, e.g. `read-write-locks-realisation`). Never push, amend, rebase, reset or force anything unless the user asks.
|
data/.claude/rules/logic.md
CHANGED
|
@@ -12,7 +12,7 @@ Related rule files (loaded for narrower paths):
|
|
|
12
12
|
- `swarm.md`: swarm architecture, element lifecycle, Ractor/Thread coding rules
|
|
13
13
|
|
|
14
14
|
## Swarm overview (details in `swarm.md`)
|
|
15
|
-
- Purpose: zombie-lock elimination. `ProbeHosts` periodically `HSET`s every host id (`rql:hst:<pid>/<thread>/<ractor>/<identity>`) with `Time.now.to_f` into `rql:swarm:hsts`; `FlushZombies` treats hosts older than `zombie_ttl` (ms) as zombies and deletes their write locks, their read locks (registry members + reader data), their entries in both request queues and the hosts themselves
|
|
15
|
+
- Purpose: zombie-lock elimination. `ProbeHosts` periodically `HSET`s every host id (`rql:hst:<pid>/<thread>/<ractor>/<identity>`) with `Time.now.to_f` into `rql:swarm:hsts`; `FlushZombies` treats hosts older than `zombie_ttl` (ms) as zombies and deletes their write locks, their read locks (registry members + reader data), their entries in both request queues and the hosts themselves; locks are found via one keyspace `SCAN` pass (default) or via the lock index (`rql:index:locks`, `ZSCAN`) with the enabled `config['lock_index_enabled']`.
|
|
16
16
|
- `Client#swarm` → `Swarm` facade (one per client) owns a `Supervisor` and the swarm elements; started by `swarmize!` / `swarm.auto_swarm`, stopped by `deswarmize!`.
|
|
17
17
|
- Elements are independent background units: a control unit (`SwarmElement::Threaded` = Thread + `SizedQueue` command channel; `SwarmElement::Isolated` = Ractor driven via its own pair of `Ractor::Port`s: results port created in the main Ractor, where the Swarm and Supervisor live; command port created inside the element Ractor) that spawns, stops and reports on a main-loop Thread with its own Redis connection (`Swarm::RedisClientBuilder`).
|
|
18
18
|
- `ProbeHosts` is Threaded (it must see the client's ractor threads); `FlushZombies` is Isolated (copied config values only).
|
|
@@ -29,9 +29,10 @@ Related rule files (loaded for narrower paths):
|
|
|
29
29
|
- Operation modules (except PoC modules such as `LockSeriesPoC`) are independent of each other: no cross-calls and no shared RBS types between them; common logic lives in `Resource`/`Utilities`, similar logic is duplicated on purpose (see `acquirer.md`, "Independence of operation modules").
|
|
30
30
|
- Public API results are hashes `{ ok: Boolean, result: Symbol|Hash }` (`Client`, `Acquirer::*`, `Swarm` facade actions); `Client` `!` methods raise `RedisQueuedLocks::*Error`. Internal swarm element APIs return bare scalars/primitives (see `swarm.md`).
|
|
31
31
|
- `Client` methods only fill defaults from `config['...']` and delegate to `Acquirer::*` / `Swarm`.
|
|
32
|
-
- Redis keys come only from `RedisQueuedLocks::Resource.prepare_*` helpers and its `*_PATTERN` / `SWARM_KEY` constants. Key families: write lock `rql:lock:` (`LOCK_PATTERN`), write requests `rql:lock_queue:` (`LOCK_QUEUE_PATTERN`), readers registry `rql:lock_readers:` (`LOCK_READERS_PATTERN`), read requests `rql:lock_read_queue:` (`READ_LOCK_QUEUE_PATTERN`), reader data `rql:lock_reader:<name>:<acq_id>` (`READ_LOCK_PATTERN`); all of them match `KEY_PATTERN` (`rql:lock*`).
|
|
32
|
+
- Redis keys come only from `RedisQueuedLocks::Resource.prepare_*` helpers and its `*_PATTERN` / `SWARM_KEY` constants. Key families: write lock `rql:lock:` (`LOCK_PATTERN`), write requests `rql:lock_queue:` (`LOCK_QUEUE_PATTERN`), readers registry `rql:lock_readers:` (`LOCK_READERS_PATTERN`), read requests `rql:lock_read_queue:` (`READ_LOCK_QUEUE_PATTERN`), reader data `rql:lock_reader:<name>:<acq_id>` (`READ_LOCK_PATTERN`); all of them match `KEY_PATTERN` (`rql:lock*`); `Resource.lock_key_family(key)` classifies a key by its prefix (`:lock`, `:lock_queue`, `:lock_readers`, `:read_lock_queue`, `:read_lock`, `nil`).
|
|
33
33
|
- New key families use a distinct **prefix** (`rql:lock_<kind>:<name>`), never a suffix of the lock name: the lock name is an arbitrary string, so `rql:lock_queue:<name>:read` would collide with the queue of a lock named `<name>:read`.
|
|
34
|
-
-
|
|
34
|
+
- Lock index `rql:index:locks` (`Resource::LOCK_INDEX_KEY`), opt-in via the client-wide `config['lock_index_enabled']` (`false` by default: lock-iterating operations scan the keyspace, no lock index commands, queues without TTL): a global ZSET of lock names (score = upper bound of the lifetime of all lock keys, client ms + ttl + `lock_index_grace_period`); not matched by `KEY_PATTERN`. With the enabled lock index every key-creating command is followed by the index `ZADD GT` (queueing pipeline, obtaining `MULTI`), every extension by `ZADD XX INCR`, names are removed only with all their keys (`unlock`, `clear_locks`), lock request queues have TTL; all lock-iterating operations use it (`ZSCAN`), only `reindex_locks` scans the keyspace. Every lock-iterating module keeps both lookups (keyspace and lock index) side by side with the shared processing in one copy (details in `acquirer.md`, "Iteration" and "Lock index").
|
|
35
|
+
- Time: lock expirations that affect safety use Redis time (`PEXPIRE`, or `TIME` via `Resource.redis_time_ms` for reader scores); client `Time.now.to_f` is used only for queue positions, `ts` fields and zombie probes; lock index scores use client ms (`Resource.lock_index_time`) with the `lock_index_grace_period` margin (clock skew tolerance; the index never affects mutual exclusion).
|
|
35
36
|
- Config: `setting('key', default)` + `validate('key') { |val| ... }` in `config.rb`; dotted keys for nested groups (`swarm.flush_zombies.zombie_ttl`); units in a trailing `# NOTE: in milliseconds` comment.
|
|
36
37
|
- Errors: subclasses in `errors.rb`, written as `class XError < Error; end` (not `Class.new`) so RBS/Steep can see the superclass.
|
|
37
38
|
- Comments use `# NOTE:`, `# TODO:` and `# @type var x: T` / `#: T` for inline type hints.
|
data/.claude/rules/swarm.md
CHANGED
|
@@ -13,7 +13,11 @@ The swarm removes **zombie locks**: locks and queue entries left by dead workers
|
|
|
13
13
|
- **Host**: a `process/thread/ractor/identity` worker, with the id `rql:hst:<pid>/<thread_id>/<ractor_id>/<identity>` (`Resource.host_identifier`). Fibers are not included because `ObjectSpace` can't see Fibers or Threads once a Ractor exists. Hosts are enumerated with `Thread.list` (`Resource.possible_host_identifiers`).
|
|
14
14
|
- **Liveness**: `ProbeHosts` runs `HSET rql:swarm:hsts <host_id> <Time.now.to_f>` for every possible host of the client's ractor (`Resource::SWARM_KEY`).
|
|
15
15
|
- **Zombie**: a host whose last probe score is `< Resource.calc_zombie_score(zombie_ttl / 1_000.0)` (`now - ttl`). `zombie_ttl` is in milliseconds. Zombie locks are `rql:lock:*` whose `hst_id` field is a zombie host, plus zombie read locks: readers registry members (`rql:lock_readers:*`) whose host (derived from the acquirer id by `Resource.host_identifier_from_acquirer`, a pure function usable inside the Ractor) is a zombie host; a zombie read lock is reported as its registry key. Zombie acquirers are their `acq_id`s.
|
|
16
|
-
- **Flush** (`FlushZombies.flush_zombies`) runs these steps: `HGETALL` hosts → zombie hosts (return early if none) →
|
|
16
|
+
- **Flush** (`FlushZombies.flush_zombies(redis_client, zombie_ttl, lock_scan_size, queue_scan_size, lock_index_enabled)`) runs these steps: `HGETALL` hosts → zombie hosts (return early if none) → one pass over the locks in one of two lookup modes (`config['lock_index_enabled']`, passed by the `Swarm` facade and copied into the flusher Ractor at spawn):
|
|
17
|
+
- keyspace (`flush_keyspace_zombies`, the disabled lock index, default): one `SCAN` pass over `KEY_PATTERN` (`COUNT` = `lock_scan_size`), keys classified by `Resource.lock_key_family`: write locks and readers registries in batches of `lock_scan_size` keys, lock queues collected during the pass and processed after it in batches of `queue_scan_size`;
|
|
18
|
+
- lock index (`flush_lock_index_zombies`): drop expired lock names from the lock index (`ZREMRANGEBYSCORE rql:index:locks -inf <now - 1>`) → one `ZSCAN` pass over the lock index (`Resource::LOCK_INDEX_KEY`, lock names; `COUNT` = `lock_scan_size`), lock keys are derived from the names (the keyspace is never scanned): write locks (`prepare_lock_key`) and readers registries (`prepare_lock_readers`) in batches of `lock_scan_size` locks, both lock queues of each name in batches of `queue_scan_size` queues during the same pass (`batch_zombie_requests`);
|
|
19
|
+
- shared batch helpers: pipelined `HMGET acq_id hst_id` + one `DEL` of the zombie locks; pipelined `ZRANGE` + pipelined `ZREM` zombie readers + `DEL` their reader data (`rql:lock_reader:<name>:<acq_id>`); pipelined `ZRANGE` + pipelined `ZREM` of the requests whose host (derived from the request's acquirer id by `Resource.host_identifier_from_acquirer`) is a zombie host, their acquirers are added to the zombie acquirers → `HDEL` zombie hosts.
|
|
20
|
+
`ZombieInfo` mirrors the lock part (`each_zombie_lock` → `each_keyspace_zombie_lock` / `each_lock_index_zombie_lock`: the same passes and batches, names with an expired index score are skipped; nothing is scanned when there are no zombie hosts). It is best-effort and non-transactional. With the enabled lock index locks without a lock index entry (older RQL versions, the disabled lock index, raw writes) are invisible until `client.reindex_locks`.
|
|
17
21
|
|
|
18
22
|
## Components
|
|
19
23
|
| Object | File | Kind | Role |
|
|
@@ -24,14 +28,14 @@ The swarm removes **zombie locks**: locks and queue entries left by dead workers
|
|
|
24
28
|
| `SwarmElement::Isolated` | `swarm/swarm_element/isolated.rb` | abstract base | control `Ractor` plus a main-loop `Thread` inside it, driven through a per-element pair of `Ractor::Port`s |
|
|
25
29
|
| `ProbeHosts` | `swarm/probe_hosts.rb` | `< Threaded` | periodic host liveness probes |
|
|
26
30
|
| `FlushZombies` | `swarm/flush_zombies.rb` | `< Isolated` | periodic zombie flushing |
|
|
27
|
-
| `Acquirers`, `ZombieInfo` | `swarm/acquirers.rb`, `swarm/zombie_info.rb` | stateless `class << self` modules | read-only queries (`HGETALL` swarm hash,
|
|
31
|
+
| `Acquirers`, `ZombieInfo` | `swarm/acquirers.rb`, `swarm/zombie_info.rb` | stateless `class << self` modules | read-only queries (`HGETALL` swarm hash, keyspace `SCAN`s or lock index `ZSCAN`s) |
|
|
28
32
|
| `RedisClientBuilder` | `swarm/redis_client_builder.rb` | stateless module | `build(pooled:, sentinel:, config:, pool_config:)` returns a fresh `RedisClient` or `RedisClient::Pooled` |
|
|
29
33
|
|
|
30
34
|
`Client` methods `swarmize!`, `deswarmize!`, `swarm_status`/`swarm_state`, `swarm_info`, `probe_hosts`, `flush_zombies`, `zombie_locks`, `zombie_acquirers`, `zombie_hosts`, `zombies_info`/`zombies` only fill defaults from `config['swarm.*']` and delegate to `Swarm`. `Client#initialize` calls `swarm.swarm!` when `swarm.auto_swarm` is set.
|
|
31
35
|
|
|
32
36
|
## Why Threaded vs Isolated
|
|
33
37
|
- `ProbeHosts` **must** be Threaded. Host ids come from `Thread.list` and `Ractor.current` of the client's ractor, so a probe from another Ractor would announce the wrong hosts. It may read `rql_client` (same ractor).
|
|
34
|
-
- `FlushZombies` is Isolated. The heavy `SCAN`/`DEL` work runs in its own Ractor with its own Redis connection, isolated from the app's ractor and objects. Everything it needs is passed into `Ractor.new(...)` as copied plain values (`config.slice('swarm.flush_zombies.redis_config')`, which `dup`s values, plus Integers). It never touches `rql_client` inside the ractor.
|
|
38
|
+
- `FlushZombies` is Isolated. The heavy `SCAN`/`ZSCAN`/`DEL` work runs in its own Ractor with its own Redis connection, isolated from the app's ractor and objects. Everything it needs is passed into `Ractor.new(...)` as copied plain values (`config.slice('swarm.flush_zombies.redis_config')`, which `dup`s values, plus Integers and the `lock_index_enabled` boolean). It never touches `rql_client` inside the ractor (a changed `config['lock_index_enabled']` takes effect after the next `swarm!`).
|
|
35
39
|
|
|
36
40
|
## Element lifecycle (both bases)
|
|
37
41
|
Each element has two layers, with different code for each base:
|
|
@@ -89,7 +93,7 @@ Public element API (called by `Swarm`/`Supervisor` only):
|
|
|
89
93
|
- The supervisor block swallows errors (`yield rescue nil`), so a failing element never kills the visor.
|
|
90
94
|
|
|
91
95
|
## Config (`config.rb`)
|
|
92
|
-
`swarm.auto_swarm`, `swarm.supervisor.liveness_probing_period` (s). Per element: `swarm.<el>.enabled_for_swarm`, a period (`probe_hosts.probe_period` s, `flush_zombies.zombie_flush_period` s), and the 4-key `swarm.<el>.redis_config.{sentinel,pooled,config,pool_config}` group. Flush extras: `zombie_ttl` (ms), `zombie_lock_scan_size
|
|
96
|
+
`swarm.auto_swarm`, `swarm.supervisor.liveness_probing_period` (s). Per element: `swarm.<el>.enabled_for_swarm`, a period (`probe_hosts.probe_period` s, `flush_zombies.zombie_flush_period` s), and the 4-key `swarm.<el>.redis_config.{sentinel,pooled,config,pool_config}` group. Flush extras: `zombie_ttl` (ms), `zombie_lock_scan_size` (`SCAN` count or lock index `ZSCAN` count, and lock batch size), `zombie_queue_scan_size` (lock queue batch size). The lookup mode is the global `lock_index_enabled` (no swarm-specific setting).
|
|
93
97
|
|
|
94
98
|
## Claude rules
|
|
95
99
|
1. Swarm operations are stateless class-level functions (`ProbeHosts.probe_hosts`, `FlushZombies.flush_zombies`, `ZombieInfo.*`, `Acquirers.acquirers`) that take `redis_client` plus plain values. The manual public API and the main loop call the same function. Don't put Redis logic in instance methods.
|
|
@@ -102,7 +106,7 @@ Public element API (called by `Swarm`/`Supervisor` only):
|
|
|
102
106
|
8. Main loop threads must have `abort_on_exception = false` (both control units set this) and let errors kill only the loop. Recovery belongs to the supervisor, so don't add retry loops inside elements.
|
|
103
107
|
9. Stop the supervisor before killing or re-creating elements, or it will race and resurrect them.
|
|
104
108
|
10. Report element status as the nested `{ running:, state: }` hashes with `'non_initialized'` for missing parts. Update `swarm_status` RBS types when the shape changes.
|
|
105
|
-
11. Zombie detection compares probe scores (`Time.now.to_f`) with `Resource.calc_zombie_score`. Keep `zombie_ttl` in ms at the API and convert with `/ 1_000.0`. Use `Resource::SWARM_KEY`/`*_PATTERN` and never hardcode keys.
|
|
109
|
+
11. Zombie detection compares probe scores (`Time.now.to_f`) with `Resource.calc_zombie_score`. Keep `zombie_ttl` in ms at the API and convert with `/ 1_000.0`. Use `Resource::SWARM_KEY`/`LOCK_INDEX_KEY`/`*_PATTERN` and never hardcode keys. Find locks with both lookups (`lock_index_enabled`): one keyspace `SCAN` pass (`KEY_PATTERN` + `Resource.lock_key_family`) or the lock index (`ZSCAN` + `Resource.prepare_*` keys of each name, never a keyspace `SCAN` then); keep the batch helpers shared between them.
|
|
106
110
|
12. Mirror every change in `sig/redis_queued_locks/swarm/**` and keep the existing `# steep:ignore` on nil-narrowed `Thread?`/`Ractor?` calls (Steep doesn't narrow `attr_reader` results).
|
|
107
111
|
13. Isolated communication goes only through `Ractor::Port`s: create the results port in the main Ractor (`swarm!`) and the command port inside the element Ractor (`.swarm_loop`), and register `Ractor#monitor` on the results port before waiting on it. Every command gets exactly one reply, sent before the Ractor leaves its loop. Treat Symbol replies as death notices. Don't use `Ractor.receive`, `Ractor#send` or the default port, and never share one port between elements. Naming: port variables, attributes and params end in `_port` (`swarm_element_results_port`, `swarm_element_commands_port`). Block (proc) params use short initial-letter names (`r_res_p` for the results port, `r_com_p` for the commands port), like the other abbreviated Ractor block params (`rc`, `z_ttl`).
|
|
108
112
|
14. Inside an element Ractor, terminate threads with kill + join (`.terminate_thread`) and keep `:kill` terminating every thread of the Ractor. Otherwise its status lags as `running`.
|
|
@@ -116,5 +120,6 @@ Public element API (called by `Swarm`/`Supervisor` only):
|
|
|
116
120
|
- `sleep(0.1)` "give a timespot" waits in `swarm!`/`deswarm!`/`observe!` instead of real readiness signalling.
|
|
117
121
|
- RBS runtime type checking (`rbs/test/setup`, the `typecheck-runtime` CI job) can't run inside Ractors. Its hooks on methods called in the element Ractor (`.swarm_loop`, `.flush_zombies`) read `RBS.logger` and raise `Ractor::IsolationError`, so Isolated elements die at startup and the swarm specs fail only under that job.
|
|
118
122
|
- Swarm has no logging or instrumentation, and supervisor errors are silently dropped (`TODO: (CHECK)`).
|
|
119
|
-
-
|
|
120
|
-
- Zombie
|
|
123
|
+
- Every process with a swarm runs its own zombie flusher: N processes make N lookup passes per flush period (no leader election yet); with the default keyspace lookup each pass is a full keyspace `SCAN` (the reason for the lock index).
|
|
124
|
+
- Zombie write locks are checked (`HMGET`) and deleted (`DEL`) in two round trips without `WATCH`: a lock that expires and is re-acquired by a live acquirer in between can be deleted (kept by the user's decision).
|
|
125
|
+
- `zombie_acquirers`/`zombies_info` report only the acquirers of zombie locks, while `flush_zombies` also drops the requests of zombie hosts that hold no locks and reports their acquirers in `deleted_zombie_acquirers`.
|
data/.claude/rules/tests.md
CHANGED
|
@@ -19,9 +19,15 @@ paths:
|
|
|
19
19
|
- Logger/instrumenter fakes are anonymous classes (`Class.new { def debug(...); def notify(event, payload = {}) }`) collecting calls into arrays.
|
|
20
20
|
- Concurrency is tested with real `Thread.new` and `sleep` to wait for async swarm elements; assertions use `match(...)`, `eq(...).or(eq(...))`, `include`, `raise_error(RedisQueuedLocks::...Error)`.
|
|
21
21
|
- Every example cleans up its own state (`client.clear_locks`, `deswarmize!`, `redis.close`).
|
|
22
|
+
- Command-layout specs observe the sent commands through a separate pool on db 0 built with a recording `redis-client` middleware (a `Module` with `define_method(:call_pipelined) { |commands, config, &block| ...; super(commands, config, &block) }`: the block must be forwarded explicitly), e.g. `lock state pipeline layout` (it reads the position constants of `TryToLock`). Update their expectations deliberately when a pipeline layout changes.
|
|
22
23
|
- Coverage: SimpleCov line + branch, HTML only; `minimum_coverage 100` is a TODO.
|
|
24
|
+
- Both lock lookup modes (`config['lock_index_enabled']`): the behavior examples are `RSpec.shared_examples 'redis_queued_locks'` (the `redis` pool and the FLUSHDB hooks are `RSpec.shared_context 'redis_queued_locks redis'`), included by `RSpec.describe RedisQueuedLocks` twice: `context 'with the disabled lock index (keyspace SCAN)', lock_index_enabled: false` and `context 'with the enabled lock index (lock index ZSCAN)', lock_index_enabled: true`; the `around` hook of `spec_helper` switches the default of `config['lock_index_enabled']` for the example by this metadata (`RedisQueuedLocks::Config.setting`, the library default is restored after the example), so `RedisQueuedLocks::Client.new(redis)` inside the shared examples gets the mode of the context (explicit config values win). One `bundle exec rake rspec` runs the shared examples in both modes (~2x time).
|
|
25
|
+
- Mode-dependent expectations inside the shared examples branch on `client.config['lock_index_enabled']` (e.g. `lock state pipeline layout`: 5 queueing commands, 9 with the lock index; `lock-iterating operations: one pass (keyspace or lock index) with small batches`: `#keys` lists other `rql:lock*` keys only with the keyspace lookup); keys written directly with raw commands (`ZADD` queues, `HSET` locks) are found by the keyspace SCAN and are indexed with `client.reindex_locks if client.config['lock_index_enabled']` before lock-iterating assertions.
|
|
26
|
+
- Lock index specifics live in `describe 'lock index'` (outside the shared examples: run once, every client enables the lock index explicitly with `config['lock_index_enabled'] = true`; `let(:index_score_ahead)` = index score minus the current client ms, `let(:upper_bound)` = `be_between(lifetime + grace - 1_000, lifetime + grace)`); `the lock obtaining transaction indexes the lock` drops the index entry right before the obtaining `MULTI` with a middleware (keep it when touching the obtaining transaction); `the disabled lock index (default): no lock index commands, keyspace lookups` records all commands of the default mode (no `rql:index:locks` argument, no `ZSCAN`, queues without TTL); `enabling the lock index: ...` covers the migration (`#reindex_locks` after enabling).
|
|
23
27
|
|
|
24
28
|
## Claude rules
|
|
29
|
+
The suite is pending a total rework (overview "Conventions"): these rules guide new examples, but their violations in `*_spec.rb` are not code review findings until the rework; the review checks only that the changed library behavior is covered and that changed expectations are correct (`.claude/skills/rql-code-review`, §3).
|
|
30
|
+
|
|
25
31
|
1. Run specs with a local Redis available: `bundle exec rake rspec` (single example: `bundle exec rspec spec/redis_queued_locks_spec.rb:<line>`).
|
|
26
32
|
2. Add new examples to `spec/redis_queued_locks_spec.rb` with `specify '<feature>'`, inside an existing `describe` when one fits; don't create new spec files unless asked (the suite is pending a rework).
|
|
27
33
|
3. Use only `expect` syntax; no `should`, no monkey-patched DSL, no `focus` left behind.
|
|
@@ -25,6 +25,7 @@ paths:
|
|
|
25
25
|
- `x != nil` does not narrow `String?`/`Float?` in `elsif` branches or `&&` chains on locals typed from tuples: use truthiness (`elsif x`) or total conversions (`x.to_f > now`, `ttl.to_i`);
|
|
26
26
|
- `pipelined`/`call` return `untyped`: annotate destructured pipeline results with a tuple (`# @type var lock_state: [String?, Array[String], Float?, Array[[String, Float]]]`);
|
|
27
27
|
- `LockSeriesPoC` has no RBS: new calls/definitions there need `# steep:ignore` (like the existing ones).
|
|
28
|
+
- splatting a `Set` (`call(*set)`) is rejected (`Ruby::ArgumentTypeMismatch`: splat arguments must be arrays), and no honest inline annotation fixes it; at runtime Ruby converts a splatted `Set` with `Set#to_a` on every call.
|
|
28
29
|
- Steep 2 narrows on `==` with literals (`x == :sym` makes `x` the literal type `:sym`) and rejects `# @type var` annotations that widen a narrowed variable; RBS 4's `Kernel#Array` has a `(nil) -> []` overload that `untyped` arguments resolve to, so `Array(rconn.call(...)).first` needs a `#: T` assertion.
|
|
29
30
|
- Runtime checking: CI runs specs with `RUBYOPT=-rrbs/test/setup RBS_TEST_TARGET='RedisQueuedLocks::*'`, so signatures must match real runtime values, not just Steep's view.
|
|
30
31
|
|
|
@@ -33,7 +34,7 @@ paths:
|
|
|
33
34
|
2. Create new `.rbs` files at the mirrored path with nested module blocks and `use RedisQueuedLocks as RQL` / `use RedisClient as RC` aliases when needed.
|
|
34
35
|
3. Reuse shared types (`RQL::loggerObj`, `RQL::instrObj`, `samplerObj`, `RC::client`) instead of `untyped`; use `untyped` only for truly dynamic values (e.g. user `meta`, `instrument`).
|
|
35
36
|
4. Name public API result hashes with a `type xxxResult = { ok: bool, result: ... }` alias next to the method. Type internal swarm element replies with plain types (`bool`, `String`, a flat record such as `{ alive: bool, state: String }`, optional `?` for `nil`) instead of `{ ok:, result: }` records.
|
|
36
|
-
5. Prefer fixing types or adding `# @type var` / `#: T` annotations over `# steep:ignore`; add `# steep:ignore` only where Steep reports a diagnostic. Strict mode fails the build on `Ruby::RedundantIgnoreComment`, so never add a speculative ignore and remove ones that become redundant.
|
|
37
|
+
5. Prefer fixing types or adding `# @type var` / `#: T` annotations over `# steep:ignore`; add `# steep:ignore` only where Steep reports a diagnostic. Strict mode fails the build on `Ruby::RedundantIgnoreComment`, so never add a speculative ignore and remove ones that become redundant. Never shape runtime code around Steep limitations (conversions, copies or allocations made only to satisfy Steep): annotate it, or use a targeted `# steep:ignore` with a `NOTE: (steep)` comment when no honest annotation exists. Keep a conversion only when it is cheaper at runtime and say so in its comment (e.g. a `Set` splatted in a loop: build the array once instead of a `Set#to_a` per call).
|
|
37
38
|
6. Don't loosen `Steep::Diagnostic::Ruby.strict` or add `ignore` entries to the `Steepfile`.
|
|
38
39
|
7. New third-party gem used in `lib/`: add it to `rbs_collection.yaml` (and `sig/manifest.yml` for stdlib) or write a minimal stub in `sig/vendor/`.
|
|
39
40
|
8. Verify with `bundle exec rbs collection install && bundle exec rake steep:check`; for signature/runtime mismatches run the specs under RBS runtime testing (command in `.github/workflows/typecheck-runtime.yml`).
|
data/.claude/rules/visitors.md
CHANGED
|
@@ -57,7 +57,7 @@ Not covered by visitors: `release_lock`, `release_read_lock`, `release_all_locks
|
|
|
57
57
|
9. Instrumentation: `instrumenter.notify('redis_queued_locks.<event>', { ... }) rescue nil` with shorthand Symbol keys; reuse the standard payload keys (`lock_key`, `rw_mode`, `ttl`, `acq_id`, `hst_id`, `ts`, `acq_time`, `hold_time`, `instrument`). Event names are public API: never rename or remove one without being asked, and document new ones in the README/CHANGELOG.
|
|
58
58
|
10. Use the same method name in LogVisitor and InstrVisitor when both report the same event.
|
|
59
59
|
11. Add the method to the mirrored RBS visitor file (`RQL::loggerObj` / `RQL::instrObj`, `bool` sampled flag, `-> void`) and cover the new event in specs via the fake logger/notifier (assert on the `[redis_queued_locks.<event>]` prefix or event name).
|
|
60
|
-
12. Never pass Redis reads (or other costly computations) as visitor arguments unconditionally: arguments are evaluated before the visitor's `return unless` guard. Gate them at the call site with the same condition (`(log_sampled && log_lock_try) ? rconn.call('HGETALL', ...).to_h : {}`).
|
|
60
|
+
12. Never pass Redis reads (or other costly computations) as visitor arguments unconditionally: arguments are evaluated before the visitor's `return unless` guard. Gate them at the call site with the same condition (`(log_sampled && log_lock_try) ? rconn.call('HGETALL', ...).to_h : NO_LOCK_DATA`; the fallback is the shared frozen `TryToLock::NO_LOCK_DATA`, not a new `{}`).
|
|
61
61
|
13. A new branch/exit of the read/write algorithm gets its own TryToLock event (`exit__write_request_ahead`, `exit__read_request_ahead`, `exit__read_lock_still_obtained`, `single_process_lock_conflict__lock_upgrade` are the existing ones). Don't add events to the default write success path: specs assert its exact log sequence (10 lines with `log_lock_try`).
|
|
62
62
|
14. Any change of observable data is documented in README in the same change: a new log event or a new/renamed/removed logged key (`LogVisitor`s) → `## Logging` (both lists and the matching `config['logger']` / `config['log_lock_try']` comments in `### Logging Configuration`); a new instrumentation event or a new/renamed/removed payload key (`InstrVisitor`s and the inline `notify` of release modules) → `### Instrumentation Events` (event list + payload line with type and semantics, e.g. requested vs held `rw_mode`). Add a CHANGELOG `[Unreleased]` line as well. Before finishing, compare the README lists with the visitor code (event names and key order).
|
|
63
63
|
|