redis_queued_locks 1.21.0 → 1.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/project-overview.md +294 -51
- data/.claude/rules/acquirer.md +16 -11
- data/.claude/rules/arguments.md +2 -2
- data/.claude/rules/changelog.md +27 -0
- data/.claude/rules/git.md +1 -1
- data/.claude/rules/logic.md +14 -15
- data/.claude/rules/swarm.md +3 -4
- data/.claude/rules/tests.md +9 -9
- data/.claude/rules/type-checking.md +14 -16
- data/.claude/rules/visitors.md +11 -9
- data/.claude/skills/rql-code-review/SKILL.md +129 -47
- data/.ruby-version +1 -1
- data/CHANGELOG.md +122 -245
- data/CLAUDE.md +12 -10
- data/README.md +516 -175
- data/Rakefile +11 -0
- data/lib/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rb +4 -4
- data/lib/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rb +28 -12
- data/lib/redis_queued_locks/acquirer/acquire_lock/log_visitor.rb +38 -16
- data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rb +92 -15
- data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +945 -272
- data/lib/redis_queued_locks/acquirer/acquire_lock.rb +82 -32
- data/lib/redis_queued_locks/acquirer/clear_dead_requests.rb +240 -45
- data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +15 -13
- data/lib/redis_queued_locks/acquirer/keys.rb +6 -2
- data/lib/redis_queued_locks/acquirer/lock_series/instr_visitor.rb +243 -0
- data/lib/redis_queued_locks/acquirer/lock_series/log_visitor.rb +472 -0
- data/lib/redis_queued_locks/acquirer/lock_series/release_lock_series.rb +181 -0
- data/lib/redis_queued_locks/acquirer/lock_series/try_to_lock_series.rb +1051 -0
- data/lib/redis_queued_locks/acquirer/lock_series.rb +948 -0
- data/lib/redis_queued_locks/acquirer/locks.rb +6 -2
- data/lib/redis_queued_locks/acquirer/queue_info.rb +30 -7
- data/lib/redis_queued_locks/acquirer/queues.rb +17 -6
- data/lib/redis_queued_locks/acquirer/release_all_locks.rb +6 -2
- data/lib/redis_queued_locks/acquirer/release_locks_of.rb +6 -2
- data/lib/redis_queued_locks/acquirer.rb +1 -1
- data/lib/redis_queued_locks/client.rb +116 -44
- data/lib/redis_queued_locks/config.rb +93 -1
- data/lib/redis_queued_locks/resource.rb +124 -9
- data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +6 -5
- data/lib/redis_queued_locks/swarm/zombie_info.rb +6 -2
- data/lib/redis_queued_locks/version.rb +2 -2
- data/rbs_collection.lock.yaml +4 -0
- data/rbs_collection.yaml +2 -1
- data/redis_queued_locks.gemspec +21 -14
- data/sig/manifest.yml +1 -0
- data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue/log_visitor.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rbs +12 -6
- data/sig/redis_queued_locks/acquirer/acquire_lock/log_visitor.rbs +21 -14
- data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rbs +61 -40
- data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +160 -11
- data/sig/redis_queued_locks/acquirer/acquire_lock/{yield_with_expire → yield_expire}/log_visitor.rbs +4 -4
- data/sig/redis_queued_locks/acquirer/acquire_lock/yield_expire.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/acquire_lock.rbs +5 -3
- data/sig/redis_queued_locks/acquirer/clear_dead_requests.rbs +14 -1
- data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +2 -2
- data/sig/redis_queued_locks/acquirer/lock_series/instr_visitor.rbs +102 -0
- data/sig/redis_queued_locks/acquirer/lock_series/log_visitor.rbs +154 -0
- data/sig/redis_queued_locks/acquirer/lock_series/release_lock_series.rbs +37 -0
- data/sig/redis_queued_locks/acquirer/lock_series/try_to_lock_series.rbs +198 -0
- data/sig/redis_queued_locks/acquirer/lock_series.rbs +106 -0
- data/sig/redis_queued_locks/acquirer/queues.rbs +1 -1
- data/sig/redis_queued_locks/client.rbs +77 -7
- data/sig/redis_queued_locks/config.rbs +54 -1
- data/sig/redis_queued_locks/instrument/sampler.rbs +3 -1
- data/sig/redis_queued_locks/instrument/void_notifier.rbs +3 -1
- data/sig/redis_queued_locks/logging/sampler.rbs +3 -1
- data/sig/redis_queued_locks/resource.rbs +22 -12
- data/sig/redis_queued_locks/swarm/swarm_element/threaded.rbs +1 -1
- data/sig/redis_queued_locks/utilities.rbs +2 -1
- data/sig/redis_queued_locks.rbs +7 -0
- metadata +35 -13
- data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +0 -56
- data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +0 -95
- data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +0 -484
- /data/sig/redis_queued_locks/{acquier.rbs → acquirer.rbs} +0 -0
data/.claude/rules/acquirer.md
CHANGED
|
@@ -13,8 +13,7 @@ paths:
|
|
|
13
13
|
- `Locks.extract_locks_info` / `Locks.read_lock_info` duplicate the write/read lock formatting of `LockInfo.lock_info` / `LockInfo.read_lock_info` (`Locks` fetches the data of a batch of locks in two round trips and `Locks.read_lock_info` formats the prefetched read lock data);
|
|
14
14
|
- `Queues.extract_queues_info` duplicates the request formatting of `QueueInfo.queue_info`;
|
|
15
15
|
- each module declares its own RBS aliases (`Locks::lockInfo` / `Locks::readerInfo` mirror `LockInfo::lockInfo` / `LockInfo::readerInfo`); only `Client` signatures reference them.
|
|
16
|
-
- `AcquireLock`
|
|
17
|
-
- PoC modules (`*PoC`, e.g. `LockSeriesPoC`) are not covered by this principle: they may reuse operation modules and the `AcquireLock` core (`LockSeriesPoC` uses `AcquireLock.acquire_lock` and its `YieldExpire` mixin).
|
|
16
|
+
- `AcquireLock` and `LockSeries` are the lock acquisition core, not operation modules of this list: `LockSeries` reuses the `AcquireLock` mixins (`TryToLockSeries` includes `TryToLock`, `LockSeries` extends `DelayExecution` and `YieldExpire`) and never calls `AcquireLock.acquire_lock`.
|
|
18
17
|
- Known debt (don't copy): `AcquireLock::WithAcqTimeout` calls `LockInfo.lock_info` / `QueueInfo.queue_info` for detailed timeout errors (its TODO asks to make `AcquireLock` independent of them).
|
|
19
18
|
- Known limitation (kept by the user's decision): `ReleaseLocksOf` (and `FlushZombies`) check the lock owner (`HMGET`) and delete the lock (`DEL`) in two round trips without `WATCH`: a lock re-acquired in between by another acquirer can be deleted.
|
|
20
19
|
- Known limitation (kept by the user's decision, documented in README `#lock`): `YieldExpire` releases the write lock (`EXPIRE 0`) and decreases its ttl (Lua) without an owner check: a block (without `timed`) longer than the lock ttl can delete/shorten the lock of another acquirer. Read lock release/decrease is per acquirer (owner-safe).
|
|
@@ -23,7 +22,7 @@ paths:
|
|
|
23
22
|
- Mutating operations: long positional lists ending with the fixed observability tail
|
|
24
23
|
`logger, instrumenter, instrument, log_sampling_enabled, log_sampling_percent, log_sampler, log_sample_this, instr_sampling_enabled, instr_sampling_percent, instr_sampler, instr_sample_this`
|
|
25
24
|
(the same order in all mutating modules).
|
|
26
|
-
- Only `AcquireLock.acquire_lock` and `
|
|
25
|
+
- Only `AcquireLock.acquire_lock` and `LockSeries.lock_series` use keyword args (`process_id:`, `thread_id:`, ...).
|
|
27
26
|
- **Two-layer mutating operations** (`release_lock`, `release_read_lock`, `release_all_locks`, `release_locks_of`, `extend_lock_ttl`, `clear_dead_requests`, `reindex_locks`; observability rule: `logic.md#16`):
|
|
28
27
|
1. argument validations (`RedisQueuedLocks::ArgumentError`) first, then `rel_start_time = clock_gettime` (`ext_`/`idx_` prefixes for the extension/reindexing)
|
|
29
28
|
2. call a private `fully_*` helper with the Redis work (returns `{ ok:, result: }`, destructured with `fully_x(...) => { ok:, result: }`, or the raw data: a `Set`/`Integer`)
|
|
@@ -39,10 +38,10 @@ paths:
|
|
|
39
38
|
- Raw commands: `redis.call('CMD', ...)` with uppercase string command names and string args (`'0'`, `'-inf'`, `'+inf'`).
|
|
40
39
|
- Pooled connection: wrap multi-command work in `redis.with do |rconn| ... end`.
|
|
41
40
|
- Atomic writes: `rconn.multi do |transact| ... end` (or `multi(watch: [lock_key])` in `TryToLock`); batch reads: `pipelined do |pipeline| ... end` then index `result[0]`, `result[1]` into named vars (`hget_cmd_res`, `pttl_cmd_res`).
|
|
42
|
-
- Internal timeouts (`LockAcquirementIntermediateTimeoutError`, `TimedLockIntermediateTimeoutError`) inherit from `RedisQueuedLocks::Exception` (`::Exception`, `logic.md#15`): redis-client sends `UNWATCH` on an interrupted `multi(watch:)` only for `StandardError`s, so a connection interrupted inside a WATCH block would return to the pool with the active WATCH (the next `EXEC` on it is aborted when a watched key changes). Every `redis.with do |rconn| ... end` block with a `multi(watch:)` (`TryToLock`, `ExtendLockTTL` read extensions, `
|
|
41
|
+
- Internal timeouts (`LockAcquirementIntermediateTimeoutError`, `TimedLockIntermediateTimeoutError`) inherit from `RedisQueuedLocks::Exception` (`::Exception`, `logic.md#15`): redis-client sends `UNWATCH` on an interrupted `multi(watch:)` only for `StandardError`s, so a connection interrupted inside a WATCH block would return to the pool with the active WATCH (the next `EXEC` on it is aborted when a watched key changes). Every `redis.with do |rconn| ... end` block with a `multi(watch:)` (`TryToLock`, `TryToLockSeries`, `ExtendLockTTL` read extensions, `ReleaseLockSeries#release_write_lock_series`) ends with `rescue <both internal timeouts>, ::Timeout::ExitException` (the user's `Timeout.timeout` without an exception class interrupts with it the same way) → `rconn.close` → `raise`.
|
|
43
42
|
- Iteration: two lock lookup modes, switched by `config['lock_index_enabled']` (client-wide, `false` by default): `Client` (and the `Swarm` facade, the zombie flusher Ractor via a copied value) passes `lock_index_enabled` to every lock-iterating module (`Locks`, `Queues`, `Keys`, `ReleaseAllLocks`, `ReleaseLocksOf`, `ClearDeadRequests`, `FlushZombies`, `ZombieInfo`). Each module keeps both lookups side by side and shares the processing (lock info formatting, flush/release helpers, instrumentation) in one copy; the lookup is chosen in one place (`lock_index_enabled ? index_x : scan_x` or one `if lock_index_enabled ... else ... end` with `(lock index)` / `(keyspace)` step comments):
|
|
44
43
|
- keyspace (the disabled lock index, the pre-index code): one SCAN pass over `KEY_PATTERN` (or `ALL_LOCK_QUEUES_PATTERN` for queues) per operation, keys classified by `Resource.lock_key_family`, per-key commands in pipelined batches collected across SCAN pages (`scan_locks`, `scan_queues`, `each_lock_queue`, `flush_keyspace_zombies`, `each_keyspace_zombie_lock`); no lock index command is sent;
|
|
45
|
-
- lock index (enabled): one `ZSCAN` pass over the lock index per operation (`each_lock_name_batch(rconn, scan_size) { |lock_names| ... }`: `zscan(Resource::LOCK_INDEX_KEY, count: scan_size)`, names with an expired score are skipped, batches of `scan_size` names collected across ZSCAN pages, the rest yielded after the pass); keys are derived from the names (`Resource.prepare_*`, reader data via registry members) and per-key commands are sent in one pipeline (or one variadic command / `MULTI`) per batch; info queries check existence (`EXISTS`, empty `ZRANGE` = missing registry/queue); the keyspace is never scanned (only `ReindexLocks` scans `KEY_PATTERN` with `Resource.lock_key_family` to adopt keys without index entries, in both modes). Each operation module has its own `each_lock_name_batch` copy (same name and shape, duplicated on purpose; the swarm modules iterate inline: `flush_lock_index_zombies`, `each_lock_index_zombie_lock`).
|
|
44
|
+
- lock index (enabled): one `ZSCAN` pass over the lock index per operation (`each_lock_name_batch(rconn, scan_size) { |lock_names| ... }`: `zscan(Resource::LOCK_INDEX_KEY, count: scan_size)`, names with an expired score are skipped (`Resource.lock_index_score_expired?`: ZSCAN scores are Strings, the `+inf` score of a key without TTL is `"inf"` and never expires; never `to_f` alone: it gives `0.0`), batches of `scan_size` names collected across ZSCAN pages, the rest yielded after the pass); keys are derived from the names (`Resource.prepare_*`, reader data via registry members) and per-key commands are sent in one pipeline (or one variadic command / `MULTI`) per batch; info queries check existence (`EXISTS`, empty `ZRANGE` = missing registry/queue); the keyspace is never scanned (only `ReindexLocks` scans `KEY_PATTERN` with `Resource.lock_key_family` to adopt keys without index entries, in both modes). Each operation module has its own `each_lock_name_batch` copy (same name and shape, duplicated on purpose; the swarm modules iterate inline: `flush_lock_index_zombies`, `each_lock_index_zombie_lock`).
|
|
46
45
|
- Never one round trip per key in either mode.
|
|
47
46
|
- Lua: frozen heredoc constant (`<<~LUA_SCRIPT.strip.tr("\n", '').freeze`) + `call('EVAL', SCRIPT, 1, key, arg)`.
|
|
48
47
|
- Release = `EXPIRE key 0` / `ZREMRANGEBYSCORE queue -inf +inf`; a read lock is released with `ZREM <readers> <acq_id>` + `DEL <reader data>` (owner-safe, other readers untouched); Redis TTL sentinels handled explicitly (`PTTL` `-2` = missing, `-1` = no expiry → `Float::INFINITY`).
|
|
@@ -55,17 +54,24 @@ paths:
|
|
|
55
54
|
- a command that **extends** a lock key is followed by `ZADD <index> XX INCR <ms> <lock_name>` in the same transaction (`XX`: names without an entry are not indexed with a wrong score); ttl decreases and releases never touch the index (the bound stays an upper bound);
|
|
56
55
|
- lock request queues get `PEXPIRE NX` + `PEXPIRE GT` with the queue ttl (ms, `(queue_ttl * 1_000).ceil`) on each queueing: every indexed key has a bounded lifetime;
|
|
57
56
|
- pruning is `ZREMRANGEBYSCORE <index> -inf <lock_index_time - 1>` (each queueing, `flush_zombies`, `clear_locks`, `clear_dead_requests`); keys written without the index (older versions, raw writes) are adopted by `ReindexLocks` (SCAN pass, `PEXPIRE NX` for queues, `PTTL` => `ZADD GT`).
|
|
57
|
+
- **Lock request priorities**: queue scores are the enqueue time shifted by the priority band (`Resource.calc_initial_acquirer_position(priority)` = `ts - priority * Resource::LOCK_PRIORITY_BAND`, 2**32 s; priority `Resource::MIN_LOCK_PRIORITY`..`MAX_LOCK_PRIORITY`, -100..100, or a priority name of `config['lock_priorities']` resolved to its Integer in `AcquireLock` Step 0.1 and `LockSeries` Step 0.1: code after the resolution never sees Symbols). Never prune a lock queue with one `-inf .. now - ttl` range (it drops alive high-priority requests and keeps dead low-priority ones): prune a band with `Resource.lock_priority_band_floor(p)` .. `Resource.lock_priority_band_dead_score(dead_score, p)` (`TryToLock#pipeline_dead_requests_removal`, `ClearDeadRequests.drop_dead_requests` / `drop_dead_requests_by_priority`); full clears (`-inf .. +inf`) and member-based removals (`ZREM`) are band-independent. Never enumerate the priority range per attempt or per queue (the cost must not depend on it): prune the bands found in the data (decode the band and the liveness of a score with `Resource.lock_request_priority(score)` and `lock_priority_band_dead_score`): the attempt prunes its own band and the bands of the dead blocking requests (`TryToLock#drop_dead_requests_ahead`; a lock series: `TryToLockSeries#drop_series_dead_requests_ahead`, one round trip for all its locks), `ClearDeadRequests` the bands whose first request is dead (all requests of the batch queues are read; `by_priority: true`: a walk over the bands reads only the first request of each band, `ZRANGE <queue> <bound> +inf BYSCORE LIMIT 0 1`, one round trip per band of the batch queue with the most bands + 1, the dead bands are pruned by the next round trip after its reads, `drop_dead_requests_by_priority`; every walk step must make progress: an exclusive bound when the next band edge rounds to the position). Decode priorities with `Resource.lock_request_priority(score)` (info results); the cross-mode `(score, acq_id)` comparison works on shifted scores as is. Infinite positions (manual writes, never written by RQL) are out of all bands: the decoding maps `-inf`/`+inf` to `MAX_LOCK_PRIORITY`/`MIN_LOCK_PRIORITY` (never `Float#floor` of an infinity: `FloatDomainError`); a request at `-inf` is always dead and no band range covers it: it is dropped by its position (`ZREMRANGEBYSCORE <queue> -inf -inf`: `drop_dead_requests_ahead` for a blocking one, `ClearDeadRequests` for the first ones of a queue); a request at `+inf` is never dead.
|
|
58
58
|
- Redis never keeps empty sorted sets/hashes: an empty `ZRANGE`/`HGETALL` result already means a missing key, so don't pair such reads with `EXISTS` (`QueueInfo`, `Queues`).
|
|
59
59
|
- **Data normalization**: Redis hash strings are converted with `Float(...)` / `Integer(...)` inside `hget_cmd_res.tap do |lock_data| ... end`, optional fields guarded with `if lock_data['x']`.
|
|
60
60
|
- **AcquireLock**:
|
|
61
61
|
- Main module `require_relative`s its parts, then `extend`s the mixins (`TryToLock`, `DelayExecution`, `YieldExpire`, `WithAcqTimeout`, `DequeueFromLockQueue`); mixins are plain modules with instance methods (`def try_to_lock(...)`), visitors are `class << self` modules called explicitly (see `visitors.md`).
|
|
62
62
|
- The algorithm is a numbered step script (`# Step 0`, `# Step 2.1`, `# Step 2.2.a`) driven by a mutable `acq_process` hash (`:should_try`, `:tries`, `:acquired`, `:result`, `:lock_info`, timings).
|
|
63
63
|
- Failure modes are Symbols (`:fail_fast_no_try`, `:fail_fast_after_try`, `:conflict_dead_lock`, `:conflict_lock_upgrade`, ...); exceptions are raised only when `raise_errors` is true, with a message naming the lock key / acquirer id.
|
|
64
|
-
- `TryToLock` pre-check (retries: `acquire_lock` passes `retry_attempt = tries > 0`, and `fail_fast` attempts): `precheck_lock` reads the lock state with the same pipeline (`pipeline_lock_state`; with the queueing for retries, without it for `fail_fast`) without `WATCH`; it returns the failure result (`lock_obstacle` / `:fail_fast_no_try`) and logs the same events as the failed attempt, or `nil` to continue the attempt (lock obtainable or a same-process conflict). The pre-check never obtains the lock: mutual exclusion stays with `WATCH`/`MULTI`.
|
|
65
|
-
- `TryToLock` attempt (both modes): `multi(watch:)` (write: `[lock_key, lock_readers_key]`, read: `[lock_key]`) → one pipelined state read (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only, the write-only tail of the pipeline) → same-process conflict detection (`sp_conflict_rw_mode` = held `:write`/`:read`) → the lock request queueing in one pipelined round trip (`queue_lock_request`: `ZADD NX` into the queue of the requested mode
|
|
64
|
+
- `TryToLock` pre-check (retries: `acquire_lock` passes `retry_attempt = tries > 0`, and `fail_fast` attempts): `precheck_lock` reads the lock state with the same pipeline (`pipeline_lock_state`; with the queueing for retries, without it for `fail_fast`) without `WATCH`; it returns the failure result (`lock_obstacle` / `:fail_fast_no_try`) and logs the same events as the failed attempt in the same order (the dead requests dropped by Step 2.2 after the queueing events: `drop_dead_requests_ahead` returns them, `log_dropped_requests` logs them in both paths; a pre-check that does not fail logs them before the attempt), or `nil` to continue the attempt (lock obtainable or a same-process conflict). The pre-check never obtains the lock: mutual exclusion stays with `WATCH`/`MULTI`.
|
|
65
|
+
- `TryToLock` attempt (both modes): `multi(watch:)` (write: `[lock_key, lock_readers_key]`, read: `[lock_key]`) → one pipelined state read (`HGET acq_id`, `TIME`, own reader score, longest reader: write requests only, the write-only tail of the pipeline) → same-process conflict detection (`sp_conflict_rw_mode` = held `:write`/`:read`) → the lock request queueing in one pipelined round trip (`queue_lock_request`: prune dead requests of the request's priority band in **both** queues (before `ZADD NX`: the old dead position of a requeued request is dropped before its new position is added; the reverse order keeps the old position with `NX` and drops it: the request is missing in the queue for the attempt) → `ZADD NX` into the queue of the requested mode (the position is shifted by the priority band) → queue heads of both queues (`WITHSCORES`) → (the enabled lock index only) request queue `PEXPIRE NX`+`GT` → lock index `ZADD GT` → lock index pruning (5 replies, 9 with the lock index; new commands go to the end, replies after the queue heads are not read); eager queueing (`:wait_for_lock`/`:dead_locking` without `fail_fast`) sends these commands in the state pipeline right after its head, so `lock {}` = 4 round trips; the queueing log events are still emitted in the classic branch in the same order) → Step 2.2 (priorities, `:queued`, also in the pre-check, before the obstacle decision): the dead blocking requests of other bands (the write queue head for writers and the earlier opposite-mode request, decoded from their scores by `dead_request_priority`) are dropped with their bands in both queues and both heads are re-read in one more round trip (`drop_dead_requests_ahead`: one or two bands per round trip, the heads are read from the end of the pipeline, up to `MAX_DEAD_REQUEST_REMOVALS` round trips, `try_lock.dead_request_removed` log per dropped request, written by the caller (`log_dropped_requests`) after the queueing events; no round trip while the blocking requests are alive; a live write queue head stops the removal) (`:queued`: write must be first in its queue; any mode waits for an earlier `(score, acq_id)` request of the opposite mode) → lock state (write held → `:lock_is_still_acquired`; live readers for write / own live read for read → `:read_lock_is_still_acquired`) → MULTI (write: `ZREM` + `HSET` + `PEXPIRE` (+ lock index `ZADD GT`); read: `ZREM` + prune registry + `ZADD expiry` + registry `PEXPIRE NX`+`GT` + reader data `DEL`+`HSET`+`PEXPIRE` (+ lock index `ZADD GT`)). With the enabled lock index the `:extendable_work_through` extension `MULTI` ends with the lock index `ZADD XX INCR` (write: 4 replies, 5 with the lock index: `result.size == (lock_index_enabled ? 5 : 4)`).
|
|
66
66
|
- The try result carries internal `rw_mode` (the **held** lock mode); `acquire_lock` stores it as `acq_process[:held_rw_mode]` and passes it to `yield_expire`, which releases/decreases the held lock. The public `lock` result shape does not include it.
|
|
67
67
|
- Read lock data (`rql:lock_reader:<name>:<acq_id>`) uses the write-lock field format without `acq_id`/`hst_id` (`ts`, `ini_ttl`, meta, `spc_*`: both ids are derived from the acquirer id, `LockInfo.read_lock_info`/`Locks.read_lock_info` put them first in each reader and still accept older data that contains them) and the read lock TTL; it is a data carrier only (writers never read it). Its key (`read_lock_key`) is built only for read lock requests (`nil` for write requests: they never hold read locks).
|
|
68
68
|
- Timing via monotonic `clock_gettime` (microseconds); wall time only for `at:`/`ts` fields.
|
|
69
|
+
- **LockSeries** (`lock_series.rb`, `lock_series/try_to_lock_series.rb`, `lock_series/release_lock_series.rb`; overview "Lock series"):
|
|
70
|
+
- The attempt is the single lock attempt for N locks, built from the shared `TryToLock` helpers: lock state and queueing commands (`pipeline_lock_state`, `queue_lock_request`), decisions (`same_process_conflict_status`, `find_opposite_request_ahead`, `lock_obstacle`, `find_dead_requests_ahead`, `pipeline_dead_requests_ahead_removal`), obtaining and reentrant commands (`transact_*`). A change of these helpers applies to both attempts; a change of the single lock attempt flow is checked against `try_to_lock_series`/`precheck_lock_series` (two copies of the flow, not of the commands).
|
|
71
|
+
- Per-lock keys are parallel arrays in the order of the lock names (`lock_keys, rw_mode, lock_key_queues, read_lock_key_queues, lock_readers_keys, read_lock_keys, lock_names, watched_keys, acquirer_id, ...`), computed once in `lock_series`; `lock_states` (`series_lock_states`) is the per-attempt state of each lock.
|
|
72
|
+
- One `acquirer_position` per series in all its queues: never compute a position per lock (deadlock freedom depends on the same order in all queues).
|
|
73
|
+
- Nothing is obtained outside the one transaction of the attempt, nothing is held between attempts.
|
|
74
|
+
- Every failure and interruption goes through `drop_failed_lock_series` (the `ensure` of the acquirement) and one failure section (events, `raise_lock_series_error`, the failure result): the acquirement loop never raises.
|
|
69
75
|
- **Inline typing**: `# @type var result: Symbol`, `{} #: Hash[String,String|Float|Integer]`, and `# steep:ignore` on pattern-matching destructures and splats (`rconn.call('DEL', *keys) # steep:ignore`).
|
|
70
76
|
|
|
71
77
|
## Claude rules
|
|
@@ -83,18 +89,17 @@ paths:
|
|
|
83
89
|
12. Read/write checklist for any change in this directory:
|
|
84
90
|
- lock state checks: write requires no write lock and no **live** readers; read requires no write lock; same-acquirer reader/writer cases go through `conflict_strategy` (read→write upgrade never "works through");
|
|
85
91
|
- WATCH: writers watch the write key + readers registry, readers watch the write key only (never the registry: readers must not abort each other); don't modify a watched key outside MULTI after WATCH (it aborts your own EXEC);
|
|
86
|
-
- every attempt prunes dead requests in both queues; dequeue/cleanup removes the acquirer from the queue of its mode;
|
|
92
|
+
- every attempt prunes dead requests of its priority band in both queues and the bands of the dead requests that block it (head-driven: never every band of the range); dequeue/cleanup removes the acquirer from the queue of its mode;
|
|
87
93
|
- every release/cleanup/info/zombie path covers write lock, readers registry, reader data and both queues;
|
|
88
94
|
- registry/reader-data TTLs: `PEXPIRE NX` + `PEXPIRE GT` pair to keep max TTL (`GT` alone never sets TTL on a key without one);
|
|
89
95
|
- keep write-only result shapes unchanged; add read keys/fields only when read data exists;
|
|
90
96
|
- lock index (the enabled lock index only, `if lock_index_enabled`): every key-creating command is followed by the index `ZADD GT` (the obtaining `MULTI` too), every extension by `ZADD XX INCR`, names are removed only with all their keys in one `MULTI` (see "Lock index" above); the disabled lock index sends no lock index command.
|
|
91
97
|
13. Values used only for logs (e.g. `HGETALL` of lock data) are fetched only when the log is enabled (`(log_sampled && log_lock_try) ? ... : NO_LOCK_DATA`, a shared frozen hash instead of a new `{}` per attempt): visitor arguments are evaluated before the visitor's guard.
|
|
92
|
-
14. Keep operation modules independent (see "Independence of operation modules";
|
|
93
|
-
15. Pipelines/transactions whose replies are read by position (`lock_state[N]`, `result[N]`): document the reply positions next to the commands (`NOTE: (pipeline layout)`). Conditionally sent commands (e.g. `... unless read_mode`) form a tail after all unconditional commands; read their replies only under the same condition (a command appended after a conditional tail gets a mode-dependent position and its reply is read as another command's reply). Insert new commands before the conditional tail and update the positions, the reply reading, the position constants and the layout spec in the same change. `TryToLock` lock state pipeline (`pipeline_lock_state`, shared by the attempt and the pre-check): head `[LOCK_STATE_OBTAINER_REPLY, LOCK_STATE_REDIS_TIME_REPLY, LOCK_STATE_OWN_READ_LOCK_REPLY]`, queueing commands at `LOCK_STATE_QUEUE_REPLIES_AT` (`queue_lock_request` replies, used ones at `QUEUE_HEAD_REPLY`/`OPPOSITE_QUEUE_HEAD_REPLY`), the write-only tail last (`lock_state.last`); read replies only by these constants (both reply readers: `try_to_lock` Step 0.1 and `precheck_lock`); the obstacle decision and its exit logs are shared too (`find_opposite_request_ahead`, `lock_obstacle`, `log_lock_obstacle`): change them in one place; spec `lock state pipeline layout` records the pipelines of both modes with and without eager queueing and fails on a broken layout or stale position constants.
|
|
98
|
+
14. Keep operation modules independent (see "Independence of operation modules"; the acquisition core, `AcquireLock` and `LockSeries`, is exempt): never call another operation module's function (public or private) or reference its RBS types from an operation module. When similar logic is needed, duplicate it inside the module (same method name and shape as the original copy, with a `NOTE` that it is duplicated on purpose) or move a pure, data-only helper (key names, id parsing, time conversion) to `Resource` / `Utilities`. Helpers used by one module stay `private`. When changing logic that has copies, update every copy in the same change.
|
|
99
|
+
15. Pipelines/transactions whose replies are read by position (`lock_state[N]`, `result[N]`): document the reply positions next to the commands (`NOTE: (pipeline layout)`). Conditionally sent commands (e.g. `... unless read_mode`) form a tail after all unconditional commands; read their replies only under the same condition (a command appended after a conditional tail gets a mode-dependent position and its reply is read as another command's reply). Insert new commands before the conditional tail and update the positions, the reply reading, the position constants and the layout spec in the same change. `TryToLock` lock state pipeline (`pipeline_lock_state`, shared by the attempt and the pre-check): head `[LOCK_STATE_OBTAINER_REPLY, LOCK_STATE_REDIS_TIME_REPLY, LOCK_STATE_OWN_READ_LOCK_REPLY]`, queueing commands at `LOCK_STATE_QUEUE_REPLIES_AT` (`queue_lock_request` replies, used ones at `QUEUE_HEAD_REPLY`/`OPPOSITE_QUEUE_HEAD_REPLY`), the write-only tail last (`lock_state.last`); read replies only by these constants (both reply readers: `try_to_lock` Step 0.1 and `precheck_lock`); the obstacle decision and its exit logs are shared too (`find_opposite_request_ahead`, `lock_obstacle`, `log_lock_obstacle`, the log-only priority decoding `logged_request_priority`): change them in one place; spec `lock state pipeline layout` records the pipelines of both modes with and without eager queueing and fails on a broken layout or stale position constants.
|
|
94
100
|
16. Lock index (see "Lock index" above): a new command that creates a lock key must be followed (with the enabled lock index: `if lock_index_enabled`, as a tail of the pipeline/transaction) by the index `ZADD <LOCK_INDEX_KEY> GT <Resource.lock_index_time + lifetime + lock_index_grace_period> <lock_name>` in the same pipeline/transaction (after the key command), a new extension by `ZADD XX INCR <ms> <lock_name>`; a new name removal is allowed only in the same `MULTI` that deletes all indexed keys of the name; a new key family with its own lifetime must get a bounded TTL (the enabled lock index) and be covered by the index score and by `ReindexLocks`; never decrease index scores and never derive mutual exclusion from the index; never send lock index commands with the disabled lock index. Cover new index writes in `describe 'lock index'` specs (they enable the lock index explicitly).
|
|
95
101
|
|
|
96
102
|
## Recommendations (proposed, not yet project policy)
|
|
97
103
|
Apply to new or touched code; don't refactor existing code for these unless asked.
|
|
98
104
|
1. Load Lua scripts once (`SCRIPT LOAD` + `EVALSHA`, falling back to `EVAL` on `NOSCRIPT`), as the TODO in `extend_lock_ttl.rb` asks.
|
|
99
105
|
2. Measure lock-iterating operations by the lock index size (distinct lock names x (max(queue ttl, lock ttl) + grace)): most of the visited names are dead; prefer one cheap existence check per name (`EXISTS` with several keys) before reading the details when adding new iterating features.
|
|
100
|
-
3. Treat `lock_series_poc.rb` as experimental: don't build new features on it without asking. Its locks are released only through `release_lock_series` (obtained-by-series locks only, owner-checked, on success and on every failure path).
|
data/.claude/rules/arguments.md
CHANGED
|
@@ -50,12 +50,12 @@ This is a deliberate framework-level decision, **not** something to refactor int
|
|
|
50
50
|
2. New public option: add it as an explicit keyword to **every** affected `Client` method (`x` and `x!`, e.g. `lock` and `lock!`), with the default taken from `config['...']` (add the `setting` + `validate` first) or a literal for per-call-only flags (`raise_errors: false`, `meta: nil`).
|
|
51
51
|
3. Forward it explicitly by name through every layer using shorthand (`new_option:`); never forward via `**kwargs`, `...`, or `method(__method__).parameters` tricks.
|
|
52
52
|
4. Keep defaults only at the public boundary (`Client`); internal functions take required keywords/positionals with no defaults so a missed forward fails loudly.
|
|
53
|
-
5. Internal positional lists: append new params in the established order (subject → domain data → observability tail `logger, instrumenter, instrument, log_sampling_*, log_sample_this, instr_sampling_*, instr_sample_this`) and update every call site and the RBS signature in the same change. Lock key params keep this order: `lock_key,
|
|
53
|
+
5. Internal positional lists: append new params in the established order (subject → domain data → observability tail `logger, instrumenter, instrument, log_sampling_*, log_sample_this, instr_sampling_*, instr_sample_this`) and update every call site and the RBS signature in the same change. Lock key params keep this order: `lock_key, rw_mode, lock_key_queue, read_lock_key_queue, lock_readers_key, read_lock_key, lock_name, acquirer_id, host_id, acquirer_position, priority, ...` (`lock_name` is the lock index member; `acquirer_position` is the priority-shifted queue score, `priority` its band; queueing helpers take `acquirer_position, priority` right after the queue keys; `yield_expire`: `lock_key, lock_readers_key, read_lock_key, rw_mode, acquirer_id, ...`); TTL params: `ttl, queue_ttl, lock_index_enabled, lock_index_grace_period`; compute keys once in `acquire_lock` and pass them down (`read_lock_key` is `nil` for write lock requests). Queueing helpers (`queue_lock_request`, `pipeline_lock_state`) take the lock index params last: `lock_name, queue_ttl, lock_index_enabled, lock_index_grace_period`. Lock-iterating modules take `lock_index_enabled` after their domain data (`scan_size:`/`with_info:` keywords + `lock_index_enabled:`; positional lists: right after the domain data (batch sizes, `dead_ttl`, `by_priority`), before the observability tail; swarm functions: last).
|
|
54
54
|
6. Document each new option with its own `@option`/`@param` YARD line (type, meaning, default source) and add it to the RBS method signature as a named param/keyword.
|
|
55
55
|
7. Keep hot-path code allocation-light: no per-call wrapper objects, no hash merging, no splats to build args, no `tap`/closures purely for argument plumbing; prefer passing existing locals.
|
|
56
56
|
8. Silence length cops locally (`# rubocop:disable Metrics/MethodLength`) rather than shortening signatures; `Metrics/ParameterLists` is disabled project-wide on purpose.
|
|
57
57
|
9. Client-wide modes are not per-call options: `config['lock_index_enabled']` is read inside the `Client` (and `Swarm`) method bodies and forwarded by name to the acquirers (`lock_index_enabled: config['lock_index_enabled']` / positional `config['lock_index_enabled']`), never added as a keyword of `lock`/`locks`/`clear_locks`/...: one call in another mode would break the lock index invariant (all lock acquirements of a client must maintain the index when its operations read it).
|
|
58
|
-
10. `lock`-family options exist in four `Client` methods: `lock`, `lock!`, `lock_series`, `lock_series!` (and `
|
|
58
|
+
10. `lock`-family options exist in four `Client` methods: `lock`, `lock!`, `lock_series`, `lock_series!` (`AcquireLock.acquire_lock` and `LockSeries.lock_series` have their own keyword lists, with the same validations: duplicated on purpose). A new lock option is added to all of them; `lock_series` keeps one value for the whole series (e.g. `rw_mode`, `priority`). Client-wide registries are read from `config` inside these methods and forwarded as internal keywords, never public keywords (`lock_priorities: config['lock_priorities']`, the priority names: `lock` and `lock_series` pass it, `lock!`/`lock_series!` delegate to them).
|
|
59
59
|
|
|
60
60
|
## Recommendations (proposed, not yet project policy)
|
|
61
61
|
1. For new internal functions with many same-typed params (e.g. several Integers/Booleans in a row), consider required keywords instead of positionals to prevent misordering; keep the list flat and explicit.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
---
|
|
2
|
+
paths:
|
|
3
|
+
- "CHANGELOG.md"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# CHANGELOG rules (`CHANGELOG.md`)
|
|
7
|
+
|
|
8
|
+
## Observed conventions
|
|
9
|
+
- Keep a Changelog layout: `## [Unreleased]` on top, released versions as `## [X.Y.Z] - YYYY-MM-DD`; sections `### Added`, `### Changed`, `### Fixed` (`### Removed` once); entries are `- ` bullets, mostly ending with `;`.
|
|
10
|
+
- Markers: `[**Breaking**]` for incompatible changes, `*Brand New Feature*` for a fundamentally new feature (rule 8; older spellings: `Brand new Feature`, `Brand New Extremely Major Feature` of the swarm mode).
|
|
11
|
+
- Code identifiers in backticks: methods `#lock`/`#lock!`, options `priority:`, config keys `config['key']`, errors `RedisQueuedLocks::ArgumentError`, events `redis_queued_locks.<event>`.
|
|
12
|
+
- Anti-pattern: the 1.18.0-1.21.0 and lock priorities sections grew into implementation reports (algorithm steps, Redis commands, round trips, score formulas, full lists of affected events, multi-line sub-bullets) before they were compacted by this rule (`git show 28adc76:CHANGELOG.md` keeps the full text).
|
|
13
|
+
|
|
14
|
+
## Claude rules
|
|
15
|
+
1. Compact entries: one bullet per subject-level change, 1-2 lines (3 at most). A subject-level change is what a gem user gets, loses or has to do: a new method/option/config key/event/log key/error, a changed default/result shape/behavior, a fixed wrong behavior, a breaking change or an upgrade requirement.
|
|
16
|
+
2. No deep details: no algorithm steps, Redis commands, round trips/pipelines/transactions, internal modules/methods/constants, key layouts, score formulas, costs, implementation edge cases, reasoning or alternatives. They live in README (usage), YARD, `.claude/project-overview.md` and the rules; an entry may point to its README section instead.
|
|
17
|
+
3. Internal-only changes (refactoring, specs, CI, internal RBS, `.claude` docs, perf work without a visible effect) get no entry; a visible effect of such work (a typing change for Steep users, a notable speedup) gets one short line.
|
|
18
|
+
4. Sub-bullets only to list separate user-visible items of one feature (its new options, config keys, events), one short line each; one nesting level at most.
|
|
19
|
+
5. Collapse lists that share one change: "lock acquirement logs and instrumentation events carry the Integer `priority`", not the 20 event names (the exact lists live in README `## Logging` / `### Instrumentation Events`).
|
|
20
|
+
6. A fix: the wrong behavior and the scope in one sentence (`` `#zombies_info`: the default `lock_scan_size` was read from the wrong config key ``), not the mechanism of the bug.
|
|
21
|
+
7. Breaking changes and upgrade requirements: the `[**Breaking**]` marker or one sentence of what to do (`upgrade all processes before using positive priorities`), without the reasoning.
|
|
22
|
+
8. A fundamentally new feature gets its own entry marked `*Brand New Feature*`: a new capability of the library as a whole (Swarm Mode, Lock series, Read/Write locks, Lock Index, Lock Request Prioritization) or globally new support (a Redis-compatible server such as Garnet, a new runtime or platform); not an extension of the existing functionality (a new option, method variant, result field, event or log of an existing feature). Format: `- *Brand New Feature*: **<Feature Name>**: <one-sentence summary> (see README "<section>"):` first in `### Added`, the feature's methods/options/config keys/events and upgrade requirements as its sub-bullets (rule 4).
|
|
23
|
+
9. One feature = one entry: a later step of the same feature updates its entry instead of adding a bullet per step. Released sections are not rewritten unless asked.
|
|
24
|
+
|
|
25
|
+
## Example
|
|
26
|
+
- Bad: `` - every lock request drops dead lock requests of its own priority in both lock queues (in the same queueing pipeline: no new commands and round trips); dead lock requests of other priorities that block the lock request ... are dropped ... in one more round trip of the same attempt (up to 5 such round trips per attempt); ``
|
|
27
|
+
- Good: `` - *Brand New Feature*: **Lock Request Prioritization**: `priority:` of `#lock`/`#lock_series` (`-100..100` or a priority name, `0` by default) orders waiting lock requests (strict priority, FIFO inside a priority), see README "Lock Request Prioritization"; ``
|
data/.claude/rules/git.md
CHANGED
|
@@ -28,7 +28,7 @@ Applies to every commit Claude makes in this repository (no `paths:` scope: alwa
|
|
|
28
28
|
|
|
29
29
|
**When to commit**
|
|
30
30
|
7. Never commit on your own. When a step with changes reaches completion, ask the user for permission to commit and wait for the answer:
|
|
31
|
-
- completion = the feature/functionality required by the prompt is fully implemented (code, RBS, specs, README/CHANGELOG, `.claude/project-overview.md` and rules kept in sync) and checks are green: the touched examples and the full suite (`bundle exec rake rspec`), `bundle exec rake rubocop`, `bundle exec rake steep:check` (no `ERROR`/`FATAL` lines); changes without code (docs, `.claude` rules) need no test run;
|
|
31
|
+
- completion = the feature/functionality required by the prompt is fully implemented (code, RBS, specs, README/CHANGELOG, `.claude/project-overview.md` and rules kept in sync) and checks are green: the touched examples and the full suite (`bundle exec rake rspec:parallel` by default, the serial `bundle exec rake rspec` when the change needs a check in one process: `tests.md#1`), `bundle exec rake rubocop`, `bundle exec rake steep:check` (no `ERROR`/`FATAL` lines); changes without code (docs, `.claude` rules) need no test run;
|
|
32
32
|
- pre-commit review: once the step is complete, and only then, run the `rql-code-review` skill on the step's changes (scope `uncommitted`) before the question; never during the implementation or for incomplete/red states. Blocker/Major findings in the step's own changes mean the step is not complete yet: fix them, re-run the affected checks and review the fixes; findings that need the user's decision are reported instead of the question (rule 9);
|
|
33
33
|
- the question contains: the proposed commit subject (following the format rules above), a short recap of what was done in the changes (features/fixes, touched areas, check results), the review verdict with the remaining Minor/Nit findings and the list of files that will be committed;
|
|
34
34
|
- commit only after an explicit "yes" (or with the subject the user corrects); on "no" leave the changes uncommitted (staging also only on request).
|
data/.claude/rules/logic.md
CHANGED
|
@@ -25,38 +25,39 @@ Related rule files (loaded for narrower paths):
|
|
|
25
25
|
- Namespace files (`acquirer.rb`, `swarm.rb`, `logging.rb`, `swarm/swarm_element.rb`) only `require_relative` their children; the root `lib/redis_queued_locks.rb` requires the namespaces in dependency order.
|
|
26
26
|
- YARD doc block on every class, module, constant, attr and method:
|
|
27
27
|
`@param name [Type]`, `@option`, `@return [Type]`, blank `#` line, then `@api public|private`, `@since X.Y.Z`, optional `@version X.Y.Z` (methods: the latest behavior/signature change; classes/modules: the latest interface change, see rule 2).
|
|
28
|
+
- Mixin lines (`include`/`extend`/`prepend`) need no YARD block, only `# @since X.Y.Z` right above each of them: the version that added the mixin to this class/module, not the `@since` of the class/module (`AcquireLock` is `@since 1.0.0`, its `extend RedisQueuedLocks::Utilities` is `@since 1.14.0`).
|
|
28
29
|
- Operations are stateless modules with `class << self` functions; dependencies (`redis_client`, logger, instrumenter, sampling options) are passed as explicit args, never read from globals.
|
|
29
|
-
- Operation modules (except
|
|
30
|
+
- Operation modules (except the lock acquisition core: `AcquireLock`, `LockSeries`) are independent of each other: no cross-calls and no shared RBS types between them; common logic lives in `Resource`/`Utilities`, similar logic is duplicated on purpose (see `acquirer.md`, "Independence of operation modules").
|
|
30
31
|
- Public API results are hashes `{ ok: Boolean, result: Symbol|Hash }` (`Client`, `Acquirer::*`, `Swarm` facade actions); `Client` `!` methods raise `RedisQueuedLocks::*Error`. Internal swarm element APIs return bare scalars/primitives (see `swarm.md`).
|
|
31
32
|
- `Client` methods only fill defaults from `config['...']` and delegate to `Acquirer::*` / `Swarm`.
|
|
32
33
|
- Redis keys come only from `RedisQueuedLocks::Resource.prepare_*` helpers and its `*_PATTERN` / `SWARM_KEY` constants. Key families: write lock `rql:lock:` (`LOCK_PATTERN`), write requests `rql:lock_queue:` (`LOCK_QUEUE_PATTERN`), readers registry `rql:lock_readers:` (`LOCK_READERS_PATTERN`), read requests `rql:lock_read_queue:` (`READ_LOCK_QUEUE_PATTERN`), reader data `rql:lock_reader:<name>:<acq_id>` (`READ_LOCK_PATTERN`); all of them match `KEY_PATTERN` (`rql:lock*`); `Resource.lock_key_family(key)` classifies a key by its prefix (`:lock`, `:lock_queue`, `:lock_readers`, `:read_lock_queue`, `:read_lock`, `nil`).
|
|
33
34
|
- New key families use a distinct **prefix** (`rql:lock_<kind>:<name>`), never a suffix of the lock name: the lock name is an arbitrary string, so `rql:lock_queue:<name>:read` would collide with the queue of a lock named `<name>:read`.
|
|
34
35
|
- Lock index `rql:index:locks` (`Resource::LOCK_INDEX_KEY`), opt-in via the client-wide `config['lock_index_enabled']` (`false` by default: lock-iterating operations scan the keyspace, no lock index commands, queues without TTL): a global ZSET of lock names (score = upper bound of the lifetime of all lock keys, client ms + ttl + `lock_index_grace_period`); not matched by `KEY_PATTERN`. With the enabled lock index every key-creating command is followed by the index `ZADD GT` (queueing pipeline, obtaining `MULTI`), every extension by `ZADD XX INCR`, names are removed only with all their keys (`unlock`, `clear_locks`), lock request queues have TTL; all lock-iterating operations use it (`ZSCAN`), only `reindex_locks` scans the keyspace. Every lock-iterating module keeps both lookups (keyspace and lock index) side by side with the shared processing in one copy (details in `acquirer.md`, "Iteration" and "Lock index").
|
|
35
|
-
- Time: lock expirations that affect safety use Redis time (`PEXPIRE`, or `TIME` via `Resource.redis_time_ms` for reader scores); client `Time.now.to_f` is used only for queue positions, `ts` fields and zombie probes; lock index scores use client ms (`Resource.lock_index_time`) with the `lock_index_grace_period` margin (clock skew tolerance; the index never affects mutual exclusion).
|
|
36
|
-
- Config: `setting('key', default)` + `validate('key') { |val| ... }` in `config.rb`; dotted keys for nested groups (`swarm.flush_zombies.zombie_ttl`); units in a trailing `# NOTE: in milliseconds` comment.
|
|
36
|
+
- Time: lock expirations that affect safety use Redis time (`PEXPIRE`, or `TIME` via `Resource.redis_time_ms` for reader scores); client `Time.now.to_f` is used only for queue positions (shifted by the priority bands: `Resource.calc_initial_acquirer_position(priority)`), `ts` fields and zombie probes; lock index scores use client ms (`Resource.lock_index_time`) with the `lock_index_grace_period` margin (clock skew tolerance; the index never affects mutual exclusion).
|
|
37
|
+
- Config: `setting('key', default)` + `validate('key') { |val| ... }` in `config.rb`; dotted keys for nested groups (`swarm.flush_zombies.zombie_ttl`); units in a trailing `# NOTE: in milliseconds` comment. A validator sees one value: checks across keys live in `Config` itself (the only one: a priority name of `default_lock_priority` must be a key of `lock_priorities`, `prevent_default_priority__unknown_name`): once after a `configure` block (`Client.new { }`, `Client#configure`; any key order, the pair is restored on a failure) and before a direct `[]=` of either key is applied.
|
|
37
38
|
- Errors: subclasses in `errors.rb`, written as `class XError < Error; end` (not `Class.new`) so RBS/Steep can see the superclass. Each error inherits from the global base error of its branch: `Error < ::StandardError` (the main hierarchy), `ArgumentError < ::ArgumentError` (`SwarmArgumentError`), `Exception < ::Exception` (the internal timeouts `LockAcquirementIntermediateTimeoutError`, `TimedLockIntermediateTimeoutError`); inside `module RedisQueuedLocks` the bare `Error`/`ArgumentError`/`Exception` are these bases.
|
|
38
39
|
- Comments use `# NOTE:`, `# TODO:` and `# @type var x: T` / `#: T` for inline type hints.
|
|
39
|
-
- Rubocop is suppressed locally with `# rubocop:disable Metrics/MethodLength` etc. (paired `enable`), mostly for large algorithm methods.
|
|
40
|
+
- Rubocop is suppressed locally with `# rubocop:disable Metrics/MethodLength` etc. (paired `enable`), mostly for large algorithm methods. The inherited armitage config enforces `yield_self` over `then` (`Style/ObjectThen`).
|
|
40
41
|
- Shared mutable state is guarded by `RedisQueuedLocks::Utilities::Lock#synchronize`; Ractor code must not capture non-shareable objects (build a fresh Redis client via `Swarm::RedisClientBuilder`).
|
|
41
42
|
|
|
42
43
|
## Claude rules
|
|
43
44
|
1. Start new files with `# frozen_string_literal: true` and use compact constant paths.
|
|
44
|
-
2. Add a full YARD block to every new public/private method; new code gets `@since <next version>`. `@version <next version>`:
|
|
45
|
-
- a method: when its behavior or its signature changes;
|
|
45
|
+
2. Add a full YARD block to every new public/private method and `# @since <next version>` right above every new `include`/`extend`/`prepend` line; new code gets `@since <next version>`. `@version <next version>`:
|
|
46
|
+
- a method: when its behavior or its signature changes; a typo fix in the method name or in a keyword argument name is a signature change too (callers change): the method gets `@version <next version>` (and the class/module header, see below); a typo fix in a positional parameter name is not a signature change (callers don't change) and bumps nothing;
|
|
46
47
|
- a class/module header: only when its interface changes:
|
|
47
48
|
- a method signature changes: its parameters (their kinds and names); the default values and the return values are behavior;
|
|
48
49
|
- a method is added or removed (public or private, `attr_*` and aliases too);
|
|
49
50
|
- the ancestors change: `extend`/`include`/`prepend` of a module or a changed superclass (e.g. the internal timeout errors in 1.21.0);
|
|
50
|
-
- behavior changes inside the methods, constants, config settings (`setting`/`validate
|
|
51
|
+
- behavior changes inside the methods, constants, config settings (`setting`/`validate` are not methods of `Config`: its header changed in 1.22.0 only for `prevent_default_priority__unknown_name`) and comments bump only the method tags, never the class/module header;
|
|
51
52
|
- the header keeps the version of the latest interface change; no `@version` when the interface never changed since `@since` (the headers of the versions >= 1.17.0 are aligned with this rule).
|
|
52
53
|
3. New operation = new `Acquirer::*` module + thin `Client` method delegating to it (+ `!` variant only if it must raise); details in `acquirer.md`.
|
|
53
54
|
4. Return `{ ok:, result: }` hashes from public API operations (`Client`, `Acquirer::*`, `Swarm` facade actions); raise only in `!` methods or on invalid arguments (`RedisQueuedLocks::ArgumentError`). Internal APIs, and swarm element internals in particular, use bare scalars/primitives (`true`/`false`, String, Symbol, Integer, `nil`, a flat Hash of scalars) without the wrapper.
|
|
54
55
|
5. Never hardcode `rql:` strings; add a `Resource.prepare_*` helper or constant instead.
|
|
55
56
|
6. Use WATCH/MULTI or a Lua constant for any read-modify-write on lock/queue keys; never do check-then-set in Ruby without a transaction.
|
|
56
|
-
7. New config option: `setting` with default + `validate` with a type check and the valid range (TTLs, periods, batch/scan sizes are positive Integers: `val.is_a?(Integer) && val > 0`), both in `config.rb
|
|
57
|
+
7. New config option: `setting` with default + `validate` with a type check and the valid range (TTLs, periods, batch/scan sizes are positive Integers: `val.is_a?(Integer) && val > 0`), both in `config.rb`, plus the key's overload line in `sig/redis_queued_locks/config.rbs` (`Config#[]` is typed per key; `spec/redis_queued_locks/config_spec.rb` fails on a missing/extra line); read it in `Client` as `config['key']`. A per-call numeric option (a `Client` keyword with a `config` default) is validated with the same type and range at the start of the operation module (the `Swarm` facade for the swarm actions), before any Redis command: `RedisQueuedLocks::ArgumentError` (`SwarmArgumentError` in `Swarm`) with the option name and the value (`` "`:scan_size` argument should be a positive Integer, got 0." ``).
|
|
57
58
|
8. Logs and instrumentation events go through visitor modules (see `visitors.md`); event names follow `redis_queued_locks.<snake_case>`.
|
|
58
59
|
9. Keep thread/Ractor safety: guard shared state with `Utilities::Lock`, never share a `RedisClient` across Ractors.
|
|
59
|
-
10. Disable rubocop cops only locally with a matching `rubocop:enable`, and only for Metrics/Layout on large methods (the only exception: the line-level `Lint/InheritException` of the base `RedisQueuedLocks::Exception < ::Exception` in `errors.rb`, see rule 15).
|
|
60
|
+
10. Follow the rubocop rules of the project (`.rubocop.yml` inheriting the armitage config, `.rubocop.rbs.yml` for RBS): `bundle exec rake rubocop` runs two passes (Ruby sources, then RBS) and both must be clean; read the whole output, not only its tail. Never change `.rubocop.yml` to make an offense pass: rewrite the code (e.g. `yield_self`, not `then`); a config change is a separate decision of the user. Disable rubocop cops only locally with a matching `rubocop:enable`, and only for Metrics/Layout on large methods (the only exception: the line-level `Lint/InheritException` of the base `RedisQueuedLocks::Exception < ::Exception` in `errors.rb`, see rule 15); never `# rubocop:disable all` (it hid real offenses until 1.21.1: an unpaired one in `client.rb` switched the linter off till the end of the file).
|
|
60
61
|
11. After any change here, update the mirrored `sig/` file (see `type-checking.md`) and add/adjust a spec (see `tests.md`).
|
|
61
62
|
12. Keep `.claude/project-overview.md` and the affected `.claude/rules/*.md` in sync in the same change (keys, options, results, events, algorithm steps, limitations); read the relevant overview sections before implementing and follow their invariants.
|
|
62
63
|
13. Read/write awareness: every new or changed operation over locks/queues (release, cleanup, info, zombies, series) must handle the read key family too (see the key families above) and keep the result shape for write-only usage unchanged (add keys/branches only when read data exists).
|
|
@@ -69,7 +70,7 @@ Related rule files (loaded for narrower paths):
|
|
|
69
70
|
- new public errors are `@api public` and documented in README/CHANGELOG; the `Exception` branch is `@api private`.
|
|
70
71
|
16. Observability of the logic (logs and instrumentation):
|
|
71
72
|
- every logic that creates, changes or deletes Redis keys has both logs and instrumentation; logic that only reads the lock data (`locked?`, `queued?`, `lock_info`, `queue_info`, `locks`, `queues`, `keys`, `*_info`, zombie queries) has neither;
|
|
72
|
-
- lock acquirement (`AcquireLock`, `
|
|
73
|
+
- lock acquirement (`AcquireLock`, `LockSeries`): each meaningful step/outcome is logged and instrumented through the visitors (`visitors.md`): obtained (classic and reentrant), held and released, not obtained (`lock_obtaining_failed`, also before a raised acquirement error); the per-attempt `try_lock.*` steps are logs only (gated by `log_lock_try`), never instrumented (no `notify` per attempt);
|
|
73
74
|
- lock mutation operations (release, extension, cleanup, reindexing): exactly one log line and one instrumentation event per operation with the time of the whole work of the root operation module called by the public `Client` method (the two-layer pattern of `acquirer.md`: the timer wraps the `fully_*` helper), never of its internal parts; the log line and the event have the same name (`redis_queued_locks.<event>`) and the same data in the same order (the log has no `instrument`), the payload ends with the user's `instrument`; they are written inline in the module (`run_non_critical`, gated by `Logging.should_log?` / `Instrument.should_instrument?`);
|
|
74
75
|
- every such public `Client` method takes `logger:`, `instrumenter:`, `instrument:` and the log/instrumentation sampling keywords (defaults from `config`) and forwards them by name to the operation module;
|
|
75
76
|
- known debt: the swarm elements and the manual swarm actions (`ProbeHosts`, `FlushZombies`: `#probe_hosts`, `#flush_zombies`) are neither logged nor instrumented yet (the flusher Ractor can not get the user's logger/instrumenter).
|
|
@@ -80,8 +81,6 @@ Related rule files (loaded for narrower paths):
|
|
|
80
81
|
|
|
81
82
|
## Recommendations (proposed, not yet project policy)
|
|
82
83
|
Apply to new or touched code; don't refactor existing code for these unless asked.
|
|
83
|
-
1.
|
|
84
|
-
2.
|
|
85
|
-
3.
|
|
86
|
-
4. Include context in raised errors (lock name, acquirer id, timeout) so failures are diagnosable.
|
|
87
|
-
5. Prefer `then` (the modern alias) over `yield_self` in new code.
|
|
84
|
+
1. Document every option fully in YARD.
|
|
85
|
+
2. Spell new identifiers correctly; fix a found typo only in a dedicated change.
|
|
86
|
+
3. Include context in raised errors (lock name, acquirer id, timeout) so failures are diagnosable.
|
data/.claude/rules/swarm.md
CHANGED
|
@@ -17,7 +17,7 @@ The swarm removes **zombie locks**: locks and queue entries left by dead workers
|
|
|
17
17
|
- keyspace (`flush_keyspace_zombies`, the disabled lock index, default): one `SCAN` pass over `KEY_PATTERN` (`COUNT` = `lock_scan_size`), keys classified by `Resource.lock_key_family`: write locks and readers registries in batches of `lock_scan_size` keys, lock queues collected during the pass and processed after it in batches of `queue_scan_size`;
|
|
18
18
|
- lock index (`flush_lock_index_zombies`): drop expired lock names from the lock index (`ZREMRANGEBYSCORE rql:index:locks -inf <now - 1>`) → one `ZSCAN` pass over the lock index (`Resource::LOCK_INDEX_KEY`, lock names; `COUNT` = `lock_scan_size`), lock keys are derived from the names (the keyspace is never scanned): write locks (`prepare_lock_key`) and readers registries (`prepare_lock_readers`) in batches of `lock_scan_size` locks, both lock queues of each name in batches of `queue_scan_size` queues during the same pass (`batch_zombie_requests`);
|
|
19
19
|
- shared batch helpers: pipelined `HMGET acq_id hst_id` + one `DEL` of the zombie locks; pipelined `ZRANGE` + pipelined `ZREM` zombie readers + `DEL` their reader data (`rql:lock_reader:<name>:<acq_id>`); pipelined `ZRANGE` + pipelined `ZREM` of the requests whose host (derived from the request's acquirer id by `Resource.host_identifier_from_acquirer`) is a zombie host, their acquirers are added to the zombie acquirers → `HDEL` zombie hosts.
|
|
20
|
-
`ZombieInfo` mirrors the lock part (`each_zombie_lock` → `each_keyspace_zombie_lock` / `each_lock_index_zombie_lock`: the same passes and batches, names with an expired index score are skipped; nothing is scanned when there are no zombie hosts). It is best-effort and non-transactional. With the enabled lock index locks without a lock index entry (older RQL versions, the disabled lock index, raw writes) are invisible until `client.reindex_locks`.
|
|
20
|
+
`ZombieInfo` mirrors the lock part (`each_zombie_lock` → `each_keyspace_zombie_lock` / `each_lock_index_zombie_lock`: the same passes and batches, names with an expired index score are skipped (`Resource.lock_index_score_expired?`: `+inf` never expires); nothing is scanned when there are no zombie hosts). It is best-effort and non-transactional. With the enabled lock index locks without a lock index entry (older RQL versions, the disabled lock index, raw writes) are invisible until `client.reindex_locks`.
|
|
21
21
|
|
|
22
22
|
## Components
|
|
23
23
|
| Object | File | Kind | Role |
|
|
@@ -42,7 +42,7 @@ Each element has two layers, with different code for each base:
|
|
|
42
42
|
1. **Control unit** (`swarm_element`). Threaded: a `Thread` reading `swarm_element_commands` (`Thread::SizedQueue.new(1)`) and replying on `swarm_element_results` (`SizedQueue(1)`). Isolated: a `Ractor` running `self.swarm_loop(swarm_element_results_port)` that reads `swarm_element_commands_port` and replies on `swarm_element_results_port`, both `Ractor::Port`s (see "Isolated ports" below).
|
|
43
43
|
2. **Main loop**: a `Thread` that does the actual periodic work (`loop { op(redis_client, ...); sleep(period) }`), with a Redis client built inside the thread by `RedisClientBuilder` from `swarm.<element>.redis_config.*`.
|
|
44
44
|
|
|
45
|
-
Phases: **init** `swarm!` (create control unit, loop not started) → **start** `swarm_loop__start` (kill old main loop, spawn new) → **stop** `swarm_loop__stop` kills only the main loop → **kill** `
|
|
45
|
+
Phases: **init** `swarm!` (create control unit, loop not started) → **start** `swarm_loop__start` (kill old main loop, spawn new) → **stop** `swarm_loop__stop` kills only the main loop → **kill** `swarm_element__terminate` (Threaded: kill both threads, close and clear queues, nil them) / `swarm_loop__kill` (Isolated: `:kill` makes the ractor kill and join all its threads, ack, and leave its loop; the host then `join`s the ractor).
|
|
46
46
|
|
|
47
47
|
Command protocol: the same bare-value replies in both bases (see "Internal API: scalars and primitives" below).
|
|
48
48
|
| Command | Reply |
|
|
@@ -114,11 +114,10 @@ Public element API (called by `Swarm`/`Supervisor` only):
|
|
|
114
114
|
16. Swarm specs (in `describe 'swarm'`) deal with real timing: set short periods via config, kill elements with `try_kill!`, wait longer than `liveness_probing_period`, and match states loosely (`eq('running').or(eq('blocking'))`, `'sleep'`/`'run'`).
|
|
115
115
|
|
|
116
116
|
## Known quirks (don't copy; fix only in a dedicated change)
|
|
117
|
-
- Typo in a method name: `swarm_element__termiante` (rename it only in a dedicated change).
|
|
118
117
|
- `swarm_loop__stop` is not used by the swarm itself, only by specs.
|
|
119
118
|
- Killing an element while its main loop is still connecting to Redis makes Ruby's `Socket.tcp` helper threads print `terminated with exception ... IO#write` reports. This is harmless noise from the stdlib.
|
|
120
119
|
- `sleep(0.1)` "give a timespot" waits in `swarm!`/`deswarm!`/`observe!` instead of real readiness signalling.
|
|
121
|
-
- RBS runtime type checking (`rbs/test/setup`, the `typecheck-runtime` CI job) can't run inside Ractors
|
|
120
|
+
- RBS runtime type checking (`rbs/test/setup`, the `typecheck-runtime` CI job) can't run inside Ractors: its hooks read `RBS.logger` and raise `Ractor::IsolationError`. The Ractor-side classes (`FlushZombies`, `SwarmElement::Isolated`, `RedisClientBuilder`) are excluded with `RBS_TEST_SKIP` and the shared helpers they call carry `%a{rbs:test:skip}` (`type-checking.md`); a new function called inside the element Ractor needs the same.
|
|
122
121
|
- Swarm has no logging or instrumentation, and supervisor errors are silently dropped (`TODO: (CHECK)`).
|
|
123
122
|
- Every process with a swarm runs its own zombie flusher: N processes make N lookup passes per flush period (no leader election yet); with the default keyspace lookup each pass is a full keyspace `SCAN` (the reason for the lock index).
|
|
124
123
|
- Zombie write locks are checked (`HMGET`) and deleted (`DEL`) in two round trips without `WATCH`: a lock that expires and is re-acquired by a live acquirer in between can be deleted (kept by the user's decision).
|
data/.claude/rules/tests.md
CHANGED
|
@@ -7,22 +7,22 @@ paths:
|
|
|
7
7
|
# Test rules (`spec/`)
|
|
8
8
|
|
|
9
9
|
## Observed conventions
|
|
10
|
-
- All behavior specs live in one integration file, `spec/redis_queued_locks_spec.rb` (~5.6k lines), under `RSpec.describe RedisQueuedLocks`; groups: `describe 'Lock Series
|
|
10
|
+
- All behavior specs live in one integration file, `spec/redis_queued_locks_spec.rb` (~5.6k lines), under `RSpec.describe RedisQueuedLocks`; the only other spec is the Redis-free `spec/redis_queued_locks/config_spec.rb` (the typed `Config#[]` overload table of `config.rbs` == the `setting` registry); groups: `describe 'Lock Series'`, `describe 'swarm'`, `describe 'read/write locks'`, plus top-level `specify` blocks (all of them in the shared examples), and `describe 'lock index'` (outside the shared examples, see below). The file header says it will be reworked; rspec-retry (5 retries) masks flakiness meanwhile.
|
|
11
11
|
- Known flaky examples (timing-dependent: they pass in isolation, rspec-retry usually hides them; check them with `RSPEC_RETRY_RETRY_COUNT=1` before blaming a change):
|
|
12
|
-
- `all in + notifications`: `sleep(1)` before the lock/queue info checks;
|
|
13
|
-
- `:queued access strategy`: the 4th attempt of `trd2` (~9 s, `retry_delay: 3_000`) races the expiration of the `trd1` lock (obtained at ~7 s with ttl 2 s), and the default `try_to_lock_timeout` (10 s) leaves no later attempt;
|
|
12
|
+
- `all in + notifications`: `sleep(1)` before the lock/queue info checks; it was order-dependent (wrong `locks_info.size` in full runs, e.g. `--seed 6170`, `--seed 28462`): `detailed lock acquirement timeout error` left an unbounded waiter thread that obtained its lock in the next example (now bounded and joined); check new failures without retries before blaming a change;
|
|
13
|
+
- `:queued access strategy` and `:queued access strategy (manually defined)` (the same scenario): the 4th attempt of `trd2` (~9 s, `retry_delay: 3_000`) races the expiration of the `trd1` lock (obtained at ~7 s with ttl 2 s), and the default `try_to_lock_timeout` (10 s) leaves no later attempt;
|
|
14
14
|
- `(auto-swarming!): zombie locks (with hosts and acquirers)` and `manual zombie flushing (with statuses)`: swarm timings (`sleep`-based waits for the probes and the zombie flusher).
|
|
15
|
-
- `.rspec` auto-requires `spec_helper`; `spec_helper.rb` loads SimpleCov first (`setup_simplecov.rb`), then `rspec/retry`, `pry`, the gem.
|
|
15
|
+
- `.rspec` auto-requires `spec_helper`; `spec_helper.rb` loads SimpleCov first (`setup_simplecov.rb`; not started for the `rake rspec:parallel` listing, `RQL_SPEC_LISTING`; a parallel worker, `RQL_SPEC_WORKER`, only stores its result in `coverage/parallel/worker-<n>`: the task collates the report), then `rspec/retry`, `pry`, the gem, then the parallel workers' database offset (`RQL_SPEC_DB_OFFSET`: prepended to `RedisClient::Config::Common#initialize`, which gets the resolved database (`db:` or the database of `url:`), `0` by default).
|
|
16
16
|
- RSpec config: random order (`Kernel.srand config.seed`), `disable_monkey_patching!`, `expect` syntax only, `filter_run_when_matching :focus`, `Thread.abort_on_exception = true`.
|
|
17
17
|
- Tests hit a real Redis (db 0) via `let(:redis) { RedisClient.config(db: 0).new_pool(timeout: 5, size: 50, ...) }` with long timeouts (RBS runtime-check runs are slow); `all in + notifications` also flushes db 1. Check the local Redis dbs before running the suite on a machine with other data.
|
|
18
18
|
- Read/write lock specs use: a monotonic `timeline` hash + `mark` lambda (guarded by a `Mutex`) to assert ordering (`reader_in < other_reader_out`, `writer_in >= reader_out`); an invariant spec with shared reader/writer counters checked inside lock blocks; helper threads that take locks without a block (each thread is a separate acquirer).
|
|
19
19
|
- `before`: `FLUSHDB`, `DEL Resource::SWARM_KEY`, `RedisQueuedLocks.enable_debugger!`; `after`: `DEL SWARM_KEY`, `FLUSHDB`.
|
|
20
|
-
- Examples are written with `specify '<feature>'` (few `it`); grouped with `describe` only for big areas (`'Lock Series
|
|
20
|
+
- Examples are written with `specify '<feature>'` (few `it`); grouped with `describe` only for big areas (`'Lock Series'`, `'swarm'`).
|
|
21
21
|
- Clients are built inline: `RedisQueuedLocks::Client.new(redis) { |config| ... }`.
|
|
22
22
|
- Logger/instrumenter fakes are anonymous classes (`Class.new { def debug(...); def notify(event, payload = {}) }`) collecting calls into arrays.
|
|
23
23
|
- Concurrency is tested with real `Thread.new` and `sleep` to wait for async swarm elements; assertions use `match(...)`, `eq(...).or(eq(...))`, `include`, `raise_error(RedisQueuedLocks::...Error)`.
|
|
24
24
|
- Every example cleans up its own state (`client.clear_locks`, `deswarmize!`, `redis.close`).
|
|
25
|
-
- Command-layout specs observe the sent commands through a separate pool on db 0 built with a recording `redis-client` middleware (a `Module` with `define_method(:call_pipelined) { |commands, config, &block| ...; super(commands, config, &block) }`: the block must be forwarded explicitly), e.g. `lock state pipeline layout` (it reads the position constants of `TryToLock`). Update their expectations deliberately when a pipeline layout changes.
|
|
25
|
+
- Command-layout specs observe the sent commands through a separate pool on db 0 built with a recording `redis-client` middleware (a `Module` with `define_method(:call_pipelined) { |commands, config, &block| ...; super(commands, config, &block) }`: the block must be forwarded explicitly), e.g. `lock state pipeline layout` (it reads the position constants of `TryToLock`). Update their expectations deliberately when a pipeline layout changes. `lock series pipeline layout` does the same for the series (the commands of each lock, the separate queueing pipeline, the read lock extensions first in the transaction); `#lock_series: round trips do not depend on the size of the series` fixes the round trips.
|
|
26
26
|
- Coverage: SimpleCov line + branch, HTML only; `minimum_coverage 100` is a TODO.
|
|
27
27
|
- Both lock lookup modes (`config['lock_index_enabled']`): the behavior examples are `RSpec.shared_examples 'redis_queued_locks'` (the `redis` pool and the FLUSHDB hooks are `RSpec.shared_context 'redis_queued_locks redis'`), included by `RSpec.describe RedisQueuedLocks` twice: `context 'with the disabled lock index (keyspace SCAN)', lock_index_enabled: false` and `context 'with the enabled lock index (lock index ZSCAN)', lock_index_enabled: true`; the `around` hook of `spec_helper` switches the default of `config['lock_index_enabled']` for the example by this metadata (`RedisQueuedLocks::Config.setting`, the library default is restored after the example), so `RedisQueuedLocks::Client.new(redis)` inside the shared examples gets the mode of the context (explicit config values win). One `bundle exec rake rspec` runs the shared examples in both modes (~2x time).
|
|
28
28
|
- Mode-dependent expectations inside the shared examples branch on `client.config['lock_index_enabled']` (e.g. `lock state pipeline layout`: 5 queueing commands, 9 with the lock index; `lock-iterating operations: one pass (keyspace or lock index) with small batches`: `#keys` lists other `rql:lock*` keys only with the keyspace lookup); keys written directly with raw commands (`ZADD` queues, `HSET` locks) are found by the keyspace SCAN and are indexed with `client.reindex_locks if client.config['lock_index_enabled']` before lock-iterating assertions.
|
|
@@ -31,8 +31,8 @@ paths:
|
|
|
31
31
|
## Claude rules
|
|
32
32
|
The suite is pending a total rework (overview "Conventions"): these rules guide new examples, but their violations in `*_spec.rb` are not code review findings until the rework; the review checks only that the changed library behavior is covered and that changed expectations are correct (`.claude/skills/rql-code-review`, §3).
|
|
33
33
|
|
|
34
|
-
1. Run specs with a local Redis available: `bundle exec rake rspec` (
|
|
35
|
-
2. Add new examples to `spec/redis_queued_locks_spec.rb` with `specify '<feature>'`, inside an existing `describe` when one fits; don't create new spec files unless asked (the suite is pending a rework).
|
|
34
|
+
1. Run specs with a local Redis available. Run the suite in parallel processes by default: `bundle exec rake rspec:parallel` (~1 min instead of ~6 of the serial `bundle exec rake rspec`; the RBS runtime checks: the same environment with `rake rspec:parallel` (locally; the CI `typecheck-runtime` job stays serial: parallel workers under the RBS hooks starve the timing-sensitive examples on its 2-core runner); `WORKERS=<n>` flushes db `0..2n-1`, see the `RQL_SPEC_DB_OFFSET` note in `spec_helper.rb`; failures are listed with their rerun command, worker logs in `tmp/rspec_parallel/`). Use the serial run only when something has to be checked in one process: an order-dependent failure or a whole-suite `--seed` (each parallel worker has its own seed and order), state leaking between examples (threads, Redis keys of a finished example), changes of the parallel infrastructure (`spec_helper.rb`, `spec/support/parallel_runner.rb`: check both runs), the serial duration itself. Targeted runs of a few examples (development, the flakiness checks of rule 10) use `bundle exec rspec spec/redis_queued_locks_spec.rb:<line>` / `-e '<group>'` directly. Examples must stay independent of other processes: no server-wide Redis commands (`FLUSHALL`, `CONFIG SET`, `SWAPDB`, `SELECT`), Redis clients only through `RedisClient.config` (the database offset of the parallel workers is applied there).
|
|
35
|
+
2. Add new examples to `spec/redis_queued_locks_spec.rb` with `specify '<feature>'`, inside an existing `describe` when one fits; don't create new spec files unless asked (the suite is pending a rework); Redis-free consistency checks (e.g. the `config_spec.rb` table sync) are the exception: one small file per subject under `spec/redis_queued_locks/`.
|
|
36
36
|
3. Use only `expect` syntax; no `should`, no monkey-patched DSL, no `focus` left behind.
|
|
37
37
|
4. Use the shared `redis` pool and rely on the global `before`/`after` FLUSHDB; never use a different DB or flush in the middle of an example without reason.
|
|
38
38
|
5. Build fakes as anonymous classes implementing the duck-typed interface (`debug`, `notify`, `sampling_happened?`), not with doubles of real loggers.
|
|
@@ -43,7 +43,7 @@ The suite is pending a total rework (overview "Conventions"): these rules guide
|
|
|
43
43
|
10. Check new or changed examples for flakiness without retries: `RSPEC_RETRY_RETRY_COUNT=1 bundle exec rspec spec/redis_queued_locks_spec.rb -e '<group>'`, several runs in a row; also watch for `RSpec::Retry: 2nd try` lines in normal runs.
|
|
44
44
|
11. Threads in specs: `Thread.abort_on_exception = true`, so never raise expected errors inside threads (call non-raising `lock` there and assert results in the main thread); keep references to helper threads until the end of the example (acquirer ids contain `Thread#object_id`, which can be reused after GC).
|
|
45
45
|
12. Timing assertions on TTLs allow the redis time shift error: extendable reentrant locks return the extension minus the time spent in the inner block and `Resource::REDIS_TIMESHIFT_ERROR` (2 ms), so the remaining TTL can slightly exceed the initial one.
|
|
46
|
-
13. New lock features get specs for both modes when relevant (`
|
|
46
|
+
13. New lock features get specs for both modes when relevant (`rw_mode: :write` and `:read`): ordering, mutual exclusion, reentrancy per conflict strategy, `fail_fast`, timeouts/dequeue, release/cleanup/info/zombie paths, and that no `rql:*` keys are left (`client.keys` is empty) after blocks finish.
|
|
47
47
|
|
|
48
48
|
## Recommendations (proposed, not yet project policy)
|
|
49
49
|
Apply to new tests; don't restructure the existing suite unless asked.
|
|
@@ -9,25 +9,24 @@ paths:
|
|
|
9
9
|
# Type-checking rules (`sig/`, Steep, RBS)
|
|
10
10
|
|
|
11
11
|
## Observed conventions
|
|
12
|
-
- `sig/` mirrors `lib/` one file per file (`lib/redis_queued_locks/acquirer/is_locked.rb` → `sig/redis_queued_locks/acquirer/is_locked.rbs`).
|
|
12
|
+
- `sig/` mirrors `lib/` one file per file (`lib/redis_queued_locks/acquirer/is_locked.rb` → `sig/redis_queued_locks/acquirer/is_locked.rbs`).
|
|
13
13
|
- RBS uses fully nested `module RedisQueuedLocks / module Acquirer / ...` blocks (unlike compact Ruby constants).
|
|
14
14
|
- Common aliases at the top: `use RedisQueuedLocks as RQL`, `use RedisClient as RC`; Redis connections typed as `RC::client`.
|
|
15
|
-
- Shared duck types live in `sig/redis_queued_locks.rbs`: `_Loggable`, `_Instrumentable`, `loggerObj`, `instrObj`; samplers as `RQL::Logging::samplerObj` / `RQL::Instrument::samplerObj
|
|
15
|
+
- Shared duck types live in `sig/redis_queued_locks.rbs`: `_Loggable`, `_Instrumentable`, `loggerObj`, `instrObj`; samplers as `RQL::Logging::samplerObj` / `RQL::Instrument::samplerObj`; option literal unions there too: `RQL::readWriteMode` (`:read | :write`), `RQL::accessStrategy` (`:queued | :random`), `RQL::conflictStrategy` (the four conflict strategies), plus `RQL::lockPriority` (`Integer | Symbol`: an Integer priority or a priority name; `Client`, `AcquireLock.acquire_lock`, `LockSeries.lock_series`, `config['default_lock_priority']`; code after the name resolution types `priority` as `Integer`) type `rw_mode`, `access_strategy` and `conflict_strategy` everywhere (`Client`, `AcquireLock` and its mixins, visitors, `ExtendLockTTL`, the config defaults); never `Symbol` for them. The one exception is the entry point that validates the value at runtime (`AcquireLock.acquire_lock` Step 0.1 (RW), `LockSeries.lock_series` Step 0.1 (RW), `ExtendLockTTL.extend_lock_ttl`): its `rw_mode` stays `Symbol` with a `NOTE: (rbs test)` (the RBS runtime checks of the specs must reach the `ArgumentError` of an invalid Symbol), and the validation is a `case rw_mode when :read, :write then nil else raise(...) end` so Steep narrows the variable to `RQL::readWriteMode` for everything below (an `if x != :read && x != :write` guard leaves `(:read | Symbol)`).
|
|
16
16
|
- Result shapes are named record aliases inside the module, e.g. `type extendResult = { ok: bool, result: { extended_locks_count: Integer } | Symbol }`.
|
|
17
|
-
- Each `Acquirer::*` operation module
|
|
17
|
+
- Each `Acquirer::*` operation module declares its own aliases and never references another operation module's aliases (`Locks::lockInfo` / `Locks::readerInfo` mirror `LockInfo::lockInfo` / `LockInfo::readerInfo`); `Client` signatures may reference any of them.
|
|
18
18
|
- Module functions are declared `def self.name: (...) -> T`; long signatures put one param per line with names.
|
|
19
19
|
- Instance variables are declared (`@config_setters: configSetters`) and attr readers typed.
|
|
20
|
-
- Third-party types: `sig/vendor/*.rbs` hand-written stubs (redis_client, active_support, semantic_logger) plus `rbs collection` gems (redis-client, securerandom, timeout, logger, monitor) installed into `.gem_rbs_collection
|
|
20
|
+
- Third-party types: `sig/vendor/*.rbs` hand-written stubs (redis_client, active_support, semantic_logger) plus `rbs collection` gems (redis-client, securerandom, timeout, logger, monitor, json) installed into `.gem_rbs_collection/`; the `gem_rbs_collection` source is pinned to a commit `revision` in `rbs_collection.yaml` (never `main`), so installs are reproducible and `rbs_collection.lock.yaml` stays stable.
|
|
21
21
|
- `Steepfile`: target `lib`, signatures `sig`, ignores `spec`, libraries timeout/securerandom/logger/monitor, diagnostics `Steep::Diagnostic::Ruby.strict`.
|
|
22
|
-
- In Ruby code: `# @type var x: T` and `x = ... #: T` annotations narrow types; `# steep:ignore` silences spots Steep really reports (nil-narrowed `attr_reader` results, pattern-matching destructures, splats). `Config#[]`
|
|
22
|
+
- In Ruby code: `# @type var x: T` and `x = ... #: T` annotations narrow types; `# steep:ignore` silences spots Steep really reports (nil-narrowed `attr_reader` results, pattern-matching destructures, splats). `Config#[]` is typed per key (one overload per `setting` with a string literal key in `config.rbs`), so `config['...']` reads are precisely typed, unknown keys are Steep errors (`Cannot find compatible overloading`) and a dynamic `String` key does not type-check (lookups by variable are not used in `lib/`).
|
|
23
23
|
- Steep 2.1 pitfalls found in this codebase:
|
|
24
24
|
- record types with String keys (`{ 'acq_id' => String, ... }`) crash Steep's subtyping (`RuntimeError`/`Unexpected error` logged as `ERROR`/`FATAL`, while the summary still says "No type error detected"): use `Hash[String, ...]` aliases instead (see `QueueInfo::queueInfo`, `LockInfo::lockInfo`);
|
|
25
25
|
- `x != nil` does not narrow `String?`/`Float?` in `elsif` branches or `&&` chains on locals typed from tuples: use truthiness (`elsif x`) or total conversions (`x.to_f > now`, `ttl.to_i`);
|
|
26
26
|
- `pipelined`/`call` return `untyped`: annotate destructured pipeline results with a tuple (`# @type var lock_state: [String?, Array[String], Float?, Array[[String, Float]]]`);
|
|
27
|
-
- `LockSeriesPoC` has no RBS: new calls/definitions there need `# steep:ignore` (like the existing ones).
|
|
28
27
|
- splatting a `Set` (`call(*set)`) is rejected (`Ruby::ArgumentTypeMismatch`: splat arguments must be arrays), and no honest inline annotation fixes it; at runtime Ruby converts a splatted `Set` with `Set#to_a` on every call.
|
|
29
28
|
- Steep 2 narrows on `==` with literals (`x == :sym` makes `x` the literal type `:sym`) and rejects `# @type var` annotations that widen a narrowed variable; RBS 4's `Kernel#Array` has a `(nil) -> []` overload that `untyped` arguments resolve to, so `Array(rconn.call(...)).first` needs a `#: T` assertion.
|
|
30
|
-
- Runtime checking: CI runs specs with `RUBYOPT
|
|
29
|
+
- Runtime checking: CI runs specs with `RUBYOPT='-rbundler/setup -rrbs/test/setup' RBS_TEST_TARGET='RedisQueuedLocks::*'` (the blocking `typecheck-runtime` job, command in the workflow and README; the job stays serial (`bundle exec rspec`): parallel workers under the RBS hooks starve the timing-sensitive examples on the 2-core runner, locally the same environment works with `rake rspec:parallel`; `bundler/setup` goes first so `rbs`/`prism` come from `Gemfile.lock`, not from newer system gems), so signatures must match real runtime values, not just Steep's view. Not hooked (rbs/test hooks read `RBS.logger` and die with `Ractor::IsolationError` inside the flusher Ractor): the Ractor-side classes via `RBS_TEST_SKIP` (`Swarm::FlushZombies`, `Swarm::SwarmElement::Isolated`, `Swarm::RedisClientBuilder`) and the shared helpers they call via `%a{rbs:test:skip}` in RBS (`Resource.prepare_*`, `lock_key_family`, `lock_index_time`, `lock_name_from_readers`, `host_identifier_from_acquirer`, `calc_zombie_score`, `Utilities.thread_state`); `VoidNotifier.notify` and the samplers' `sampling_happened?` are skipped because `Instrument.valid_interface?` / `valid_sampler?` introspect their `#parameters`. `RBS_TEST_SKIP` entries need the absolute form (`::RedisQueuedLocks::...`): rbs/test compares them with absolute type names without normalizing them. Specs that pass type-invalid values on purpose skip those expectations under `RBS_RUNTIME_TEST` (`spec_helper.rb`); value-invalid cases of the right type (`:reed`) stay.
|
|
31
30
|
|
|
32
31
|
## Claude rules
|
|
33
32
|
1. Every change to a `lib/` file's public or private API (new method, param, return shape, constant, ivar) must be reflected in the mirrored `.rbs` file in the same change.
|
|
@@ -37,15 +36,14 @@ paths:
|
|
|
37
36
|
5. Prefer fixing types or adding `# @type var` / `#: T` annotations over `# steep:ignore`; add `# steep:ignore` only where Steep reports a diagnostic. Strict mode fails the build on `Ruby::RedundantIgnoreComment`, so never add a speculative ignore and remove ones that become redundant. Never shape runtime code around Steep limitations (conversions, copies or allocations made only to satisfy Steep): annotate it, or use a targeted `# steep:ignore` with a `NOTE: (steep)` comment when no honest annotation exists. Keep a conversion only when it is cheaper at runtime and say so in its comment (e.g. a `Set` splatted in a loop: build the array once instead of a `Set#to_a` per call).
|
|
38
37
|
6. Don't loosen `Steep::Diagnostic::Ruby.strict` or add `ignore` entries to the `Steepfile`.
|
|
39
38
|
7. New third-party gem used in `lib/`: add it to `rbs_collection.yaml` (and `sig/manifest.yml` for stdlib) or write a minimal stub in `sig/vendor/`.
|
|
40
|
-
8. Verify with `bundle exec rbs collection install && bundle exec rake steep:check`; for signature/runtime mismatches run the specs under RBS runtime testing (
|
|
41
|
-
9.
|
|
39
|
+
8. Verify with `bundle exec rbs collection install && bundle exec rake steep:check`; for signature/runtime mismatches run the specs under RBS runtime testing (the environment of `.github/workflows/typecheck-runtime.yml`; in parallel processes by default: the same environment with `bundle exec rake rspec:parallel` instead of `bundle exec rspec`, `tests.md#1`).
|
|
40
|
+
9. Keep RBS file and directory names identical to their `lib/` counterparts (`acquirer.rbs`, `acquire_lock/yield_expire/`).
|
|
42
41
|
10. After `steep:check`, also grep its output for `ERROR`/`FATAL` lines: an internal Steep crash skips the file but still reports "No type error detected".
|
|
43
|
-
11. `
|
|
42
|
+
11. Bump the pinned `gem_rbs_collection` revision only deliberately, in its own change: set the new commit in `rbs_collection.yaml`, run `bundle exec rbs collection update`, commit both files together and keep Steep green.
|
|
43
|
+
12. New `setting` in `config.rb` = new overload line `| ("<key>") -> <value type>` in `Config#[]` (`config.rbs`), in the `setting` order; the value type is the validator's type (`Integer`, `Integer?` for nullable keys, `bool`, shared aliases such as `RQL::loggerObj`, `RQL::conflictStrategy`).
|
|
44
|
+
13. Keep the runtime type-check job green: run its command before proposing a commit that touches `sig/` or method signatures; a new method executed inside the flusher Ractor gets `%a{rbs:test:skip}` (with the `(rbs test)` NOTE) or its class goes to `RBS_TEST_SKIP` in the workflow and README; a new spec that passes a type-invalid value wraps that expectation in `unless RBS_RUNTIME_TEST`.
|
|
45
|
+
14. `bundle exec rbs collection install` may rewrite `rbs_collection.lock.yaml` with environment-specific entries; restore it (`git checkout -- rbs_collection.lock.yaml`) unless the change is intended.
|
|
44
46
|
|
|
45
47
|
## Recommendations (proposed, not yet project policy)
|
|
46
|
-
1.
|
|
47
|
-
2.
|
|
48
|
-
3. Pin the `gem_rbs_collection` revision in `rbs_collection.yaml` instead of `main` for reproducible checks.
|
|
49
|
-
4. Rename `sig/redis_queued_locks/acquier.rbs` to `acquirer.rbs` in a dedicated change.
|
|
50
|
-
5. Replace remaining `untyped` in signatures with precise unions or interfaces where the value set is known (e.g. strategy symbols as `:queued | :random`).
|
|
51
|
-
6. Type strategy and mode options as literal unions (`conflict_strategy: :wait_for_lock | :work_through | :extendable_work_through | :dead_locking`) so Steep catches invalid values.
|
|
48
|
+
1. Type `Config#[]=` per key like `Config#[]` (an overload table of key literal => value type) so Steep checks assignments in user code too.
|
|
49
|
+
2. Replace remaining `untyped` in signatures with precise unions or interfaces where the value set is known (`client.rbs` results and `meta`, the `void_logger.rbs` and `redis_client_builder.rbs` args).
|
data/.claude/rules/visitors.md
CHANGED
|
@@ -19,10 +19,13 @@ The project calls these modules "visitors", but they are not GoF Visitor (no `ac
|
|
|
19
19
|
|---|---|---|
|
|
20
20
|
| `AcquireLock::LogVisitor` | `acquire_lock/log_visitor.rb` | 7: `start_lock_obtaining`, `start_try_to_lock_cycle`, `dead_score_reached__reset_acquirer_position`, `extendable_reentrant_lock_obtained`, `reentrant_lock_obtained`, `lock_obtained`, `lock_obtaining_failed` |
|
|
21
21
|
| `AcquireLock::InstrVisitor` | `acquire_lock/instr_visitor.rb` | 6: `extendable_reentrant_lock_obtained`, `reentrant_lock_obtained`, `lock_obtained`, `lock_obtaining_failed`, `reentrant_lock_hold_completes`, `lock_hold_and_release` |
|
|
22
|
-
| `AcquireLock::TryToLock::LogVisitor` | `acquire_lock/try_to_lock/log_visitor.rb` |
|
|
22
|
+
| `AcquireLock::TryToLock::LogVisitor` | `acquire_lock/try_to_lock/log_visitor.rb` | 19 step-level events (`start`, `rconn_fetched`, `acq_added_to_queue`, `dead_request_removed`, `exit__no_first`, `exit__read_lock_still_obtained`, `obtain__free_to_acquire`, ...); every lock event logs `rw_mode` |
|
|
23
23
|
| `AcquireLock::YieldExpire::LogVisitor` | `acquire_lock/yield_expire/log_visitor.rb` | `expire_lock`, `decrease_lock` |
|
|
24
24
|
| `AcquireLock::DequeueFromLockQueue::LogVisitor` | `acquire_lock/dequeue_from_lock_queue/log_visitor.rb` | `dequeue_from_lock_queue` |
|
|
25
|
-
| `
|
|
25
|
+
| `LockSeries::LogVisitor` | `lock_series/log_visitor.rb` | 11: `start_lock_series_obtaining`, `start_try_to_lock_series_cycle`, `dead_score_reached__reset_series_position`, `lock_series_intermediate_lock_obtained`, `lock_series_lock_obtained`, `lock_series_lock_obtaining_failed`, `lock_series_obtained`, `lock_series_obtaining_failed`, `dequeue_from_lock_series_queues`, `expire_lock_series`, `decrease_lock_series`; `rw_mode` (the mode of the series) and `priority` in all of them |
|
|
26
|
+
| `LockSeries::InstrVisitor` | `lock_series/instr_visitor.rb` | 6: `lock_series_intermediate_lock_obtained`, `lock_series_lock_obtained`, `lock_series_lock_obtaining_failed`, `lock_series_obtained`, `lock_series_hold_and_release`, `lock_series_obtaining_failed`; `rw_mode` and `priority` in all of them |
|
|
27
|
+
|
|
28
|
+
`TryToLockSeries` writes the try-lock logs of each lock through `TryToLock::LogVisitor` (the `LogVisitor` constant there: `TryToLock` is included; the per-lock loops are gated by `log_sampled && log_lock_try`): no own try-lock events.
|
|
26
29
|
|
|
27
30
|
Not covered by visitors (lock mutation operations, `logic.md#16`): `ReleaseLock`, `ReleaseReadLock`, `ReleaseAllLocks`, `ReleaseLocksOf`, `ExtendLockTTL`, `ClearDeadRequests`, `ReindexLocks` write one log line (`logger.debug`) and one event (`instrumenter.notify`) inline inside `run_non_critical` after the whole operation: `explicit_lock_release`, `explicit_read_lock_release`, `explicit_all_locks_release`, `release_locks_of`, `extend_lock_ttl`, `clear_dead_requests`, `reindex_locks` (the log line has the event name and the payload data in the same order, without `instrument`; the payload ends with `instrument`). `lock_obtaining_failed` is reported by `acquire_lock` in Step 3.b (the failure result) and in the method-level `rescue` of the raised acquirement errors (Step 3.c: only while the lock is not acquired; errors of the block of code are not reported).
|
|
28
31
|
|
|
@@ -31,20 +34,20 @@ Not covered by visitors (lock mutation operations, `logic.md#16`): `ReleaseLock`
|
|
|
31
34
|
- **Shape**: `module <Component>::LogVisitor` / `::InstrVisitor`, `# @api private`, all methods inside `class << self`, full YARD block per method; return `void`.
|
|
32
35
|
- **Method name = event name**: snake_case; double underscore separates phase and detail (`exit__queue_ttl_reached`, `reentrant_lock__work_through`). The LogVisitor and InstrVisitor use the same method name for the same event.
|
|
33
36
|
- **Parameter order**: `(port, sampled_flag, [extra gate], lock_key, ...event data..., [instrument])`:
|
|
34
|
-
- LogVisitor: `(logger, log_sampled, ...)`; `TryToLock::LogVisitor` adds `log_lock_try` as the third arg (fine-grained step logs are opt-in through `config['log_lock_try']`). Lock lifecycle events (AcquireLock, TryToLock, YieldExpire, DequeueFromLockQueue,
|
|
35
|
-
- InstrVisitor: `(instrumenter, instr_sampled, lock_key, rw_mode, ttl, acq_id, hst_id, ts, acq_time, [hold_time], instrument)`; the payload carries `rw_mode:` (requested mode); the user's `instrument` value is always last.
|
|
36
|
-
- `rw_mode` in AcquireLock/TryToLock/DequeueFromLockQueue events is the **requested** mode; in YieldExpire events it is the **held** mode (the lock that is released/decreased); in
|
|
37
|
+
- LogVisitor: `(logger, log_sampled, ...)`; `TryToLock::LogVisitor` adds `log_lock_try` as the third arg (fine-grained step logs are opt-in through `config['log_lock_try']`). Lock lifecycle events (AcquireLock, TryToLock, YieldExpire, DequeueFromLockQueue, LockSeries) take `rw_mode` right after `access_strategy`, before event-specific data, and log it as `rw_mode => '<mode>'` right after `acs_strat`; the lock acquirement logs of `AcquireLock::LogVisitor` (all of them), `LockSeries::LogVisitor` (the priority of the series) and `TryToLock::LogVisitor.acq_added_to_queue` take `priority` right after `rw_mode` and log `priority => N` right after it (the release logs of `YieldExpire` and `DequeueFromLockQueue` don't); try-lock logs that name another request log its decoded priority right after its acquirer (`first_priority_in_queue` after `first_acq_id_in_queue`, `ahead_priority` after `ahead_acq_id`, `dead_priority` after `dead_acq_id`; `Integer?` params decoded at the call site only when `log_sampled && log_lock_try`, by one helper: `TryToLock#logged_request_priority`, see rule 12).
|
|
38
|
+
- InstrVisitor: `(instrumenter, instr_sampled, lock_key, rw_mode, ttl, acq_id, hst_id, ts, acq_time, [hold_time], instrument)`; the payload carries `rw_mode:` (requested mode); the user's `instrument` value is always last. All lock acquirement events (`lock_obtained`, `lock_hold_and_release`, `lock_obtaining_failed`, the reentrant ones: the requested priority, they never queue; the lock series ones: the priority of the series) take `priority` (the lock request priority: always the Integer, a priority name is resolved before) right after `rw_mode` (payload `priority:` after `rw_mode:`).
|
|
39
|
+
- `rw_mode` in AcquireLock/TryToLock/DequeueFromLockQueue events is the **requested** mode; in YieldExpire events it is the **held** mode (the lock that is released/decreased); in LockSeries events it is the mode of the series (one for all locks of the series). LockSeries instrumentation takes `rw_mode` right after `lock_keys`: the series events have `lock_keys` instead of `lock_key`, the events of a lock of the series (`lock_series_*lock_*`) have both (`lock_key, lock_keys`). The per-lock series events are not per-attempt events (`logic.md#16`): `lock_series_intermediate_lock_obtained` is reported once per lock by `lock_series`, out of the WATCH block of the attempt.
|
|
37
40
|
- **Guard first**: `return unless log_sampled` (`&& log_lock_try` for try-lock steps) / `return unless instr_sampled`. The sampling decision is computed **once per operation** in the caller via `Logging.should_log?` / `Instrument.should_instrument?` and passed down as a boolean.
|
|
38
|
-
- **Log format**: one `logger.debug { ... }` block (lazy string), message = `"[redis_queued_locks.<event>] "` (`[redis_queued_locks.try_lock.<event>]` for TryToLock steps) followed by `key => value` pairs; string values in single quotes (`lock_key => '...'`), numbers bare; abbreviated keys `acq_id`, `hst_id`, `acs_strat`. Built with `\` line continuations.
|
|
41
|
+
- **Log format**: one `logger.debug { ... }` block (lazy string), message = `"[redis_queued_locks.<event>] "` (`[redis_queued_locks.try_lock.<event>]` for TryToLock steps) followed by `key => value` pairs; string values in single quotes (`lock_key => '...'`), numbers bare (a missing number of an `Integer?` param is logged as `nil` with `#inspect`, never as an empty value: `first_priority_in_queue => nil`); abbreviated keys `acq_id`, `hst_id`, `acs_strat`. Built with `\` line continuations.
|
|
39
42
|
- **Instrumentation format**: `instrumenter.notify('redis_queued_locks.<event>', { lock_key:, ttl:, acq_id:, hst_id:, ts:, acq_time:, instrument: })` using shorthand hash syntax and Symbol keys.
|
|
40
43
|
- **Never raise**: every emit ends with `rescue nil` (modifier), so a broken logger/instrumenter can't affect lock correctness.
|
|
41
44
|
- **Call sites**: plain module calls with positional args, usually grouped on 2-3 lines (`LogVisitor.lock_obtained(logger, log_sampled, lock_key, ...)`); data comes from local vars or the `result` hash of the step.
|
|
42
|
-
- **Typing**: each visitor
|
|
45
|
+
- **Typing**: each visitor has an RBS file with `def self.<event>: (RQL::loggerObj logger, bool log_sampled, ...) -> void` / `RQL::instrObj instrumenter`.
|
|
43
46
|
- **User docs**: README documents every log event with its logged keys (`## Logging`: the default, `log_lock_try`, lock series and operation logs lists, mirrored in the `config['logger']` / `config['log_lock_try']` comments of `### Logging Configuration` and `### Configuration`) and every instrumentation event with its payload keys, types and semantics (`### Instrumentation Events`).
|
|
44
47
|
|
|
45
48
|
## Claude rules
|
|
46
49
|
**When to use**
|
|
47
|
-
1. Any new log line or instrumentation event inside `AcquireLock` (and
|
|
50
|
+
1. Any new log line or instrumentation event inside `AcquireLock`, `LockSeries` (and their mixins) must go through a visitor method; never call `logger.*` or `instrumenter.notify` directly in algorithm code.
|
|
48
51
|
2. Add a LogVisitor event for each new meaningful step or branch of the algorithm (start, decision, exit, success). Add an InstrVisitor event for the outcomes users would want to measure (obtained, not obtained, held/released, completed); not for internal per-attempt steps (`logic.md#16`).
|
|
49
52
|
3. Fine-grained per-attempt logging belongs in `TryToLock::LogVisitor` and must be gated by `log_lock_try`.
|
|
50
53
|
4. For a new algorithm component (new mixin under `acquire_lock/`), create its own `<component>/log_visitor.rb` (and `instr_visitor.rb` if needed) and `require_relative` it from the component.
|
|
@@ -65,4 +68,3 @@ Not covered by visitors (lock mutation operations, `logic.md#16`): `ReleaseLock`
|
|
|
65
68
|
Apply to new or touched code; don't refactor existing code for these unless asked.
|
|
66
69
|
1. Use one error-swallowing style consistently (visitors use modifier `rescue nil`, mutation modules use `run_non_critical`). Both silently hide bugs such as a `NoMethodError` inside the log block, so consider reporting swallowed errors when the debugger is enabled.
|
|
67
70
|
2. Keep log and instrumentation event names in one registry constant (e.g. `RedisQueuedLocks::Instrument::EVENTS`) so README, specs and RBS can be checked against it.
|
|
68
|
-
3. Fix the RBS path mismatch: `sig/.../acquire_lock/yield_with_expire/log_visitor.rbs` mirrors `lib/.../acquire_lock/yield_expire/log_visitor.rb`; add RBS for `lock_series_poc` visitors when the PoC stabilizes.
|