redis_queued_locks 1.16.2 → 1.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/project-overview.md +281 -0
  3. data/.claude/rules/acquirer.md +81 -0
  4. data/.claude/rules/arguments.md +60 -0
  5. data/.claude/rules/git.md +46 -0
  6. data/.claude/rules/logic.md +63 -0
  7. data/.claude/rules/swarm.md +120 -0
  8. data/.claude/rules/tests.md +48 -0
  9. data/.claude/rules/type-checking.md +50 -0
  10. data/.claude/rules/visitors.md +69 -0
  11. data/.rubocop.rbs.yml +31 -0
  12. data/.rubocop.yml +3 -3
  13. data/.ruby-version +1 -1
  14. data/CHANGELOG.md +80 -0
  15. data/CLAUDE.md +36 -0
  16. data/README.md +367 -82
  17. data/Rakefile +31 -2
  18. data/github_ci/.keep +0 -0
  19. data/lib/redis_queued_locks/acquirer/acquire_lock/delay_execution.rb +2 -2
  20. data/lib/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue/log_visitor.rb +6 -3
  21. data/lib/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rb +9 -7
  22. data/lib/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rb +20 -10
  23. data/lib/redis_queued_locks/acquirer/acquire_lock/log_visitor.rb +34 -16
  24. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rb +240 -28
  25. data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +398 -132
  26. data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire/log_visitor.rb +12 -6
  27. data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire.rb +33 -9
  28. data/lib/redis_queued_locks/acquirer/acquire_lock.rb +73 -24
  29. data/lib/redis_queued_locks/acquirer/clear_dead_requests.rb +9 -1
  30. data/lib/redis_queued_locks/acquirer/extend_lock_ttl.rb +182 -3
  31. data/lib/redis_queued_locks/acquirer/is_locked.rb +19 -2
  32. data/lib/redis_queued_locks/acquirer/is_queued.rb +6 -2
  33. data/lib/redis_queued_locks/acquirer/lock_info.rb +96 -6
  34. data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +8 -4
  35. data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +18 -7
  36. data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +126 -29
  37. data/lib/redis_queued_locks/acquirer/locks.rb +103 -6
  38. data/lib/redis_queued_locks/acquirer/queue_info.rb +38 -9
  39. data/lib/redis_queued_locks/acquirer/queues.rb +28 -4
  40. data/lib/redis_queued_locks/acquirer/release_all_locks.rb +35 -1
  41. data/lib/redis_queued_locks/acquirer/release_lock.rb +40 -9
  42. data/lib/redis_queued_locks/acquirer/release_locks_of.rb +43 -3
  43. data/lib/redis_queued_locks/acquirer/release_read_lock.rb +143 -0
  44. data/lib/redis_queued_locks/acquirer.rb +1 -0
  45. data/lib/redis_queued_locks/client.rb +274 -158
  46. data/lib/redis_queued_locks/config/dsl.rb +4 -10
  47. data/lib/redis_queued_locks/resource.rb +117 -9
  48. data/lib/redis_queued_locks/swarm/flush_zombies.rb +46 -6
  49. data/lib/redis_queued_locks/swarm/supervisor.rb +1 -1
  50. data/lib/redis_queued_locks/swarm/swarm_element/isolated.rb +166 -44
  51. data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +45 -47
  52. data/lib/redis_queued_locks/swarm/zombie_info.rb +42 -0
  53. data/lib/redis_queued_locks/version.rb +2 -2
  54. data/rbs_collection.lock.yaml +10 -2
  55. data/rbs_collection.yaml +0 -2
  56. data/redis_queued_locks.gemspec +1 -1
  57. data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue/log_visitor.rbs +2 -1
  58. data/sig/redis_queued_locks/acquirer/acquire_lock/dequeue_from_lock_queue.rbs +0 -1
  59. data/sig/redis_queued_locks/acquirer/acquire_lock/instr_visitor.rbs +5 -0
  60. data/sig/redis_queued_locks/acquirer/acquire_lock/log_visitor.rbs +12 -6
  61. data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock/log_visitor.rbs +74 -7
  62. data/sig/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rbs +4 -2
  63. data/sig/redis_queued_locks/acquirer/acquire_lock/yield_expire.rbs +3 -0
  64. data/sig/redis_queued_locks/acquirer/acquire_lock/yield_with_expire/log_visitor.rbs +4 -2
  65. data/sig/redis_queued_locks/acquirer/extend_lock_ttl.rbs +19 -1
  66. data/sig/redis_queued_locks/acquirer/lock_info.rbs +9 -1
  67. data/sig/redis_queued_locks/acquirer/locks.rbs +10 -1
  68. data/sig/redis_queued_locks/acquirer/queue_info.rbs +2 -4
  69. data/sig/redis_queued_locks/acquirer/queues.rbs +1 -1
  70. data/sig/redis_queued_locks/acquirer/release_lock.rbs +8 -1
  71. data/sig/redis_queued_locks/acquirer/release_read_lock.rbs +44 -0
  72. data/sig/redis_queued_locks/client.rbs +22 -2
  73. data/sig/redis_queued_locks/resource.rbs +9 -2
  74. data/sig/redis_queued_locks/swarm/flush_zombies.rbs +1 -1
  75. data/sig/redis_queued_locks/swarm/swarm_element/isolated.rbs +13 -5
  76. data/sig/redis_queued_locks/swarm/swarm_element/threaded.rbs +6 -6
  77. data/sig/redis_queued_locks/swarm/zombie_info.rbs +5 -0
  78. metadata +17 -5
  79. data/github_ci/ruby3.3.gemfile +0 -17
  80. data/github_ci/ruby3.3.gemfile.lock +0 -216
@@ -0,0 +1,120 @@
1
+ ---
2
+ paths:
3
+ - "lib/redis_queued_locks/swarm.rb"
4
+ - "lib/redis_queued_locks/swarm/**/*.rb"
5
+ - "sig/redis_queued_locks/swarm.rbs"
6
+ - "sig/redis_queued_locks/swarm/**/*.rbs"
7
+ ---
8
+
9
+ # Swarm rules (`lib/redis_queued_locks/swarm/**`)
10
+
11
+ ## Purpose
12
+ The swarm removes **zombie locks**: locks and queue entries left by dead workers.
13
+ - **Host**: a `process/thread/ractor/identity` worker, with the id `rql:hst:<pid>/<thread_id>/<ractor_id>/<identity>` (`Resource.host_identifier`). Fibers are not included because `ObjectSpace` can't see Fibers or Threads once a Ractor exists. Hosts are enumerated with `Thread.list` (`Resource.possible_host_identifiers`).
14
+ - **Liveness**: `ProbeHosts` runs `HSET rql:swarm:hsts <host_id> <Time.now.to_f>` for every possible host of the client's ractor (`Resource::SWARM_KEY`).
15
+ - **Zombie**: a host whose last probe score is `< Resource.calc_zombie_score(zombie_ttl / 1_000.0)` (`now - ttl`). `zombie_ttl` is in milliseconds. Zombie locks are `rql:lock:*` whose `hst_id` field is a zombie host, plus zombie read locks: readers registry members (`rql:lock_readers:*`) whose host (derived from the acquirer id by `Resource.host_identifier_from_acquirer`, a pure function usable inside the Ractor) is a zombie host; a zombie read lock is reported as its registry key. Zombie acquirers are their `acq_id`s.
16
+ - **Flush** (`FlushZombies.flush_zombies`) runs these steps: `HGETALL` hosts → zombie hosts (return early if none) → `SCAN MATCH rql:lock:*` + `HMGET acq_id hst_id` → `DEL` zombie locks → `SCAN MATCH rql:lock_readers:*` + `ZRANGE` → `ZREM` zombie readers + `DEL` their reader data (`rql:lock_reader:<name>:<acq_id>`) → `SCAN MATCH rql:lock_queue:*` and `rql:lock_read_queue:*` + `ZREM` zombie acquirers → `HDEL` zombie hosts. `ZombieInfo` mirrors the read-lock part (`each_zombie_read_lock`). It is best-effort and non-transactional, with full keyspace scans (`TODO: indexing`).
17
+
18
+ ## Components
19
+ | Object | File | Kind | Role |
20
+ |---|---|---|---|
21
+ | `Swarm` | `swarm.rb` | facade (one per `Client`, `client.swarm`) | owns the supervisor and elements. Public API: `swarm!`/`deswarm!`, `swarm_status`, `swarm_info`, `probe_hosts`, `flush_zombies`, `zombie_*`/`zombies_info`. Guards everything with its own `sync` |
22
+ | `Swarm::Supervisor` | `swarm/supervisor.rb` | plain `Thread` (`visor`) | every `swarm.supervisor.liveness_probing_period` seconds it runs the `observable` block, which calls `reswarm_if_dead!` on every element |
23
+ | `SwarmElement::Threaded` | `swarm/swarm_element/threaded.rb` | abstract base | control `Thread` plus a main-loop `Thread` |
24
+ | `SwarmElement::Isolated` | `swarm/swarm_element/isolated.rb` | abstract base | control `Ractor` plus a main-loop `Thread` inside it, driven through a per-element pair of `Ractor::Port`s |
25
+ | `ProbeHosts` | `swarm/probe_hosts.rb` | `< Threaded` | periodic host liveness probes |
26
+ | `FlushZombies` | `swarm/flush_zombies.rb` | `< Isolated` | periodic zombie flushing |
27
+ | `Acquirers`, `ZombieInfo` | `swarm/acquirers.rb`, `swarm/zombie_info.rb` | stateless `class << self` modules | read-only queries (`HGETALL` swarm hash, lock `SCAN`s) |
28
+ | `RedisClientBuilder` | `swarm/redis_client_builder.rb` | stateless module | `build(pooled:, sentinel:, config:, pool_config:)` returns a fresh `RedisClient` or `RedisClient::Pooled` |
29
+
30
+ `Client` methods `swarmize!`, `deswarmize!`, `swarm_status`/`swarm_state`, `swarm_info`, `probe_hosts`, `flush_zombies`, `zombie_locks`, `zombie_acquirers`, `zombie_hosts`, `zombies_info`/`zombies` only fill defaults from `config['swarm.*']` and delegate to `Swarm`. `Client#initialize` calls `swarm.swarm!` when `swarm.auto_swarm` is set.
31
+
32
+ ## Why Threaded vs Isolated
33
+ - `ProbeHosts` **must** be Threaded. Host ids come from `Thread.list` and `Ractor.current` of the client's ractor, so a probe from another Ractor would announce the wrong hosts. It may read `rql_client` (same ractor).
34
+ - `FlushZombies` is Isolated. The heavy `SCAN`/`DEL` work runs in its own Ractor with its own Redis connection, isolated from the app's ractor and objects. Everything it needs is passed into `Ractor.new(...)` as copied plain values (`config.slice('swarm.flush_zombies.redis_config')`, which `dup`s values, plus Integers). It never touches `rql_client` inside the ractor.
35
+
36
+ ## Element lifecycle (both bases)
37
+ Each element has two layers, with different code for each base:
38
+ 1. **Control unit** (`swarm_element`). Threaded: a `Thread` reading `swarm_element_commands` (`Thread::SizedQueue.new(1)`) and replying on `swarm_element_results` (`SizedQueue(1)`). Isolated: a `Ractor` running `self.swarm_loop(swarm_element_results_port)` that reads `swarm_element_commands_port` and replies on `swarm_element_results_port`, both `Ractor::Port`s (see "Isolated ports" below).
39
+ 2. **Main loop**: a `Thread` that does the actual periodic work (`loop { op(redis_client, ...); sleep(period) }`), with a Redis client built inside the thread by `RedisClientBuilder` from `swarm.<element>.redis_config.*`.
40
+
41
+ Phases: **init** `swarm!` (create control unit, loop not started) → **start** `swarm_loop__start` (kill old main loop, spawn new) → **stop** `swarm_loop__stop` kills only the main loop → **kill** `swarm_element__termiante` (Threaded: kill both threads, close and clear queues, nil them) / `swarm_loop__kill` (Isolated: `:kill` makes the ractor kill and join all its threads, ack, and leave its loop; the host then `join`s the ractor).
42
+
43
+ Command protocol: the same bare-value replies in both bases (see "Internal API: scalars and primitives" below).
44
+ | Command | Reply |
45
+ |---|---|
46
+ | `:status` | `{ alive: Boolean, state: String }` |
47
+ | `:is_active` | `true` / `false` |
48
+ | `:start` / `:stop` | `true` (ack) |
49
+ | `:kill` | Isolated only: `true` (ack), then the ractor finishes. Threaded is terminated from outside. |
50
+
51
+ Every request/reply pair is wrapped in `sync.synchronize` so concurrent callers can't interleave on the channel, and each reply belongs to the command just sent. Both bases send every command through one helper, `swarm_loop__send_command(command)`, which returns the reply, or `nil` when the element is dead (or, in Threaded, terminating). Abstract hooks (`spawn_main_loop!`, `spawn_swarm_element!`) raise `NotImplementedError` in the bases.
52
+
53
+ ## Internal API: scalars and primitives
54
+ `{ ok:, result: }` is the **public API** contract only: `Client` methods, `Acquirer::*` operations, and the `Swarm` facade's public methods (`swarm!`/`deswarm!`, plus the stateless operations behind `client.probe_hosts` / `client.flush_zombies`). Status reports (`swarm_status`, element `#status`, `Supervisor#status`) are plain nested Hashes with no wrapper.
55
+
56
+ Everything inside swarm elements is internal API and uses bare scalars and primitives, with no `{ ok:, result: }` wrapper. That covers control-unit commands and replies, the Isolated handshake, port and queue messages, `swarm_loop__*` helpers, `swarmed__*` predicates and main-loop plumbing.
57
+ - **Commands**: Symbols (`:status`, `:is_active`, `:start`, `:stop`, `:kill`).
58
+ - **Replies**: the value itself. Use `true`/`false` for questions and `true` as the ack for actions. Use a String/Symbol/Integer/Float for a single datum, and a flat Hash of scalars (`{ alive:, state: }`) only when a reply carries several related values. Don't nest.
59
+ - **Handshake**: the commands port itself, sent as the first message.
60
+ - **"No answer" / dead element**: `nil`. On the results port, a Symbol (`:exited`/`:aborted`) is reserved for the `Ractor#monitor` death notice, so never reply with a Symbol there. If a datum is a Symbol, send it as a String.
61
+ - **Errors**: don't wrap them in replies. A failing main loop just dies, and a dead control unit is detected through liveness, the monitor notice or `nil`.
62
+
63
+ ## Isolated ports (`Ractor::Port`)
64
+ Only the Ractor that created a port can `receive` from it; any Ractor can send to it. So each Isolated element owns two ports:
65
+ - **`swarm_element_results_port`**: created in the **main Ractor** by `swarm!`. The Swarm and its Supervisor live there, and so do all callers of the element API. It receives command replies and also the Ractor's termination notice: `swarm!` calls `swarm_element.monitor(results)`, which sends `:exited`/`:aborted`, immediately if the Ractor has already finished.
66
+ - **`swarm_element_commands_port`**: created **inside the element Ractor** by `.swarm_loop`. It's handed to the main Ractor as the first message on the results port (the handshake: the port itself; any other first message means the Ractor died during startup), which `swarm!` waits for.
67
+
68
+ Ports belong to the element instance, so any number of Isolated elements work side by side without sharing channels. A new `swarm!` creates a fresh pair, and stale messages left in an old results port are dropped with it.
69
+
70
+ Failure handling in `swarm_loop__send_command`:
71
+ - A non-Hash reply (`:exited`/`:aborted`) means the Ractor died, so the request returns `nil`.
72
+ - Sending to a finished Ractor's port raises `Ractor::ClosedError`, which is also rescued to `nil`.
73
+ - No request can block forever, because the monitor notice always arrives.
74
+
75
+ Thread hygiene inside the Ractor: a Ractor stays `running` until **all** its threads have finished. Killed but un-joined threads count, and so do helper threads that `Socket.tcp` leaves behind when the loop is killed mid-connect. `.terminate_thread` therefore kills and joins. `:kill` terminates every thread of the Ractor (`Thread.list - [Thread.current]`) before acking, and `swarm_loop__kill` joins the Ractor, so the status is `terminated` as soon as `try_kill!` returns.
76
+
77
+ State predicates (private, same names in both bases): `idle?` (no control unit), `swarmed?`, `swarmed__alive?`, `swarmed__dead?`, `swarmed__running?` (alive and main loop active), `swarmed__stopped?` (alive, loop inactive). Threaded also has `terminating?` (queues nil/closed). In Isolated, `swarmed__running?`/`swarmed__stopped?` return `false` when the request returned `nil` (the element died mid-request). Liveness: Threaded uses `Thread#alive?`; Isolated uses `Utilities.ractor_alive?`/`ractor_status`, which parse `Ractor#to_s` because Ractor has no status API. Thread state uses `Utilities.thread_state` (`'dead'`/`'failed'`/`Thread#status`).
78
+
79
+ Public element API (called by `Swarm`/`Supervisor` only):
80
+ - `try_swarm!`: no-op unless `enabled?`; terminate, then `swarm!`, then start.
81
+ - `reswarm_if_dead!`: no-op unless `enabled?`; `swarmed__stopped?` → restart the loop; `swarmed__dead? || idle?` → `swarm!` and start. This is how a crashed main loop (`abort_on_exception = false`) or a killed element recovers.
82
+ - `try_kill!`: terminate regardless of `enabled?`.
83
+ - `status`: Threaded `{ enabled:, thread: { running:, state: }, main_loop: { running:, state: } }`; Isolated is the same with `ractor:` instead of `thread:`. Unborn parts report `'non_initialized'`. Isolated gets the main loop part from a single `:status` request, and a dead element reports `'non_initialized'`.
84
+
85
+ ## Swarm orchestration (`Swarm#swarm!` / `#deswarm!`)
86
+ - `swarm!` (under `sync`): `supervisor.stop!` → each `element.try_swarm!` → `supervisor.observe! { each element.reswarm_if_dead! }` unless running → `sleep(0.1)` → `{ ok: true, result: :swarming }`. It can be called again; it re-creates everything.
87
+ - `deswarm!`: `supervisor.stop!` first, so it doesn't resurrect elements, then each `element.try_kill!` → `sleep(0.1)` → `{ ok: true, result: :terminating }`.
88
+ - `swarm_status`: `{ auto_swarm:, supervisor: { running:, state:, observable: }, probe_hosts: <status>, flush_zombies: <status> }`.
89
+ - The supervisor block swallows errors (`yield rescue nil`), so a failing element never kills the visor.
90
+
91
+ ## Config (`config.rb`)
92
+ `swarm.auto_swarm`, `swarm.supervisor.liveness_probing_period` (s). Per element: `swarm.<el>.enabled_for_swarm`, a period (`probe_hosts.probe_period` s, `flush_zombies.zombie_flush_period` s), and the 4-key `swarm.<el>.redis_config.{sentinel,pooled,config,pool_config}` group. Flush extras: `zombie_ttl` (ms), `zombie_lock_scan_size`, `zombie_queue_scan_size`.
93
+
94
+ ## Claude rules
95
+ 1. Swarm operations are stateless class-level functions (`ProbeHosts.probe_hosts`, `FlushZombies.flush_zombies`, `ZombieInfo.*`, `Acquirers.acquirers`) that take `redis_client` plus plain values. The manual public API and the main loop call the same function. Don't put Redis logic in instance methods.
96
+ 2. New element: subclass `Threaded` (it needs the client's ractor: threads, `rql_client`, or non-shareable objects) or `Isolated` (self-contained work on copied values). Implement `enabled?` plus `spawn_main_loop!` (Threaded, returns the `Thread`) or `spawn_swarm_element!(swarm_element_results_port)` (Isolated: return `Ractor.new(swarm_element_results_port, <plain args>) { |r_res_p, ...| <Klass>.swarm_loop(r_res_p) { Thread.new { ... } } }`). Don't override `swarm!` in Isolated subclasses; the base wires the ports, the monitor and the handshake.
97
+ 3. Register a new element everywhere: `attr_reader` + `initialize`, `swarm!` (`try_swarm!`), the supervisor block (`reswarm_if_dead!`), `deswarm!` (`try_kill!`), `swarm_status`. Add the config group (`enabled_for_swarm`, period, `redis_config.*`) with `validate`s, and add `Client` delegators with config defaults.
98
+ 4. Main loops build their own Redis client with `RedisClientBuilder` inside the loop thread. Never reuse `rql_client.redis_client` in a loop, and never pass a client, logger or other non-shareable object into a Ractor. Pass `config.slice(...)` and scalars instead.
99
+ 5. Inside a Ractor, reference only shareable constants and module functions (`RedisQueuedLocks::Swarm::X.op`, `RedisQueuedLocks::Resource`, `RedisQueuedLocks::Utilities`). No instance state, no captured locals.
100
+ 6. Change element state only inside `sync.synchronize`. `Utilities::Lock` is a reentrant `Monitor`, so the public→private nesting is intended. Keep the single-slot request/reply pairing atomic.
101
+ 7. Keep the lifecycle API names and semantics (`try_swarm!`, `reswarm_if_dead!`, `try_kill!`, `status`, the `swarmed__*` predicates, the `:status`/`:is_active`/`:start`/`:stop`/`:kill` commands) the same across both bases. The supervisor and `Swarm` rely on that duck typing.
102
+ 8. Main loop threads must have `abort_on_exception = false` (both control units set this) and let errors kill only the loop. Recovery belongs to the supervisor, so don't add retry loops inside elements.
103
+ 9. Stop the supervisor before killing or re-creating elements, or it will race and resurrect them.
104
+ 10. Report element status as the nested `{ running:, state: }` hashes with `'non_initialized'` for missing parts. Update `swarm_status` RBS types when the shape changes.
105
+ 11. Zombie detection compares probe scores (`Time.now.to_f`) with `Resource.calc_zombie_score`. Keep `zombie_ttl` in ms at the API and convert with `/ 1_000.0`. Use `Resource::SWARM_KEY`/`*_PATTERN` and never hardcode keys.
106
+ 12. Mirror every change in `sig/redis_queued_locks/swarm/**` and keep the existing `# steep:ignore` on nil-narrowed `Thread?`/`Ractor?` calls (Steep doesn't narrow `attr_reader` results).
107
+ 13. Isolated communication goes only through `Ractor::Port`s: create the results port in the main Ractor (`swarm!`) and the command port inside the element Ractor (`.swarm_loop`), and register `Ractor#monitor` on the results port before waiting on it. Every command gets exactly one reply, sent before the Ractor leaves its loop. Treat Symbol replies as death notices. Don't use `Ractor.receive`, `Ractor#send` or the default port, and never share one port between elements. Naming: port variables, attributes and params end in `_port` (`swarm_element_results_port`, `swarm_element_commands_port`). Block (proc) params use short initial-letter names (`r_res_p` for the results port, `r_com_p` for the commands port), like the other abbreviated Ractor block params (`rc`, `z_ttl`).
108
+ 14. Inside an element Ractor, terminate threads with kill + join (`.terminate_thread`) and keep `:kill` terminating every thread of the Ractor. Otherwise its status lags as `running`.
109
+ 15. Don't use `{ ok:, result: }` inside swarm elements (see "Internal API: scalars and primitives"). Commands are Symbols. Replies, helper returns and predicates are bare scalars/primitives, or a flat Hash of scalars, with `nil` for "no answer / dead". Keep the wrapper only at the public boundary (`Client`, `Acquirer::*`, `Swarm#swarm!`/`#deswarm!`, `ProbeHosts.probe_hosts`, `FlushZombies.flush_zombies`). Type internal replies in RBS as literal types/unions (`bool`, `String`, `{ alive: bool, state: String }`) rather than `xxxResult` records.
110
+ 16. Swarm specs (in `describe 'swarm'`) deal with real timing: set short periods via config, kill elements with `try_kill!`, wait longer than `liveness_probing_period`, and match states loosely (`eq('running').or(eq('blocking'))`, `'sleep'`/`'run'`).
111
+
112
+ ## Known quirks (don't copy; fix only in a dedicated change)
113
+ - Typos: `swarm_element__termiante`, `@since 19.0.0` on `Threaded#reswarm_if_dead!`, `@api ppublic` on `Swarm#zombie_acquirers`, "lopp"/"teh" in comments.
114
+ - `swarm_loop__stop` is not used by the swarm itself, only by specs.
115
+ - Killing an element while its main loop is still connecting to Redis makes Ruby's `Socket.tcp` helper threads print `terminated with exception ... IO#write` reports. This is harmless noise from the stdlib.
116
+ - `sleep(0.1)` "give a timespot" waits in `swarm!`/`deswarm!`/`observe!` instead of real readiness signalling.
117
+ - RBS runtime type checking (`rbs/test/setup`, the `typecheck-runtime` CI job) can't run inside Ractors. Its hooks on methods called in the element Ractor (`.swarm_loop`, `.flush_zombies`) read `RBS.logger` and raise `Ractor::IsolationError`, so Isolated elements die at startup and the swarm specs fail only under that job.
118
+ - Swarm has no logging or instrumentation, and supervisor errors are silently dropped (`TODO: (CHECK)`).
119
+ - `FlushZombies` scans the whole keyspace on every run (four patterns: write locks, readers registries, write and read queues) and runs `ZREM` for every zombie acquirer on every queue.
120
+ - Zombie acquirers are collected only from held locks (write locks and read locks): requests of dead hosts that never obtained a lock are removed only by queue TTL pruning.
@@ -0,0 +1,48 @@
1
+ ---
2
+ paths:
3
+ - "spec/**/*.rb"
4
+ - ".rspec"
5
+ ---
6
+
7
+ # Test rules (`spec/`)
8
+
9
+ ## Observed conventions
10
+ - All behavior specs live in one integration file, `spec/redis_queued_locks_spec.rb` (~3.4k lines), under `RSpec.describe RedisQueuedLocks`; groups: `describe 'Lock Series PoC'`, `describe 'swarm'`, `describe 'read/write locks'`, plus top-level `specify` blocks. The file header says it will be reworked; rspec-retry (5 retries) masks flakiness meanwhile.
11
+ - Known flaky example: `all in + notifications` (timing-dependent `sleep(1)`; passes in isolation).
12
+ - `.rspec` auto-requires `spec_helper`; `spec_helper.rb` loads SimpleCov first (`setup_simplecov.rb`), then `rspec/retry`, `pry`, the gem.
13
+ - RSpec config: random order (`Kernel.srand config.seed`), `disable_monkey_patching!`, `expect` syntax only, `filter_run_when_matching :focus`, `Thread.abort_on_exception = true`.
14
+ - Tests hit a real Redis (db 0) via `let(:redis) { RedisClient.config(db: 0).new_pool(timeout: 5, size: 50, ...) }` with long timeouts (RBS runtime-check runs are slow); `all in + notifications` also flushes db 1. Check the local Redis dbs before running the suite on a machine with other data.
15
+ - Read/write lock specs use: a monotonic `timeline` hash + `mark` lambda (guarded by a `Mutex`) to assert ordering (`reader_in < other_reader_out`, `writer_in >= reader_out`); an invariant spec with shared reader/writer counters checked inside lock blocks; helper threads that take locks without a block (each thread is a separate acquirer).
16
+ - `before`: `FLUSHDB`, `DEL Resource::SWARM_KEY`, `RedisQueuedLocks.enable_debugger!`; `after`: `DEL SWARM_KEY`, `FLUSHDB`.
17
+ - Examples are written with `specify '<feature>'` (few `it`); grouped with `describe` only for big areas (`'Lock Series PoC'`, `'swarm'`).
18
+ - Clients are built inline: `RedisQueuedLocks::Client.new(redis) { |config| ... }`.
19
+ - Logger/instrumenter fakes are anonymous classes (`Class.new { def debug(...); def notify(event, payload = {}) }`) collecting calls into arrays.
20
+ - Concurrency is tested with real `Thread.new` and `sleep` to wait for async swarm elements; assertions use `match(...)`, `eq(...).or(eq(...))`, `include`, `raise_error(RedisQueuedLocks::...Error)`.
21
+ - Every example cleans up its own state (`client.clear_locks`, `deswarmize!`, `redis.close`).
22
+ - Coverage: SimpleCov line + branch, HTML only; `minimum_coverage 100` is a TODO.
23
+
24
+ ## Claude rules
25
+ 1. Run specs with a local Redis available: `bundle exec rake rspec` (single example: `bundle exec rspec spec/redis_queued_locks_spec.rb:<line>`).
26
+ 2. Add new examples to `spec/redis_queued_locks_spec.rb` with `specify '<feature>'`, inside an existing `describe` when one fits; don't create new spec files unless asked (the suite is pending a rework).
27
+ 3. Use only `expect` syntax; no `should`, no monkey-patched DSL, no `focus` left behind.
28
+ 4. Use the shared `redis` pool and rely on the global `before`/`after` FLUSHDB; never use a different DB or flush in the middle of an example without reason.
29
+ 5. Build fakes as anonymous classes implementing the duck-typed interface (`debug`, `notify`, `sampling_happened?`), not with doubles of real loggers.
30
+ 6. Clean up everything an example starts: release locks, `deswarmize!` swarm clients, join/kill threads.
31
+ 7. Keep `sleep`-based waits minimal and comment why (`# give a timespot to ...`); prefer polling with a bounded timeout when adding new async checks.
32
+ 8. Do not add new rspec-retry reliance or lower retry settings; do not enable `minimum_coverage` without being asked.
33
+ 9. Specs are also run under RBS runtime checks, so pass correctly typed arguments to public API calls (type violations are logged in CI); e.g. test invalid read lock ttl with `ttl: 0`, not `ttl: nil` (`Client#lock` types `ttl` as `Integer`).
34
+ 10. Check new or changed examples for flakiness without retries: `RSPEC_RETRY_RETRY_COUNT=1 bundle exec rspec spec/redis_queued_locks_spec.rb -e '<group>'`, several runs in a row; also watch for `RSpec::Retry: 2nd try` lines in normal runs.
35
+ 11. Threads in specs: `Thread.abort_on_exception = true`, so never raise expected errors inside threads (call non-raising `lock` there and assert results in the main thread); keep references to helper threads until the end of the example (acquirer ids contain `Thread#object_id`, which can be reused after GC).
36
+ 12. Timing assertions on TTLs allow the redis time shift error: extendable reentrant locks return the extension minus the time spent in the inner block and `Resource::REDIS_TIMESHIFT_ERROR` (2 ms), so the remaining TTL can slightly exceed the initial one.
37
+ 13. New lock features get specs for both modes when relevant (`read_write_mode: :write` and `:read`): ordering, mutual exclusion, reentrancy per conflict strategy, `fail_fast`, timeouts/dequeue, release/cleanup/info/zombie paths, and that no `rql:*` keys are left (`client.keys` is empty) after blocks finish.
38
+
39
+ ## Recommendations (proposed, not yet project policy)
40
+ Apply to new tests; don't restructure the existing suite unless asked.
41
+ 1. Move toward one spec file per feature, mirroring `lib/` (`spec/redis_queued_locks/acquirer/extend_lock_ttl_spec.rb`), when the planned rework starts.
42
+ 2. Extract the repeated fake logger/notifier classes into `spec/support/` shared helpers or `shared_context`.
43
+ 3. Add a `wait_until(timeout:) { condition }` helper and use it instead of fixed `sleep` for swarm/thread checks; this reduces flakiness and rspec-retry reliance.
44
+ 4. Add Redis-free unit specs for pure modules (`Resource`, `Config` + validators, samplers, `Logging.should_log?`); they are fast and stable.
45
+ 5. Tag slow swarm/Ractor examples (e.g. `:swarm`) so they can be run or skipped separately.
46
+ 6. Read the Redis connection from `ENV['REDIS_URL']` (default `redis://localhost:6379/0`) so specs can run against non-default Redis setups.
47
+ 7. Once the suite is stable, remove rspec-retry and raise `minimum_coverage` step by step toward 100.
48
+ 8. Cover failure paths explicitly: timeouts, retry exhaustion, `fail_fast`, `raise_errors`, each conflict and access strategy.
@@ -0,0 +1,50 @@
1
+ ---
2
+ paths:
3
+ - "sig/**/*.rbs"
4
+ - "Steepfile"
5
+ - "rbs_collection.yaml"
6
+ - "lib/**/*.rb"
7
+ ---
8
+
9
+ # Type-checking rules (`sig/`, Steep, RBS)
10
+
11
+ ## Observed conventions
12
+ - `sig/` mirrors `lib/` one file per file (`lib/redis_queued_locks/acquirer/is_locked.rb` → `sig/redis_queued_locks/acquirer/is_locked.rbs`). Known typo: `sig/redis_queued_locks/acquier.rbs` is the `Acquirer` namespace file.
13
+ - RBS uses fully nested `module RedisQueuedLocks / module Acquirer / ...` blocks (unlike compact Ruby constants).
14
+ - Common aliases at the top: `use RedisQueuedLocks as RQL`, `use RedisClient as RC`; Redis connections typed as `RC::client`.
15
+ - Shared duck types live in `sig/redis_queued_locks.rbs`: `_Loggable`, `_Instrumentable`, `loggerObj`, `instrObj`; samplers as `RQL::Logging::samplerObj` / `RQL::Instrument::samplerObj`.
16
+ - Result shapes are named record aliases inside the module, e.g. `type extendResult = { ok: bool, result: { extended_locks_count: Integer } | Symbol }`.
17
+ - Each `Acquirer::*` operation module (except PoC modules) declares its own aliases and never references another operation module's aliases (`Locks::lockInfo` / `Locks::readerInfo` mirror `LockInfo::lockInfo` / `LockInfo::readerInfo`); `Client` signatures may reference any of them.
18
+ - Module functions are declared `def self.name: (...) -> T`; long signatures put one param per line with names.
19
+ - Instance variables are declared (`@config_setters: configSetters`) and attr readers typed.
20
+ - Third-party types: `sig/vendor/*.rbs` hand-written stubs (redis_client, active_support, semantic_logger) plus `rbs collection` gems (redis-client, securerandom, timeout, logger, monitor) installed into `.gem_rbs_collection/`.
21
+ - `Steepfile`: target `lib`, signatures `sig`, ignores `spec`, libraries timeout/securerandom/logger/monitor, diagnostics `Steep::Diagnostic::Ruby.strict`.
22
+ - In Ruby code: `# @type var x: T` and `x = ... #: T` annotations narrow types; `# steep:ignore` silences spots Steep really reports (nil-narrowed `attr_reader` results, pattern-matching destructures, splats). `Config#[]` returns `untyped`, so `config['...']` defaults in `Client` need no ignore.
23
+ - Steep 2.1 pitfalls found in this codebase:
24
+ - record types with String keys (`{ 'acq_id' => String, ... }`) crash Steep's subtyping (`RuntimeError`/`Unexpected error` logged as `ERROR`/`FATAL`, while the summary still says "No type error detected"): use `Hash[String, ...]` aliases instead (see `QueueInfo::queueInfo`, `LockInfo::lockInfo`);
25
+ - `x != nil` does not narrow `String?`/`Float?` in `elsif` branches or `&&` chains on locals typed from tuples: use truthiness (`elsif x`) or total conversions (`x.to_f > now`, `ttl.to_i`);
26
+ - `pipelined`/`call` return `untyped`: annotate destructured pipeline results with a tuple (`# @type var lock_state: [String?, Array[String], Float?, Array[[String, Float]]]`);
27
+ - `LockSeriesPoC` has no RBS: new calls/definitions there need `# steep:ignore` (like the existing ones).
28
+ - Steep 2 narrows on `==` with literals (`x == :sym` makes `x` the literal type `:sym`) and rejects `# @type var` annotations that widen a narrowed variable; RBS 4's `Kernel#Array` has a `(nil) -> []` overload that `untyped` arguments resolve to, so `Array(rconn.call(...)).first` needs a `#: T` assertion.
29
+ - Runtime checking: CI runs specs with `RUBYOPT=-rrbs/test/setup RBS_TEST_TARGET='RedisQueuedLocks::*'`, so signatures must match real runtime values, not just Steep's view.
30
+
31
+ ## Claude rules
32
+ 1. Every change to a `lib/` file's public or private API (new method, param, return shape, constant, ivar) must be reflected in the mirrored `.rbs` file in the same change.
33
+ 2. Create new `.rbs` files at the mirrored path with nested module blocks and `use RedisQueuedLocks as RQL` / `use RedisClient as RC` aliases when needed.
34
+ 3. Reuse shared types (`RQL::loggerObj`, `RQL::instrObj`, `samplerObj`, `RC::client`) instead of `untyped`; use `untyped` only for truly dynamic values (e.g. user `meta`, `instrument`).
35
+ 4. Name public API result hashes with a `type xxxResult = { ok: bool, result: ... }` alias next to the method. Type internal swarm element replies with plain types (`bool`, `String`, a flat record such as `{ alive: bool, state: String }`, optional `?` for `nil`) instead of `{ ok:, result: }` records.
36
+ 5. Prefer fixing types or adding `# @type var` / `#: T` annotations over `# steep:ignore`; add `# steep:ignore` only where Steep reports a diagnostic. Strict mode fails the build on `Ruby::RedundantIgnoreComment`, so never add a speculative ignore and remove ones that become redundant.
37
+ 6. Don't loosen `Steep::Diagnostic::Ruby.strict` or add `ignore` entries to the `Steepfile`.
38
+ 7. New third-party gem used in `lib/`: add it to `rbs_collection.yaml` (and `sig/manifest.yml` for stdlib) or write a minimal stub in `sig/vendor/`.
39
+ 8. Verify with `bundle exec rbs collection install && bundle exec rake steep:check`; for signature/runtime mismatches run the specs under RBS runtime testing (command in `.github/workflows/typecheck-runtime.yml`).
40
+ 9. Don't rename `acquier.rbs` unless asked; it is a known quirk.
41
+ 10. After `steep:check`, also grep its output for `ERROR`/`FATAL` lines: an internal Steep crash skips the file but still reports "No type error detected".
42
+ 11. `bundle exec rbs collection install` may rewrite `rbs_collection.lock.yaml` with environment-specific entries; restore it (`git checkout -- rbs_collection.lock.yaml`) unless the change is intended.
43
+
44
+ ## Recommendations (proposed, not yet project policy)
45
+ 1. Make the runtime type-check CI job blocking (drop `--failure-exit-code=0`) once current violations are fixed.
46
+ 2. Type `config['...']` lookups (`Config#[]` returns `untyped`, so `Client` keyword defaults are unchecked) with a typed config accessor (e.g. per-key typed readers or an RBS overload table for `Config#[]`).
47
+ 3. Pin the `gem_rbs_collection` revision in `rbs_collection.yaml` instead of `main` for reproducible checks.
48
+ 4. Rename `sig/redis_queued_locks/acquier.rbs` to `acquirer.rbs` in a dedicated change.
49
+ 5. Replace remaining `untyped` in signatures with precise unions or interfaces where the value set is known (e.g. strategy symbols as `:queued | :random`).
50
+ 6. Type strategy and mode options as literal unions (`conflict_strategy: :wait_for_lock | :work_through | :extendable_work_through | :dead_locking`) so Steep catches invalid values.
@@ -0,0 +1,69 @@
1
+ ---
2
+ paths:
3
+ - "lib/redis_queued_locks/acquirer/**/*.rb"
4
+ - "lib/redis_queued_locks/logging.rb"
5
+ - "lib/redis_queued_locks/logging/**/*.rb"
6
+ - "lib/redis_queued_locks/instrument.rb"
7
+ - "lib/redis_queued_locks/instrument/**/*.rb"
8
+ ---
9
+
10
+ # Log & instrumentation visitors
11
+
12
+ ## What it is
13
+ The project calls these modules "visitors", but they are not GoF Visitor (no `accept`/double dispatch). They are **stateless event-hook modules**: one module per algorithm component, one method per lifecycle event. They take the observability "port" (`logger` or `instrumenter`) plus event data, and emit exactly one log line or one notification. The algorithm code only says *what happened* (`LogVisitor.lock_obtained(...)`); the visitor decides *how it is reported*.
14
+
15
+ **Why**: keeps the lock algorithm readable (one call per event instead of inline string building), centralizes event names and payload shapes, makes sampling and error-swallowing uniform, and gives RBS a typed contract for every event.
16
+
17
+ ## Where they live
18
+ | Visitor | File | Events |
19
+ |---|---|---|
20
+ | `AcquireLock::LogVisitor` | `acquire_lock/log_visitor.rb` | 6: `start_lock_obtaining`, `start_try_to_lock_cycle`, `dead_score_reached__reset_acquirer_position`, `extendable_reentrant_lock_obtained`, `reentrant_lock_obtained`, `lock_obtained` |
21
+ | `AcquireLock::InstrVisitor` | `acquire_lock/instr_visitor.rb` | 5: `extendable_reentrant_lock_obtained`, `reentrant_lock_obtained`, `lock_obtained`, `reentrant_lock_hold_completes`, `lock_hold_and_release` |
22
+ | `AcquireLock::TryToLock::LogVisitor` | `acquire_lock/try_to_lock/log_visitor.rb` | 18 step-level events (`start`, `rconn_fetched`, `acq_added_to_queue`, `exit__no_first`, `exit__read_lock_still_obtained`, `obtain__free_to_acquire`, ...); every lock event logs `rw_mode` |
23
+ | `AcquireLock::YieldExpire::LogVisitor` | `acquire_lock/yield_expire/log_visitor.rb` | `expire_lock`, `decrease_lock` |
24
+ | `AcquireLock::DequeueFromLockQueue::LogVisitor` | `acquire_lock/dequeue_from_lock_queue/log_visitor.rb` | `dequeue_from_lock_queue` |
25
+ | `LockSeriesPoC::LogVisitor` / `InstrVisitor` | `lock_series_poc/*_visitor.rb` | lock series events (no RBS, `# steep:ignore`): `start_lock_series_obtaining`, `lock_series_obtained`, `expire_lock_series` / `lock_series_obtained`, `lock_series_hold_and_release`; all of them log/carry `rw_mode` (the mode of the series) |
26
+
27
+ Not covered by visitors: `release_lock`, `release_read_lock`, `release_all_locks`, `release_locks_of` call `instrumenter.notify` inline inside `run_non_critical`, and do not log at all (their `logger` param is unused).
28
+
29
+ ## Observed conventions
30
+ - **Placement**: a visitor sits in a subdirectory named after the component it reports on (`<component>/log_visitor.rb`, `<component>/instr_visitor.rb`) and is loaded by that component with `require_relative` at the top of the module body.
31
+ - **Shape**: `module <Component>::LogVisitor` / `::InstrVisitor`, `# @api private`, all methods inside `class << self`, full YARD block per method; return `void`.
32
+ - **Method name = event name**: snake_case; double underscore separates phase and detail (`exit__queue_ttl_reached`, `reentrant_lock__work_through`). The LogVisitor and InstrVisitor use the same method name for the same event.
33
+ - **Parameter order**: `(port, sampled_flag, [extra gate], lock_key, ...event data..., [instrument])`:
34
+ - LogVisitor: `(logger, log_sampled, ...)`; `TryToLock::LogVisitor` adds `log_lock_try` as the third arg (fine-grained step logs are opt-in through `config['log_lock_try']`). Lock lifecycle events (AcquireLock, TryToLock, YieldExpire, DequeueFromLockQueue, LockSeriesPoC) take `rw_mode` right after `access_strategy`, before event-specific data, and log it as `rw_mode => '<mode>'` right after `acs_strat`.
35
+ - InstrVisitor: `(instrumenter, instr_sampled, lock_key, rw_mode, ttl, acq_id, hst_id, ts, acq_time, [hold_time], instrument)`; the payload carries `rw_mode:` (requested mode); the user's `instrument` value is always last.
36
+ - `rw_mode` in AcquireLock/TryToLock/DequeueFromLockQueue events is the **requested** mode; in YieldExpire events it is the **held** mode (the lock that is released/decreased); in LockSeriesPoC events it is the mode of the series (`read_write_mode`, one for all locks of the series). LockSeriesPoC instrumentation takes `rw_mode` right after `lock_keys`.
37
+ - **Guard first**: `return unless log_sampled` (`&& log_lock_try` for try-lock steps) / `return unless instr_sampled`. The sampling decision is computed **once per operation** in the caller via `Logging.should_log?` / `Instrument.should_instrument?` and passed down as a boolean.
38
+ - **Log format**: one `logger.debug { ... }` block (lazy string), message = `"[redis_queued_locks.<event>] "` (`[redis_queued_locks.try_lock.<event>]` for TryToLock steps) followed by `key => value` pairs; string values in single quotes (`lock_key => '...'`), numbers bare; abbreviated keys `acq_id`, `hst_id`, `acs_strat`. Built with `\` line continuations.
39
+ - **Instrumentation format**: `instrumenter.notify('redis_queued_locks.<event>', { lock_key:, ttl:, acq_id:, hst_id:, ts:, acq_time:, instrument: })` using shorthand hash syntax and Symbol keys.
40
+ - **Never raise**: every emit ends with `rescue nil` (modifier), so a broken logger/instrumenter can't affect lock correctness.
41
+ - **Call sites**: plain module calls with positional args, usually grouped on 2-3 lines (`LogVisitor.lock_obtained(logger, log_sampled, lock_key, ...)`); data comes from local vars or the `result` hash of the step.
42
+ - **Typing**: each visitor (except lock_series_poc) has an RBS file with `def self.<event>: (RQL::loggerObj logger, bool log_sampled, ...) -> void` / `RQL::instrObj instrumenter`.
43
+ - **User docs**: README documents every log event with its logged keys (`## Logging`: the default and `log_lock_try` lists, mirrored in the `config['logger']` / `config['log_lock_try']` comments of `### Logging Configuration`) and every instrumentation event with its payload keys, types and semantics (`### Instrumentation Events`).
44
+
45
+ ## Claude rules
46
+ **When to use**
47
+ 1. Any new log line or instrumentation event inside `AcquireLock` (and its mixins) or `LockSeriesPoC` must go through a visitor method; never call `logger.*` or `instrumenter.notify` directly in algorithm code.
48
+ 2. Add a LogVisitor event for each new meaningful step or branch of the algorithm (start, decision, exit, success). Add an InstrVisitor event only for outcomes users would want to measure (obtained, held/released, completed); not for internal steps.
49
+ 3. Fine-grained per-attempt logging belongs in `TryToLock::LogVisitor` and must be gated by `log_lock_try`.
50
+ 4. For a new algorithm component (new mixin under `acquire_lock/`), create its own `<component>/log_visitor.rb` (and `instr_visitor.rb` if needed) and `require_relative` it from the component.
51
+
52
+ **How to write one**
53
+ 5. Method: inside `class << self`, named exactly after the event (snake_case, `__` for phase/detail), full YARD with `@return [void]`, `@api private`, `@since <next version>`.
54
+ 6. Parameters: port first, sampled flag second, extra gates next, then `lock_key`, then event data; lock events take `rw_mode` (after `access_strategy` for logs, after `lock_key` for instrumentation); for instrumentation keep the user `instrument` value last. Keep the list explicit (see `arguments.md`).
55
+ 7. First line: `return unless <sampled>` (plus gate). Don't compute sampling inside the visitor and don't recompute it at the call site per event; reuse the operation's `log_sampled` / `instr_sampled`.
56
+ 8. Logs: `logger.debug do ... end rescue nil` with message `"[redis_queued_locks.<event>] "` (`try_lock.` prefix for TryToLock steps) and `key => value` pairs, quoting strings, using the existing abbreviations (`acq_id`, `hst_id`, `acs_strat`, `queue_ttl`).
57
+ 9. Instrumentation: `instrumenter.notify('redis_queued_locks.<event>', { ... }) rescue nil` with shorthand Symbol keys; reuse the standard payload keys (`lock_key`, `rw_mode`, `ttl`, `acq_id`, `hst_id`, `ts`, `acq_time`, `hold_time`, `instrument`). Event names are public API: never rename or remove one without being asked, and document new ones in the README/CHANGELOG.
58
+ 10. Use the same method name in LogVisitor and InstrVisitor when both report the same event.
59
+ 11. Add the method to the mirrored RBS visitor file (`RQL::loggerObj` / `RQL::instrObj`, `bool` sampled flag, `-> void`) and cover the new event in specs via the fake logger/notifier (assert on the `[redis_queued_locks.<event>]` prefix or event name).
60
+ 12. Never pass Redis reads (or other costly computations) as visitor arguments unconditionally: arguments are evaluated before the visitor's `return unless` guard. Gate them at the call site with the same condition (`(log_sampled && log_lock_try) ? rconn.call('HGETALL', ...).to_h : {}`).
61
+ 13. A new branch/exit of the read/write algorithm gets its own TryToLock event (`exit__write_request_ahead`, `exit__read_request_ahead`, `exit__read_lock_still_obtained`, `single_process_lock_conflict__lock_upgrade` are the existing ones). Don't add events to the default write success path: specs assert its exact log sequence (10 lines with `log_lock_try`).
62
+ 14. Any change of observable data is documented in README in the same change: a new log event or a new/renamed/removed logged key (`LogVisitor`s) → `## Logging` (both lists and the matching `config['logger']` / `config['log_lock_try']` comments in `### Logging Configuration`); a new instrumentation event or a new/renamed/removed payload key (`InstrVisitor`s and the inline `notify` of release modules) → `### Instrumentation Events` (event list + payload line with type and semantics, e.g. requested vs held `rw_mode`). Add a CHANGELOG `[Unreleased]` line as well. Before finishing, compare the README lists with the visitor code (event names and key order).
63
+
64
+ ## Recommendations (proposed, not yet project policy)
65
+ Apply to new or touched code; don't refactor existing code for these unless asked.
66
+ 1. Give the release operations (`release_lock`, `release_all_locks`, `release_locks_of`) their own `InstrVisitor`/`LogVisitor` instead of inline `instrumenter.notify`, and either use or drop their currently unused `logger` param.
67
+ 2. Use one error-swallowing style consistently (visitors use modifier `rescue nil`, release modules use `run_non_critical`). Both silently hide bugs such as a `NoMethodError` inside the log block, so consider reporting swallowed errors when the debugger is enabled.
68
+ 3. Keep log and instrumentation event names in one registry constant (e.g. `RedisQueuedLocks::Instrument::EVENTS`) so README, specs and RBS can be checked against it.
69
+ 4. Fix the RBS path mismatch: `sig/.../acquire_lock/yield_with_expire/log_visitor.rbs` mirrors `lib/.../acquire_lock/yield_expire/log_visitor.rb`; add RBS for `lock_series_poc` visitors when the PoC stabilizes.
data/.rubocop.rbs.yml ADDED
@@ -0,0 +1,31 @@
1
+ # NOTE: RBS cops only: RBS signatures (sig/**/*.rbs) and inline RBS annotations in Ruby sources
2
+ # (lib/**/*.rb, RBSInline/* cops). Ruby cops are disabled here: Ruby sources are linted with .rubocop.yml;
3
+ inherit_gem:
4
+ armitage-rubocop:
5
+ - lib/rubocop.rbs.yml
6
+
7
+ AllCops:
8
+ TargetRubyVersion: 4.0
9
+ NewCops: enable
10
+ DisabledByDefault: true
11
+ UseProjectIndex: false
12
+ Include:
13
+ - sig/**/*.rbs
14
+ - lib/**/*.rb
15
+ # NOTE: files with a ruby shebang are picked up implicitly;
16
+ Exclude:
17
+ - bin/**/*
18
+ SuggestExtensions: false
19
+
20
+ # NOTE: `DisabledByDefault` is not enough for Ruby sources: inline `# rubocop:enable <Cop>`
21
+ # (and `# rubocop:enable all`) directives in lib/ re-enable Ruby cops for the rest of the file;
22
+ # so Ruby cop departments are excluded explicitly (they are covered by .rubocop.yml);
23
+ Bundler: { Exclude: ['**/*'] }
24
+ Gemspec: { Exclude: ['**/*'] }
25
+ Layout: { Exclude: ['**/*'] }
26
+ Lint: { Exclude: ['**/*'] }
27
+ Metrics: { Exclude: ['**/*'] }
28
+ Migration: { Exclude: ['**/*'] }
29
+ Naming: { Exclude: ['**/*'] }
30
+ Security: { Exclude: ['**/*'] }
31
+ Style: { Exclude: ['**/*'] }
data/.rubocop.yml CHANGED
@@ -3,15 +3,15 @@ inherit_gem:
3
3
  - lib/rubocop.general.yml
4
4
  - lib/rubocop.rake.yml
5
5
  - lib/rubocop.rspec.yml
6
- - lib/rubocop.rbs.yml
7
6
 
7
+ # NOTE: Ruby sources only. RBS signatures (sig/**/*.rbs) are linted separately with
8
+ # .rubocop.rbs.yml: the Ruby cops (and the project index they use) must not see them;
8
9
  AllCops:
9
- TargetRubyVersion: 3.3
10
+ TargetRubyVersion: 4.0
10
11
  NewCops: enable
11
12
  Include:
12
13
  - lib/**/*.rb
13
14
  - spec/**/*.rb
14
- - sig/**/*.rbs
15
15
  - Gemfile
16
16
  - Rakefile
17
17
  - redis_queued_locks.gemspec
data/.ruby-version CHANGED
@@ -1 +1 @@
1
- 3.4.5
1
+ 4.0.7
data/CHANGELOG.md CHANGED
@@ -1,5 +1,85 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [1.18.0] - 2026-10-11
4
+ ### Added
5
+ - Brand new feature - **Read/Write locks**: `read_write_mode:` attribute of `#lock`/`#lock!`/`#lock_series`/`#lock_series!` (`:write` by default):
6
+ - `:write` - exclusive lock (the classic RQL lock): waits for the write lock and for all read locks;
7
+ - `:read` - shared lock: waits for the write lock only (read locks of different acquirers work in parallel);
8
+ - `:queued` access strategy orders read and write lock requests in FIFO between modes (writers are not starved by readers);
9
+ - `:random` access strategy ignores lock queues (a continuous flow of readers can starve writers);
10
+ - read locks are stored in `rql:lock_readers:<lock_name>` (`ZSET`: acquirer id => expiration time in Redis time),
11
+ read lock requests - in `rql:lock_read_queue:<lock_name>`, read lock data - in `rql:lock_reader:<lock_name>:<acquirer_id>`
12
+ (the same structure as the write lock: acquirer, host, timestamp, ttl, `:meta`, reentrant lock data);
13
+ write locks and write lock requests use the classic keys;
14
+ - mutual exclusion is based on optimistic transactions (`WATCH`/`MULTI`, no Lua): readers watch the write lock key,
15
+ writers watch the write lock key and the read locks key (readers never invalidate transactions of each other);
16
+ - reentrant locks: read lock request under the obtained lock works as a reentrant lock of the obtained lock;
17
+ read-to-write lock upgrade fails with `RedisQueuedLocks::ConflictLockObtainError` (`:conflict_lock_upgrade`)
18
+ for `:work_through`/`:extendable_work_through` conflict strategies;
19
+ - `#unlock`, `#clear_locks`, `#clear_locks_of`, `#clear_current_locks`, `#clear_dead_requests`, `#flush_zombies`,
20
+ `#zombie_locks`, `#zombie_acquirers`, `#zombies_info`, `#locked?`, `#queued?`, `#lock_info`, `#queue_info`,
21
+ `#locks`, `#locks_info`, `#queues`, `#queues_info` support read locks and read lock requests;
22
+ - logs: `rw_mode` in the lock logs and in the lock series logs (`start_lock_series_obtaining`, `lock_series_obtained`,
23
+ `expire_lock_series`: the mode of the series); new try-lock log events: `exit__write_request_ahead`, `exit__read_request_ahead`,
24
+ `exit__read_lock_still_obtained`, `single_process_lock_conflict__lock_upgrade`;
25
+ - instrumentation: `rw_mode` in `lock_obtained`, `reentrant_lock_obtained`, `extendable_reentrant_lock_obtained`,
26
+ `lock_hold_and_release`, `reentrant_lock_hold_completes` payloads and in the lock series payloads
27
+ (`lock_series_obtained`, `lock_series_hold_and_release`: the mode of the series);
28
+ - `:meta` for read locks (stored in the read lock data of each reader, returned by `#lock_info`/`#locks_info`);
29
+ - `#lock_series` with `read_write_mode: :read` (PoC): a series of read locks; only read locks obtained by the series
30
+ are released (reentrant read locks of the outer logic are kept);
31
+ - `#unlock_read` (aka `#release_read_lock`): releases the read lock of the current acquirer only
32
+ (new instrumentation event: `redis_queued_locks.explicit_read_lock_release`);
33
+ - `#extend_lock_ttl`: `read_write_mode:` (`:write` by default) and `identity:` options; `:read` mode extends the read lock
34
+ of the current acquirer only (an expired read lock or a read lock replaced by a writer is not extended);
35
+ - `#extend_lock_ttl`: `all_read_locks:` option (`false` by default, `:read` mode only, ignored in `:write` mode):
36
+ extends all live read locks of the lock (expired read locks are not revived);
37
+ - RBS: `read_write_mode` is typed as `:read | :write`;
38
+ - `#queue_info` / `#queues_info`: each lock request has `"rw_mode"` (`"write"` / `"read"`);
39
+ - `#lock_info` / `#locks_info`: write lock info has `"rw_mode" => "write"` (read locks info - `"rw_mode" => "read"`);
40
+ - `:meta`: `"rw_mode"` is a reserved lock data key (`RedisQueuedLocks::ArgumentError`), for write and read locks and lock series;
41
+ - **all processes should use the RQL version with read/write locks** before read locks are used
42
+ (write locks of older RQL versions do not check read locks);
43
+ ### Changed
44
+ - `lock_series_obtained` instrumentation event (`#lock_series`, PoC): `:ts` is an epoch timestamp (`Float`)
45
+ like in all other instrumentation events (previously a `Time#to_s` string);
46
+ - **Breaking**: `#extend_lock_ttl` success result is `{ ok: true, result: { extended_locks_count: Integer } }`
47
+ instead of `{ ok: true, result: :ttl_extended }` (the number of locks whose TTL is extended: `1` for the write lock
48
+ and for the own read lock, the number of extended read locks for `all_read_locks: true`);
49
+ the failure result is unchanged (`{ ok: false, result: :async_expire_or_no_lock }`);
50
+ - Lock acquirement attempt: the lock state is fetched in one pipelined round trip; lock data for
51
+ `exit__no_first`/`exit__lock_still_obtained` try-lock logs is extracted only when these logs are enabled
52
+ (previously `HGETALL` was called on every unsuccessful attempt);
53
+ ### Fixed
54
+ - `dequeue_from_lock_queue` log: unclosed quote in the `acs_strat` value;
55
+ - `extendable_reentrant_lock_obtained` log: the host identifier is logged as `hst_id` (was `host_id`), like in all other logs;
56
+ - `start_lock_series_obtaining` log: missing space between the `lock_keys` and `queue_ttl` values;
57
+ - `lock_series` PoC:
58
+ - locks obtained by the failed series are released when the series fails without an exception
59
+ (`raise_errors: false`); previously they lived till their TTL;
60
+ - reentrant locks of the series (already obtained by the outer logic of the same acquirer) are not released
61
+ by the series anymore (previously the outer logic continued to work without its lock);
62
+ - the series releases only write locks that are still owned by the current acquirer (compare-and-delete):
63
+ a lock expired during the block and obtained by another acquirer is not deleted anymore;
64
+
65
+ ## [1.17.0] - 2026-10-04
66
+ ### Changed
67
+ - [**Breaking**] Minimal Ruby Version - **4.0** (previously - **3.3**):
68
+ - removed `3.3` and `3.4` from `CI`;
69
+ - Swarm: isolated swarm elements (`RedisQueuedLocks::Swarm::SwarmElement::Isolated`, `FlushZombies`) are reworked
70
+ to the Ruby 4 Ractor API: `Ractor.yield`/`Ractor#take` are replaced with `Ractor::Port`s (each element owns
71
+ its own pair of ports: the results port is created in the main ractor where the swarm supervisor lives and the
72
+ command port is created inside the element ractor), element ractor termination is tracked via `Ractor#monitor`;
73
+ - Swarm: isolated element ractor is terminated deterministically on `#try_kill!` / `#deswarm!`
74
+ (all ractor threads are killed and joined);
75
+ - Swarm: `RedisQueuedLocks::Swarm::SwarmElement::Isolated` subclasses implement `#spawn_swarm_element!(results_port)`
76
+ instead of `#swarm!`;
77
+ - Swarm: internal swarm element protocol (`Threaded` and `Isolated` command replies, isolated element handshake)
78
+ uses bare scalars/primitives instead of `{ ok:, result: }` wrappers (the wrapper is kept for the public API only);
79
+ ### Fixed
80
+ - `RedisQueuedLocks::Client#zombies_info`: default `lock_scan_size` was taken from the
81
+ `swarm.flush_zombies.zombie_ttl` config instead of `swarm.flush_zombies.zombie_lock_scan_size`;
82
+
3
83
  ## [1.16.2] - 2026-02-06
4
84
  ### Fixed
5
85
  - `lock_series` PoC:
data/CLAUDE.md ADDED
@@ -0,0 +1,36 @@
1
+ # CLAUDE.md
2
+
3
+ Ruby gem: distributed Redis locks with a per-lock FIFO acquisition queue. Ruby >= 4.0; only runtime dep is `redis-client ~> 0.20`.
4
+ Full details (config keys, key layout, events, errors, stack, CI): `.claude/project-overview.md`.
5
+
6
+ ## Commands
7
+ - Tests (need a running local Redis): `bundle exec rake rspec`
8
+ - Lint: `bundle exec rake rubocop`
9
+ - Static types: `bundle exec rbs collection install && bundle exec rake steep:check`
10
+
11
+ ## Architecture
12
+ - `client.rb`: public API facade; fills defaults from `config[...]`, delegates to `Acquirer::*`.
13
+ - `acquirer/*.rb`: one stateless module per operation (`class << self`), returns `{ ok:, result: }`; `!` client methods raise.
14
+ - `acquirer/acquire_lock.rb`: composed via `extend` mixins in `acquire_lock/` (try_to_lock, with_acq_timeout, delay_execution, dequeue_from_lock_queue, yield_expire) + `LogVisitor`/`InstrVisitor` event hooks.
15
+ - Locking: `ZADD NX` into the request queue, then WATCH/MULTI; Lua only for write TTL extension/decrease.
16
+ - Read/Write locks (`read_write_mode: :write|:read`): write lock `rql:lock:<name>` + queue `rql:lock_queue:<name>`; readers registry `rql:lock_readers:<name>` (ZSET acq_id => Redis-time expiry), read queue `rql:lock_read_queue:<name>`, per-reader data `rql:lock_reader:<name>:<acq_id>`. Readers WATCH the write key only, writers WATCH the write key + registry; queues give ordering only. Details: overview "Read/Write locks".
17
+ - Strategies: `access_strategy` `:queued`|`:random`; `conflict_strategy` `:wait_for_lock`|`:work_through`|`:extendable_work_through`|`:dead_locking`.
18
+ - `resource.rb`: all key names and acquirer/host IDs. `config.rb`: `setting`/`validate` DSL, read as `config['a.b']`.
19
+ - `swarm/`: zombie-lock cleanup. `Supervisor` thread revives `ProbeHosts` (`SwarmElement::Threaded`) and `FlushZombies` (`SwarmElement::Isolated`, Ractor talking via `Ractor::Port`s); each element runs a main-loop thread with its own Redis connection.
20
+ - `logging/`, `instrument/`: Void null-object defaults, percent samplers, ActiveSupport adapter.
21
+ - `sig/`: RBS mirror of `lib/`. `spec/redis_queued_locks_spec.rb`: single integration spec.
22
+
23
+ ## Rules
24
+ Detailed, path-scoped rules in `.claude/rules/`: `logic.md` (lib, general), `acquirer.md` (acquirer modules, Redis access), `visitors.md` (log/instrumentation visitors), `arguments.md` (long keyword lists), `swarm.md` (swarm supervisor/elements, Ractor/Thread rules), `tests.md` (spec), `type-checking.md` (sig/Steep), `git.md` (commit messages and commit flow, always loaded).
25
+ - Long explicit keyword/parameter lists are intentional (minimal allocations, signature-as-DSL): never introduce parameter objects, option structs or `**opts`; add new options as explicit keywords to every `Client` variant and forward them by name.
26
+ - Update the matching `sig/*.rbs` for every `lib/` change; keep Steep green.
27
+ - Keep `# frozen_string_literal: true` and YARD `@api`/`@since`/`@version` tags.
28
+ - `{ ok:, result: }` is for the public API only (`Client`, `Acquirer::*`, `Swarm` facade actions). Internal swarm element APIs (commands, replies, helpers) use bare scalars/primitives (see `swarm.md`).
29
+ - New operation: `Acquirer::*` module, thin `Client` method (+ `!` variant), RBS, spec.
30
+ - Operation modules behind `Client` methods (all `Acquirer::*` except the `AcquireLock` acquisition core and PoC modules such as `LockSeriesPoC`, which may reuse other modules) are independent: they never call each other or reference each other's RBS types, share logic only via `Resource`/`Utilities`, and duplicate similar logic on purpose (independence over DRY; e.g. `Locks` duplicates the `LockInfo` formatting). Details: `acquirer.md`.
31
+ - New option: `setting` (+ `validate`) in `config.rb`.
32
+ - Dev gems go in `Gemfile`, not the gemspec.
33
+ - Commits (`git.md`): subject only `[<scope>] <summary>` (several features: 1-2 words each, comma-separated), no body; when a step is complete (requirement fully implemented, specs/rubocop/steep green) ask before committing, with the proposed subject and a recap of the changes; an explicit "commit" request needs no extra question; never on `master`, never push.
34
+ - Before implementing, read the relevant sections of `.claude/project-overview.md` (key layout, lock flow, "Read/Write locks", events) and keep their invariants. In the same change, update the overview and the affected `.claude/rules/*.md` for every new/changed key, option, result shape, event, algorithm step or limitation (and README/CHANGELOG `[Unreleased]` for user-visible behavior). New or changed log keys / instrumentation payload keys (visitors or inline `notify`) also go to README `## Logging` / `### Instrumentation Events` (see `visitors.md`).
35
+ - Lock-touching code is read/write-aware: any operation over locks, queues, zombies or lock info must cover the read family (`rql:lock_readers:*`, `rql:lock_read_queue:*`, `rql:lock_reader:*`) next to `rql:lock:*`/`rql:lock_queue:*`. Mutual exclusion comes only from WATCH/MULTI on lock state (no Lua in the RW acquisition path unless asked); queues never guarantee safety.
36
+ - Known debt: rspec-retry, disabled coverage minimum, runtime type-check CI uses `--failure-exit-code=0`.