redis_queued_locks 1.16.2 → 1.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/project-overview.md +205 -0
- data/.claude/rules/acquirer.md +61 -0
- data/.claude/rules/arguments.md +59 -0
- data/.claude/rules/logic.md +57 -0
- data/.claude/rules/swarm.md +119 -0
- data/.claude/rules/tests.md +42 -0
- data/.claude/rules/type-checking.md +42 -0
- data/.claude/rules/visitors.md +64 -0
- data/.rubocop.rbs.yml +31 -0
- data/.rubocop.yml +3 -3
- data/.ruby-version +1 -1
- data/CHANGELOG.md +18 -0
- data/CLAUDE.md +31 -0
- data/README.md +19 -3
- data/Rakefile +31 -2
- data/github_ci/.keep +0 -0
- data/lib/redis_queued_locks/acquirer/acquire_lock/delay_execution.rb +2 -2
- data/lib/redis_queued_locks/acquirer/acquire_lock/try_to_lock.rb +1 -4
- data/lib/redis_queued_locks/acquirer/acquire_lock/yield_expire.rb +1 -1
- data/lib/redis_queued_locks/acquirer/acquire_lock.rb +0 -4
- data/lib/redis_queued_locks/acquirer/lock_series_poc/instr_visitor.rb +0 -2
- data/lib/redis_queued_locks/acquirer/lock_series_poc/log_visitor.rb +0 -1
- data/lib/redis_queued_locks/acquirer/lock_series_poc.rb +0 -2
- data/lib/redis_queued_locks/client.rb +145 -144
- data/lib/redis_queued_locks/config/dsl.rb +4 -10
- data/lib/redis_queued_locks/swarm/flush_zombies.rb +8 -6
- data/lib/redis_queued_locks/swarm/supervisor.rb +1 -1
- data/lib/redis_queued_locks/swarm/swarm_element/isolated.rb +166 -44
- data/lib/redis_queued_locks/swarm/swarm_element/threaded.rb +45 -47
- data/lib/redis_queued_locks/version.rb +2 -2
- data/redis_queued_locks.gemspec +1 -1
- data/sig/redis_queued_locks/swarm/flush_zombies.rbs +1 -1
- data/sig/redis_queued_locks/swarm/swarm_element/isolated.rbs +13 -5
- data/sig/redis_queued_locks/swarm/swarm_element/threaded.rbs +6 -6
- metadata +14 -5
- data/github_ci/ruby3.3.gemfile +0 -17
- data/github_ci/ruby3.3.gemfile.lock +0 -216
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
---
|
|
2
|
+
paths:
|
|
3
|
+
- "sig/**/*.rbs"
|
|
4
|
+
- "Steepfile"
|
|
5
|
+
- "rbs_collection.yaml"
|
|
6
|
+
- "lib/**/*.rb"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Type-checking rules (`sig/`, Steep, RBS)
|
|
10
|
+
|
|
11
|
+
## Observed conventions
|
|
12
|
+
- `sig/` mirrors `lib/` one file per file (`lib/redis_queued_locks/acquirer/is_locked.rb` → `sig/redis_queued_locks/acquirer/is_locked.rbs`). Known typo: `sig/redis_queued_locks/acquier.rbs` is the `Acquirer` namespace file.
|
|
13
|
+
- RBS uses fully nested `module RedisQueuedLocks / module Acquirer / ...` blocks (unlike compact Ruby constants).
|
|
14
|
+
- Common aliases at the top: `use RedisQueuedLocks as RQL`, `use RedisClient as RC`; Redis connections typed as `RC::client`.
|
|
15
|
+
- Shared duck types live in `sig/redis_queued_locks.rbs`: `_Loggable`, `_Instrumentable`, `loggerObj`, `instrObj`; samplers as `RQL::Logging::samplerObj` / `RQL::Instrument::samplerObj`.
|
|
16
|
+
- Result shapes are named record aliases inside the module, e.g. `type extendResult = { ok: bool, result: Symbol }`.
|
|
17
|
+
- Module functions are declared `def self.name: (...) -> T`; long signatures put one param per line with names.
|
|
18
|
+
- Instance variables are declared (`@config_setters: configSetters`) and attr readers typed.
|
|
19
|
+
- Third-party types: `sig/vendor/*.rbs` hand-written stubs (redis_client, active_support, semantic_logger) plus `rbs collection` gems (redis-client, securerandom, timeout, logger, monitor) installed into `.gem_rbs_collection/`.
|
|
20
|
+
- `Steepfile`: target `lib`, signatures `sig`, ignores `spec`, libraries timeout/securerandom/logger/monitor, diagnostics `Steep::Diagnostic::Ruby.strict`.
|
|
21
|
+
- In Ruby code: `# @type var x: T` and `x = ... #: T` annotations narrow types; `# steep:ignore` silences spots Steep really reports (nil-narrowed `attr_reader` results, pattern-matching destructures, splats). `Config#[]` returns `untyped`, so `config['...']` defaults in `Client` need no ignore.
|
|
22
|
+
- Steep 2 narrows on `==` with literals (`x == :sym` makes `x` the literal type `:sym`) and rejects `# @type var` annotations that widen a narrowed variable; RBS 4's `Kernel#Array` has a `(nil) -> []` overload that `untyped` arguments resolve to, so `Array(rconn.call(...)).first` needs a `#: T` assertion.
|
|
23
|
+
- Runtime checking: CI runs specs with `RUBYOPT=-rrbs/test/setup RBS_TEST_TARGET='RedisQueuedLocks::*'`, so signatures must match real runtime values, not just Steep's view.
|
|
24
|
+
|
|
25
|
+
## Claude rules
|
|
26
|
+
1. Every change to a `lib/` file's public or private API (new method, param, return shape, constant, ivar) must be reflected in the mirrored `.rbs` file in the same change.
|
|
27
|
+
2. Create new `.rbs` files at the mirrored path with nested module blocks and `use RedisQueuedLocks as RQL` / `use RedisClient as RC` aliases when needed.
|
|
28
|
+
3. Reuse shared types (`RQL::loggerObj`, `RQL::instrObj`, `samplerObj`, `RC::client`) instead of `untyped`; use `untyped` only for truly dynamic values (e.g. user `meta`, `instrument`).
|
|
29
|
+
4. Name public API result hashes with a `type xxxResult = { ok: bool, result: ... }` alias next to the method. Type internal swarm element replies with plain types (`bool`, `String`, a flat record such as `{ alive: bool, state: String }`, optional `?` for `nil`) instead of `{ ok:, result: }` records.
|
|
30
|
+
5. Prefer fixing types or adding `# @type var` / `#: T` annotations over `# steep:ignore`; add `# steep:ignore` only where Steep reports a diagnostic. Strict mode fails the build on `Ruby::RedundantIgnoreComment`, so never add a speculative ignore and remove ones that become redundant.
|
|
31
|
+
6. Don't loosen `Steep::Diagnostic::Ruby.strict` or add `ignore` entries to the `Steepfile`.
|
|
32
|
+
7. New third-party gem used in `lib/`: add it to `rbs_collection.yaml` (and `sig/manifest.yml` for stdlib) or write a minimal stub in `sig/vendor/`.
|
|
33
|
+
8. Verify with `bundle exec rbs collection install && bundle exec rake steep:check`; for signature/runtime mismatches run the specs under RBS runtime testing (command in `.github/workflows/typecheck-runtime.yml`).
|
|
34
|
+
9. Don't rename `acquier.rbs` unless asked; it is a known quirk.
|
|
35
|
+
|
|
36
|
+
## Recommendations (proposed, not yet project policy)
|
|
37
|
+
1. Make the runtime type-check CI job blocking (drop `--failure-exit-code=0`) once current violations are fixed.
|
|
38
|
+
2. Type `config['...']` lookups (`Config#[]` returns `untyped`, so `Client` keyword defaults are unchecked) with a typed config accessor (e.g. per-key typed readers or an RBS overload table for `Config#[]`).
|
|
39
|
+
3. Remove duplicate entries in `rbs_collection.yaml` (`redis-client` and `securerandom` are listed twice) and pin the `gem_rbs_collection` revision instead of `main` for reproducible checks.
|
|
40
|
+
4. Rename `sig/redis_queued_locks/acquier.rbs` to `acquirer.rbs` in a dedicated change.
|
|
41
|
+
5. Replace remaining `untyped` in signatures with precise unions or interfaces where the value set is known (e.g. strategy symbols as `:queued | :random`).
|
|
42
|
+
6. Type strategy and mode options as literal unions (`conflict_strategy: :wait_for_lock | :work_through | :extendable_work_through | :dead_locking`) so Steep catches invalid values.
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
---
|
|
2
|
+
paths:
|
|
3
|
+
- "lib/redis_queued_locks/acquirer/**/*.rb"
|
|
4
|
+
- "lib/redis_queued_locks/logging.rb"
|
|
5
|
+
- "lib/redis_queued_locks/logging/**/*.rb"
|
|
6
|
+
- "lib/redis_queued_locks/instrument.rb"
|
|
7
|
+
- "lib/redis_queued_locks/instrument/**/*.rb"
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Log & instrumentation visitors
|
|
11
|
+
|
|
12
|
+
## What it is
|
|
13
|
+
The project calls these modules "visitors", but they are not GoF Visitor (no `accept`/double dispatch). They are **stateless event-hook modules**: one module per algorithm component, one method per lifecycle event. They take the observability "port" (`logger` or `instrumenter`) plus event data, and emit exactly one log line or one notification. The algorithm code only says *what happened* (`LogVisitor.lock_obtained(...)`); the visitor decides *how it is reported*.
|
|
14
|
+
|
|
15
|
+
**Why**: keeps the lock algorithm readable (one call per event instead of inline string building), centralizes event names and payload shapes, makes sampling and error-swallowing uniform, and gives RBS a typed contract for every event.
|
|
16
|
+
|
|
17
|
+
## Where they live
|
|
18
|
+
| Visitor | File | Events |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| `AcquireLock::LogVisitor` | `acquire_lock/log_visitor.rb` | 6: `start_lock_obtaining`, `start_try_to_lock_cycle`, `dead_score_reached__reset_acquirer_position`, `extendable_reentrant_lock_obtained`, `reentrant_lock_obtained`, `lock_obtained` |
|
|
21
|
+
| `AcquireLock::InstrVisitor` | `acquire_lock/instr_visitor.rb` | 5: `extendable_reentrant_lock_obtained`, `reentrant_lock_obtained`, `lock_obtained`, `reentrant_lock_hold_completes`, `lock_hold_and_release` |
|
|
22
|
+
| `AcquireLock::TryToLock::LogVisitor` | `acquire_lock/try_to_lock/log_visitor.rb` | 14 step-level events (`start`, `rconn_fetched`, `acq_added_to_queue`, `exit__no_first`, `obtain__free_to_acquire`, ...) |
|
|
23
|
+
| `AcquireLock::YieldExpire::LogVisitor` | `acquire_lock/yield_expire/log_visitor.rb` | `expire_lock`, `decrease_lock` |
|
|
24
|
+
| `AcquireLock::DequeueFromLockQueue::LogVisitor` | `acquire_lock/dequeue_from_lock_queue/log_visitor.rb` | `dequeue_from_lock_queue` |
|
|
25
|
+
| `LockSeriesPoC::LogVisitor` / `InstrVisitor` | `lock_series_poc/*_visitor.rb` | lock series events (no RBS, `# steep:ignore`) |
|
|
26
|
+
|
|
27
|
+
Not covered by visitors: `release_lock`, `release_all_locks`, `release_locks_of` call `instrumenter.notify` inline inside `run_non_critical`, and do not log at all (their `logger` param is unused).
|
|
28
|
+
|
|
29
|
+
## Observed conventions
|
|
30
|
+
- **Placement**: a visitor sits in a subdirectory named after the component it reports on (`<component>/log_visitor.rb`, `<component>/instr_visitor.rb`) and is loaded by that component with `require_relative` at the top of the module body.
|
|
31
|
+
- **Shape**: `module <Component>::LogVisitor` / `::InstrVisitor`, `# @api private`, all methods inside `class << self`, full YARD block per method; return `void`.
|
|
32
|
+
- **Method name = event name**: snake_case; double underscore separates phase and detail (`exit__queue_ttl_reached`, `reentrant_lock__work_through`). The LogVisitor and InstrVisitor use the same method name for the same event.
|
|
33
|
+
- **Parameter order**: `(port, sampled_flag, [extra gate], lock_key, ...event data..., [instrument])`:
|
|
34
|
+
- LogVisitor: `(logger, log_sampled, ...)`; `TryToLock::LogVisitor` adds `log_lock_try` as the third arg (fine-grained step logs are opt-in through `config['log_lock_try']`).
|
|
35
|
+
- InstrVisitor: `(instrumenter, instr_sampled, lock_key, ttl, acq_id, hst_id, ts, acq_time, [hold_time], instrument)`; the user's `instrument` value is always last.
|
|
36
|
+
- **Guard first**: `return unless log_sampled` (`&& log_lock_try` for try-lock steps) / `return unless instr_sampled`. The sampling decision is computed **once per operation** in the caller via `Logging.should_log?` / `Instrument.should_instrument?` and passed down as a boolean.
|
|
37
|
+
- **Log format**: one `logger.debug { ... }` block (lazy string), message = `"[redis_queued_locks.<event>] "` (`[redis_queued_locks.try_lock.<event>]` for TryToLock steps) followed by `key => value` pairs; string values in single quotes (`lock_key => '...'`), numbers bare; abbreviated keys `acq_id`, `hst_id`, `acs_strat`. Built with `\` line continuations.
|
|
38
|
+
- **Instrumentation format**: `instrumenter.notify('redis_queued_locks.<event>', { lock_key:, ttl:, acq_id:, hst_id:, ts:, acq_time:, instrument: })` using shorthand hash syntax and Symbol keys.
|
|
39
|
+
- **Never raise**: every emit ends with `rescue nil` (modifier), so a broken logger/instrumenter can't affect lock correctness.
|
|
40
|
+
- **Call sites**: plain module calls with positional args, usually grouped on 2-3 lines (`LogVisitor.lock_obtained(logger, log_sampled, lock_key, ...)`); data comes from local vars or the `result` hash of the step.
|
|
41
|
+
- **Typing**: each visitor (except lock_series_poc) has an RBS file with `def self.<event>: (RQL::loggerObj logger, bool log_sampled, ...) -> void` / `RQL::instrObj instrumenter`.
|
|
42
|
+
|
|
43
|
+
## Claude rules
|
|
44
|
+
**When to use**
|
|
45
|
+
1. Any new log line or instrumentation event inside `AcquireLock` (and its mixins) or `LockSeriesPoC` must go through a visitor method; never call `logger.*` or `instrumenter.notify` directly in algorithm code.
|
|
46
|
+
2. Add a LogVisitor event for each new meaningful step or branch of the algorithm (start, decision, exit, success). Add an InstrVisitor event only for outcomes users would want to measure (obtained, held/released, completed); not for internal steps.
|
|
47
|
+
3. Fine-grained per-attempt logging belongs in `TryToLock::LogVisitor` and must be gated by `log_lock_try`.
|
|
48
|
+
4. For a new algorithm component (new mixin under `acquire_lock/`), create its own `<component>/log_visitor.rb` (and `instr_visitor.rb` if needed) and `require_relative` it from the component.
|
|
49
|
+
|
|
50
|
+
**How to write one**
|
|
51
|
+
5. Method: inside `class << self`, named exactly after the event (snake_case, `__` for phase/detail), full YARD with `@return [void]`, `@api private`, `@since <next version>`.
|
|
52
|
+
6. Parameters: port first, sampled flag second, extra gates next, then `lock_key`, then event data; for instrumentation keep the user `instrument` value last. Keep the list explicit (see `arguments.md`).
|
|
53
|
+
7. First line: `return unless <sampled>` (plus gate). Don't compute sampling inside the visitor and don't recompute it at the call site per event; reuse the operation's `log_sampled` / `instr_sampled`.
|
|
54
|
+
8. Logs: `logger.debug do ... end rescue nil` with message `"[redis_queued_locks.<event>] "` (`try_lock.` prefix for TryToLock steps) and `key => value` pairs, quoting strings, using the existing abbreviations (`acq_id`, `hst_id`, `acs_strat`, `queue_ttl`).
|
|
55
|
+
9. Instrumentation: `instrumenter.notify('redis_queued_locks.<event>', { ... }) rescue nil` with shorthand Symbol keys; reuse the standard payload keys (`lock_key`, `ttl`, `acq_id`, `hst_id`, `ts`, `acq_time`, `hold_time`, `instrument`). Event names are public API: never rename or remove one without being asked, and document new ones in the README/CHANGELOG.
|
|
56
|
+
10. Use the same method name in LogVisitor and InstrVisitor when both report the same event.
|
|
57
|
+
11. Add the method to the mirrored RBS visitor file (`RQL::loggerObj` / `RQL::instrObj`, `bool` sampled flag, `-> void`) and cover the new event in specs via the fake logger/notifier (assert on the `[redis_queued_locks.<event>]` prefix or event name).
|
|
58
|
+
|
|
59
|
+
## Recommendations (proposed, not yet project policy)
|
|
60
|
+
Apply to new or touched code; don't refactor existing code for these unless asked.
|
|
61
|
+
1. Give the release operations (`release_lock`, `release_all_locks`, `release_locks_of`) their own `InstrVisitor`/`LogVisitor` instead of inline `instrumenter.notify`, and either use or drop their currently unused `logger` param.
|
|
62
|
+
2. Use one error-swallowing style consistently (visitors use modifier `rescue nil`, release modules use `run_non_critical`). Both silently hide bugs such as a `NoMethodError` inside the log block, so consider reporting swallowed errors when the debugger is enabled.
|
|
63
|
+
3. Keep log and instrumentation event names in one registry constant (e.g. `RedisQueuedLocks::Instrument::EVENTS`) so README, specs and RBS can be checked against it.
|
|
64
|
+
4. Fix the RBS path mismatch: `sig/.../acquire_lock/yield_with_expire/log_visitor.rbs` mirrors `lib/.../acquire_lock/yield_expire/log_visitor.rb`; add RBS for `lock_series_poc` visitors when the PoC stabilizes.
|
data/.rubocop.rbs.yml
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# NOTE: RBS cops only: RBS signatures (sig/**/*.rbs) and inline RBS annotations in Ruby sources
|
|
2
|
+
# (lib/**/*.rb, RBSInline/* cops). Ruby cops are disabled here: Ruby sources are linted with .rubocop.yml;
|
|
3
|
+
inherit_gem:
|
|
4
|
+
armitage-rubocop:
|
|
5
|
+
- lib/rubocop.rbs.yml
|
|
6
|
+
|
|
7
|
+
AllCops:
|
|
8
|
+
TargetRubyVersion: 4.0
|
|
9
|
+
NewCops: enable
|
|
10
|
+
DisabledByDefault: true
|
|
11
|
+
UseProjectIndex: false
|
|
12
|
+
Include:
|
|
13
|
+
- sig/**/*.rbs
|
|
14
|
+
- lib/**/*.rb
|
|
15
|
+
# NOTE: files with a ruby shebang are picked up implicitly;
|
|
16
|
+
Exclude:
|
|
17
|
+
- bin/**/*
|
|
18
|
+
SuggestExtensions: false
|
|
19
|
+
|
|
20
|
+
# NOTE: `DisabledByDefault` is not enough for Ruby sources: inline `# rubocop:enable <Cop>`
|
|
21
|
+
# (and `# rubocop:enable all`) directives in lib/ re-enable Ruby cops for the rest of the file;
|
|
22
|
+
# so Ruby cop departments are excluded explicitly (they are covered by .rubocop.yml);
|
|
23
|
+
Bundler: { Exclude: ['**/*'] }
|
|
24
|
+
Gemspec: { Exclude: ['**/*'] }
|
|
25
|
+
Layout: { Exclude: ['**/*'] }
|
|
26
|
+
Lint: { Exclude: ['**/*'] }
|
|
27
|
+
Metrics: { Exclude: ['**/*'] }
|
|
28
|
+
Migration: { Exclude: ['**/*'] }
|
|
29
|
+
Naming: { Exclude: ['**/*'] }
|
|
30
|
+
Security: { Exclude: ['**/*'] }
|
|
31
|
+
Style: { Exclude: ['**/*'] }
|
data/.rubocop.yml
CHANGED
|
@@ -3,15 +3,15 @@ inherit_gem:
|
|
|
3
3
|
- lib/rubocop.general.yml
|
|
4
4
|
- lib/rubocop.rake.yml
|
|
5
5
|
- lib/rubocop.rspec.yml
|
|
6
|
-
- lib/rubocop.rbs.yml
|
|
7
6
|
|
|
7
|
+
# NOTE: Ruby sources only. RBS signatures (sig/**/*.rbs) are linted separately with
|
|
8
|
+
# .rubocop.rbs.yml: the Ruby cops (and the project index they use) must not see them;
|
|
8
9
|
AllCops:
|
|
9
|
-
TargetRubyVersion:
|
|
10
|
+
TargetRubyVersion: 4.0
|
|
10
11
|
NewCops: enable
|
|
11
12
|
Include:
|
|
12
13
|
- lib/**/*.rb
|
|
13
14
|
- spec/**/*.rb
|
|
14
|
-
- sig/**/*.rbs
|
|
15
15
|
- Gemfile
|
|
16
16
|
- Rakefile
|
|
17
17
|
- redis_queued_locks.gemspec
|
data/.ruby-version
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
4.0.7
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [1.17.0] - 2026-10-04
|
|
4
|
+
### Changed
|
|
5
|
+
- [**Breaking**] Minimal Ruby Version - **4.0** (previously - **3.3**):
|
|
6
|
+
- removed `3.3` and `3.4` from `CI`;
|
|
7
|
+
- Swarm: isolated swarm elements (`RedisQueuedLocks::Swarm::SwarmElement::Isolated`, `FlushZombies`) are reworked
|
|
8
|
+
to the Ruby 4 Ractor API: `Ractor.yield`/`Ractor#take` are replaced with `Ractor::Port`s (each element owns
|
|
9
|
+
its own pair of ports: the results port is created in the main ractor where the swarm supervisor lives and the
|
|
10
|
+
command port is created inside the element ractor), element ractor termination is tracked via `Ractor#monitor`;
|
|
11
|
+
- Swarm: isolated element ractor is terminated deterministically on `#try_kill!` / `#deswarm!`
|
|
12
|
+
(all ractor threads are killed and joined);
|
|
13
|
+
- Swarm: `RedisQueuedLocks::Swarm::SwarmElement::Isolated` subclasses implement `#spawn_swarm_element!(results_port)`
|
|
14
|
+
instead of `#swarm!`;
|
|
15
|
+
- Swarm: internal swarm element protocol (`Threaded` and `Isolated` command replies, isolated element handshake)
|
|
16
|
+
uses bare scalars/primitives instead of `{ ok:, result: }` wrappers (the wrapper is kept for the public API only);
|
|
17
|
+
### Fixed
|
|
18
|
+
- `RedisQueuedLocks::Client#zombies_info`: default `lock_scan_size` was taken from the
|
|
19
|
+
`swarm.flush_zombies.zombie_ttl` config instead of `swarm.flush_zombies.zombie_lock_scan_size`;
|
|
20
|
+
|
|
3
21
|
## [1.16.2] - 2026-02-06
|
|
4
22
|
### Fixed
|
|
5
23
|
- `lock_series` PoC:
|
data/CLAUDE.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
Ruby gem: distributed Redis locks with a per-lock FIFO acquisition queue. Ruby >= 4.0; only runtime dep is `redis-client ~> 0.20`.
|
|
4
|
+
Full details (config keys, key layout, events, errors, stack, CI): `.claude/project-overview.md`.
|
|
5
|
+
|
|
6
|
+
## Commands
|
|
7
|
+
- Tests (need a running local Redis): `bundle exec rake rspec`
|
|
8
|
+
- Lint: `bundle exec rake rubocop`
|
|
9
|
+
- Static types: `bundle exec rbs collection install && bundle exec rake steep:check`
|
|
10
|
+
|
|
11
|
+
## Architecture
|
|
12
|
+
- `client.rb`: public API facade; fills defaults from `config[...]`, delegates to `Acquirer::*`.
|
|
13
|
+
- `acquirer/*.rb`: one stateless module per operation (`class << self`), returns `{ ok:, result: }`; `!` client methods raise.
|
|
14
|
+
- `acquirer/acquire_lock.rb`: composed via `extend` mixins in `acquire_lock/` (try_to_lock, with_acq_timeout, delay_execution, dequeue_from_lock_queue, yield_expire) + `LogVisitor`/`InstrVisitor` event hooks.
|
|
15
|
+
- Locking: `ZADD NX` into `rql:lock_queue:<name>`, then WATCH/MULTI on `rql:lock:<name>`; Lua for TTL extension.
|
|
16
|
+
- Strategies: `access_strategy` `:queued`|`:random`; `conflict_strategy` `:wait_for_lock`|`:work_through`|`:extendable_work_through`|`:dead_locking`.
|
|
17
|
+
- `resource.rb`: all key names and acquirer/host IDs. `config.rb`: `setting`/`validate` DSL, read as `config['a.b']`.
|
|
18
|
+
- `swarm/`: zombie-lock cleanup. `Supervisor` thread revives `ProbeHosts` (`SwarmElement::Threaded`) and `FlushZombies` (`SwarmElement::Isolated`, Ractor talking via `Ractor::Port`s); each element runs a main-loop thread with its own Redis connection.
|
|
19
|
+
- `logging/`, `instrument/`: Void null-object defaults, percent samplers, ActiveSupport adapter.
|
|
20
|
+
- `sig/`: RBS mirror of `lib/`. `spec/redis_queued_locks_spec.rb`: single integration spec.
|
|
21
|
+
|
|
22
|
+
## Rules
|
|
23
|
+
Detailed, path-scoped rules in `.claude/rules/`: `logic.md` (lib, general), `acquirer.md` (acquirer modules, Redis access), `visitors.md` (log/instrumentation visitors), `arguments.md` (long keyword lists), `swarm.md` (swarm supervisor/elements, Ractor/Thread rules), `tests.md` (spec), `type-checking.md` (sig/Steep).
|
|
24
|
+
- Long explicit keyword/parameter lists are intentional (minimal allocations, signature-as-DSL): never introduce parameter objects, option structs or `**opts`; add new options as explicit keywords to every `Client` variant and forward them by name.
|
|
25
|
+
- Update the matching `sig/*.rbs` for every `lib/` change; keep Steep green.
|
|
26
|
+
- Keep `# frozen_string_literal: true` and YARD `@api`/`@since`/`@version` tags.
|
|
27
|
+
- `{ ok:, result: }` is for the public API only (`Client`, `Acquirer::*`, `Swarm` facade actions). Internal swarm element APIs (commands, replies, helpers) use bare scalars/primitives (see `swarm.md`).
|
|
28
|
+
- New operation: `Acquirer::*` module, thin `Client` method (+ `!` variant), RBS, spec.
|
|
29
|
+
- New option: `setting` (+ `validate`) in `config.rb`.
|
|
30
|
+
- Dev gems go in `Gemfile`, not the gemspec.
|
|
31
|
+
- Known debt: rspec-retry, disabled coverage minimum, runtime type-check CI uses `--failure-exit-code=0`.
|
data/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://github.com/0exp/redis_queued_locks/actions) [](https://github.com/0exp/redis_queued_locks/actions) [](https://github.com/0exp/redis_queued_locks/actions) [](https://github.com/0exp/redis_queued_locks/actions)
|
|
4
4
|
|
|
5
|
-
<a href="https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/">
|
|
5
|
+
**RedisQueuedLocks** (aka **RQL**, aka **RQL-lock**) - <a href="https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/">distributed locks</a> with "prioritized lock acquisition queue" capabilities based on the Redis Database.
|
|
6
6
|
|
|
7
7
|
Each lock request is put into the request queue (each lock is hosted by its own queue separately from other queues) and processed in order of their priority (FIFO). Each lock request lives some period of time (RTTL) (with requeue capabilities) which guarantees the request queue will never be stacked.
|
|
8
8
|
|
|
@@ -12,6 +12,10 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
|
|
|
12
12
|
|
|
13
13
|
---
|
|
14
14
|
|
|
15
|
+
#### For future features and updates check the [ROADMAP](#roadmap) section.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
15
19
|
## Table of Contents
|
|
16
20
|
|
|
17
21
|
- [Requirements](#requirements)
|
|
@@ -89,7 +93,7 @@ Provides flexible invocation flow, parametrized limits (lock request ttl, lock t
|
|
|
89
93
|
- Redis Version: `>= 7`, `~> 8.x`;
|
|
90
94
|
- Redis Protocol: `RESP3`;
|
|
91
95
|
- gem `redis-client`: `~> 0.20`;
|
|
92
|
-
- Ruby: `>=
|
|
96
|
+
- Ruby: `>= 4.0`;
|
|
93
97
|
|
|
94
98
|
---
|
|
95
99
|
|
|
@@ -371,6 +375,8 @@ end
|
|
|
371
375
|
|
|
372
376
|
- [lock](#lock---obtain-a-lock)
|
|
373
377
|
- [lock!](#lock---exeptional-lock-obtaining)
|
|
378
|
+
- [lock_series](#lock_series---poc-acquire-a-series-of-locks) (PoC)
|
|
379
|
+
- [lock_series!](#lock_series---poc-exceptional-lock_series) (PoC)
|
|
374
380
|
- [lock_info](#lock_info)
|
|
375
381
|
- [queue_info](#queue_info)
|
|
376
382
|
- [locked?](#locked)
|
|
@@ -2449,8 +2455,17 @@ Detalized event semantics and payload structure:
|
|
|
2449
2455
|
- `write` - waits - `read`;
|
|
2450
2456
|
- `write` - waits - `write`;
|
|
2451
2457
|
- **write** mode is a default behavior for all RQL locks;
|
|
2458
|
+
- `README`-section `Issues with other Libs` about issues in other libraries that can broke RQL features:
|
|
2459
|
+
- `Sentry`'s' bugs with Ractors:
|
|
2460
|
+
- `Sentry`-OpenTelemetry problems with its global mutexes and non-sharable objects that is not accessible from the ractors (sentry tyies to
|
|
2461
|
+
intercept all `redis-client` invocations managing their logic via global Mutex instance that brokes `swarm-mode` with `non-sharable` memory
|
|
2462
|
+
access errors, cuz RQL's swarm works using its own RedisClient-instnace, that fails isnide the `Sentry`'s patch: Sentry tries to use the same
|
|
2463
|
+
Mutex (instantiated inside the main Ractor) from the our non-main Ractor (swar-mode works under the non-main ractor) and this process fails\
|
|
2464
|
+
on non-sharable memory error);
|
|
2452
2465
|
- **Minor**:
|
|
2453
2466
|
- an ability to return all insturmentation metrics from the `lock` invocation (and after block `yield`ing);
|
|
2467
|
+
- think about a PG-like `stats`-data/table, that should be helpful for lock history acquirement analyzation :thinking: (suitable for cases when we want
|
|
2468
|
+
to check if the lock was acquired by some different parallel process durign the some period of time in the past or not);
|
|
2454
2469
|
- think about "thread priority" configuration :thinking:;
|
|
2455
2470
|
- add `hst_id` to all methods that works with queues info;
|
|
2456
2471
|
- try to return the `fiber object id` to the lock host identifier (we cant use fiber object id cuz `ObjectSpace` has no access to the fiber object space after the any ractor object initialization)
|
|
@@ -2472,7 +2487,7 @@ Detalized event semantics and payload structure:
|
|
|
2472
2487
|
- better code stylization (+ some refactorings);
|
|
2473
2488
|
- `RedisQueuedLocks::Acquirer::Try.try_to_lock` - detailed successful result analization;
|
|
2474
2489
|
- Support for LIFO strategy;
|
|
2475
|
-
-
|
|
2490
|
+
- specs refactoring with 100% test coverage (total specs rework);
|
|
2476
2491
|
- statistics with UI;
|
|
2477
2492
|
- JSON log formatter;
|
|
2478
2493
|
- **automatic** deadlock detection;
|
|
@@ -2485,6 +2500,7 @@ Detalized event semantics and payload structure:
|
|
|
2485
2500
|
- yardoc docs with CI check (full doc coverage check);
|
|
2486
2501
|
- split *exception* inheritance to the groups: `lock obtaining errors`, `block invocation errors`, `swarm errors`, and other groups (research possible groups):
|
|
2487
2502
|
- in some cases we need to intercept "Lock Obtaining Process Erros", in other cases: "Block Invocation Errors", and so on (in `Lock Serirs PoC` for example);
|
|
2503
|
+
- think about `#debounce` and `#throttle` mechanisms;
|
|
2488
2504
|
|
|
2489
2505
|
---
|
|
2490
2506
|
|
data/Rakefile
CHANGED
|
@@ -11,18 +11,47 @@ require 'rubocop-rake'
|
|
|
11
11
|
require 'rubocop-on-rbs'
|
|
12
12
|
require 'rubocop-thread_safety'
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
# NOTE: Ruby sources and RBS signatures are linted by separate runs with separate configs:
|
|
15
|
+
# Ruby cops (and the project index they use) must not see RBS signatures;
|
|
16
|
+
desc 'Run RuboCop for Ruby sources'
|
|
17
|
+
RuboCop::RakeTask.new('rubocop:ruby') do |t|
|
|
18
|
+
# NOTE: replace the default "Running RuboCop..." message (printed when verbose);
|
|
19
|
+
t.verbose = false
|
|
20
|
+
puts 'Running RuboCop (Ruby sources)...'
|
|
15
21
|
config_path = File.expand_path(File.join('.rubocop.yml'), __dir__)
|
|
16
22
|
t.options = [
|
|
17
23
|
'--config', config_path,
|
|
18
24
|
'--plugin', 'rubocop-rspec',
|
|
19
25
|
'--plugin', 'rubocop-performance',
|
|
20
26
|
'--plugin', 'rubocop-rake',
|
|
21
|
-
'--plugin', 'rubocop-on-rbs',
|
|
22
27
|
'--plugin', 'rubocop-thread_safety'
|
|
23
28
|
]
|
|
24
29
|
end
|
|
25
30
|
|
|
31
|
+
desc 'Run RuboCop for RBS signatures and inline RBS annotations'
|
|
32
|
+
RuboCop::RakeTask.new('rubocop:rbs') do |t|
|
|
33
|
+
# NOTE: replace the default "Running RuboCop..." message (printed when verbose);
|
|
34
|
+
t.verbose = false
|
|
35
|
+
puts 'Running RuboCop (RBS signatures and inline RBS annotations)...'
|
|
36
|
+
config_path = File.expand_path(File.join('.rubocop.rbs.yml'), __dir__)
|
|
37
|
+
t.options = [
|
|
38
|
+
'--config', config_path,
|
|
39
|
+
'--plugin', 'rubocop-on-rbs'
|
|
40
|
+
]
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
desc 'Run RuboCop for Ruby sources and RBS signatures'
|
|
44
|
+
task :rubocop do
|
|
45
|
+
# NOTE: run both linters even if the first one fails (a failed RuboCop task aborts);
|
|
46
|
+
failed = %w[rubocop:ruby rubocop:rbs].reject do |task_name|
|
|
47
|
+
Rake::Task[task_name].invoke
|
|
48
|
+
true
|
|
49
|
+
rescue SystemExit
|
|
50
|
+
false
|
|
51
|
+
end
|
|
52
|
+
abort("RuboCop failed: #{failed.join(', ')}") if failed.any?
|
|
53
|
+
end
|
|
54
|
+
|
|
26
55
|
RSpec::Core::RakeTask.new(:rspec)
|
|
27
56
|
Steep::RakeTask.new(:steep)
|
|
28
57
|
|
data/github_ci/.keep
ADDED
|
File without changes
|
|
@@ -12,7 +12,7 @@ module RedisQueuedLocks::Acquirer::AcquireLock::DelayExecution
|
|
|
12
12
|
# @api private
|
|
13
13
|
# @since 1.0.0
|
|
14
14
|
def delay_execution(retry_delay, retry_jitter)
|
|
15
|
-
delay = (retry_delay +
|
|
16
|
-
|
|
15
|
+
delay = (retry_delay + Kernel.rand(retry_jitter)).to_f / 1_000
|
|
16
|
+
Kernel.sleep(delay)
|
|
17
17
|
end
|
|
18
18
|
end
|
|
@@ -161,7 +161,6 @@ module RedisQueuedLocks::Acquirer::AcquireLock::TryToLock
|
|
|
161
161
|
)
|
|
162
162
|
inter_result = :extendable_conflict_work_through
|
|
163
163
|
|
|
164
|
-
# @type var sp_conflict_status: Symbol
|
|
165
164
|
# @type var spc_processed_timestamp: Float
|
|
166
165
|
LogVisitor.reentrant_lock__extend_and_work_through(
|
|
167
166
|
logger, log_sampled, log_lock_try, lock_key,
|
|
@@ -188,7 +187,6 @@ module RedisQueuedLocks::Acquirer::AcquireLock::TryToLock
|
|
|
188
187
|
'l_spc_ts', spc_processed_timestamp = Time.now.to_f
|
|
189
188
|
)
|
|
190
189
|
|
|
191
|
-
# @type var sp_conflict_status: Symbol
|
|
192
190
|
# @type var spc_processed_timestamp: Float
|
|
193
191
|
LogVisitor.reentrant_lock__work_through(
|
|
194
192
|
logger, log_sampled, log_lock_try, lock_key,
|
|
@@ -200,7 +198,6 @@ module RedisQueuedLocks::Acquirer::AcquireLock::TryToLock
|
|
|
200
198
|
inter_result = :conflict_dead_lock
|
|
201
199
|
spc_processed_timestamp = Time.now.to_f
|
|
202
200
|
|
|
203
|
-
# @type var sp_conflict_status: Symbol
|
|
204
201
|
# @type var spc_processed_timestamp: Float
|
|
205
202
|
LogVisitor.single_process_lock_conflict__dead_lock(
|
|
206
203
|
logger, log_sampled, log_lock_try, lock_key,
|
|
@@ -242,7 +239,7 @@ module RedisQueuedLocks::Acquirer::AcquireLock::TryToLock
|
|
|
242
239
|
)
|
|
243
240
|
|
|
244
241
|
# Step 3: get the actual acquirer waiting in the queue
|
|
245
|
-
waiting_acquirer = Array(rconn.call('ZRANGE', lock_key_queue, '0', '0')).first
|
|
242
|
+
waiting_acquirer = Array(rconn.call('ZRANGE', lock_key_queue, '0', '0')).first #: String?
|
|
246
243
|
|
|
247
244
|
LogVisitor.get_first_from_queue(
|
|
248
245
|
logger, log_sampled, log_lock_try, lock_key,
|
|
@@ -7,7 +7,6 @@
|
|
|
7
7
|
# rubocop:disable Metrics/MethodLength
|
|
8
8
|
# rubocop:disable Metrics/ClassLength
|
|
9
9
|
# rubocop:disable Metrics/BlockNesting
|
|
10
|
-
# rubocop:disable Style/IfInsideElse
|
|
11
10
|
module RedisQueuedLocks::Acquirer::AcquireLock
|
|
12
11
|
require_relative 'acquire_lock/log_visitor'
|
|
13
12
|
require_relative 'acquire_lock/instr_visitor'
|
|
@@ -563,9 +562,7 @@ module RedisQueuedLocks::Acquirer::AcquireLock
|
|
|
563
562
|
end
|
|
564
563
|
end
|
|
565
564
|
else
|
|
566
|
-
# rubocop:disable Layout/LineLength
|
|
567
565
|
{ ok: true, result: acq_process[:lock_info] } #: { ok: bool, result: Hash[Symbol,untyped] }
|
|
568
|
-
# rubocop:enable Layout/LineLength
|
|
569
566
|
end
|
|
570
567
|
else
|
|
571
568
|
if acq_process[:result] != :retry_limit_reached &&
|
|
@@ -588,4 +585,3 @@ end
|
|
|
588
585
|
# rubocop:enable Metrics/MethodLength
|
|
589
586
|
# rubocop:enable Metrics/ClassLength
|
|
590
587
|
# rubocop:enable Metrics/BlockNesting
|
|
591
|
-
# rubocop:enable Style/IfInsideElse
|
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# NOTE: Lock Series PoC
|
|
4
|
-
# steep:ignore
|
|
5
4
|
# @api private
|
|
6
5
|
# @since 1.16.1
|
|
7
6
|
module RedisQueuedLocks::Acquirer::LockSeriesPoC::InstrVisitor # steep:ignore
|
|
@@ -48,4 +47,3 @@ module RedisQueuedLocks::Acquirer::LockSeriesPoC::InstrVisitor # steep:ignore
|
|
|
48
47
|
end
|
|
49
48
|
end
|
|
50
49
|
end
|
|
51
|
-
# steep:ignore
|
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# NOTE: Lock Series PoC
|
|
4
|
-
# steep:ignore
|
|
5
4
|
# rubocop:disable all
|
|
6
5
|
# @api private
|
|
7
6
|
# @since 1.16.0
|
|
@@ -324,5 +323,4 @@ module RedisQueuedLocks::Acquirer::LockSeriesPoC # steep:ignore
|
|
|
324
323
|
end
|
|
325
324
|
end
|
|
326
325
|
end
|
|
327
|
-
# steep:ignore
|
|
328
326
|
# rubocop:enable all
|