redis_queued_locks 0.0.35 → 0.0.36

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8e16184437150045fd98daf2ab2aaa328e0e7b0bae2a3bb513db25ea7eb535d6
4
- data.tar.gz: 1965028261e953591dc707baea43801ed0ec506a85bd4de67c0a4e17cda52820
3
+ metadata.gz: 32c3b5078ec27e7203f41fc7537a75b336d5ff25c5f6a62b27c767daf16c912f
4
+ data.tar.gz: a2ebddba52834ff1bbbbd04570f8573185f3529b3afa9c88bb7b349a3fcaf1d2
5
5
  SHA512:
6
- metadata.gz: e0cff23e13e47de8bd207597c65e0e036bbf971caad6dbd062628ca8e1975467d52834cc83787a5b4f41b3b88310eb88fe17aba93b2776687d0e5af9913d44b4
7
- data.tar.gz: a963e804304ff4884669244f07a41061c7cc7ed56ad72f702bc41a42e7e4f8985d4f8b27d330a62afdce652123343cddae390a75d8bd8cdbab354ac54f4dd118
6
+ metadata.gz: fa8f84f8059e55209c5ec354705d6ecf3dbf4a461b3971dd300ae67301e5e4063ba5382548074279a5603226fed90e72c84b8be014b15277bbd154668236e146
7
+ data.tar.gz: e2b05110dfeedbdc9d58495eaa9133156615dabe314c5f1f5226036b379f457d74998791ff60b7e91fdca7095c6f22c09f8b585f3aa0f0a71123b428b836a754
data/CHANGELOG.md CHANGED
@@ -1,12 +1,21 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.0.36] - 2024-03-28
4
+ ### Added
5
+ - Requirements:
6
+ - redis version: `>= 7.x`;
7
+ - redis protocol: `RESP3`;
8
+ - Additional debugging methods:
9
+ - `#locks_info` (or `#locks(with_info: true)`) - get obtained locks with their info;
10
+ - `#queus_info` (or `#queues(with_info: true`) - get active lock queues with their info;
11
+
3
12
  ## [0.0.35] - 2024-03-26
4
13
  ### Changed
5
- - The random unique client instance identifier now uses 16-byte strings instead of 10-btes in order to prevent potential collisions;
14
+ - The random unique client instance identifier now uses 16-byte strings instead of 10-bytes in order to prevent potential collisions;
6
15
 
7
16
  ## [0.0.34] - 2024-03-26
8
17
  ### Changed
9
- - Removing the acquirer from the request queue during a lock obtaining logic now using more proper and accurate `ZREM` instead of `ZPOPMIN` for this;
18
+ - Removing the acquirer from the request queue during the lock obtaining logic is using more proper and accurate `ZREM` now instead of `ZPOPMIN`;
10
19
 
11
20
  ## [0.0.33] - 2024-03-26
12
21
  ### Added
data/README.md CHANGED
@@ -10,6 +10,7 @@ Each lock request is put into the request queue (each lock is hosted by it's own
10
10
 
11
11
  ## Table of Contents
12
12
 
13
+ - [Requirements](#requirements)
13
14
  - [Algorithm](#algorithm)
14
15
  - [Installation](#installation)
15
16
  - [Setup](#setup)
@@ -27,6 +28,8 @@ Each lock request is put into the request queue (each lock is hosted by it's own
27
28
  - [locks](#locks---get-list-of-obtained-locks)
28
29
  - [queues](#queues---get-list-of-lock-request-queues)
29
30
  - [keys](#keys---get-list-of-taken-locks-and-queues)
31
+ - [locks_info](#locks_info---get-list-of-locks-with-their-info)
32
+ - [queues_info](#queues_info---get-list-of-queues-with-their-info)
30
33
  - [Instrumentation](#instrumentation)
31
34
  - [Instrumentation Events](#instrumentation-events)
32
35
  - [Roadmap](#roadmap)
@@ -36,6 +39,13 @@ Each lock request is put into the request queue (each lock is hosted by it's own
36
39
 
37
40
  ---
38
41
 
42
+ ### Requirements
43
+
44
+ - Redis Version: `~> 7.x`;
45
+ - Redis Protocol: `RESP3`;
46
+
47
+ ---
48
+
39
49
  ### Algorithm
40
50
 
41
51
  > Each lock request is put into the request queue (each lock is hosted by it's own queue separately from other queues) and processed in order of their priority (FIFO). Each lock request lives some period of time (RTTL) which guarantees that the request queue will never be stacked.
@@ -135,6 +145,7 @@ clinet = RedisQueuedLocks::Client.new(redis_client) do |config|
135
145
  # that have identical process_id/thread_id/fiber_id/ractor_id (identivcal acquier ids);
136
146
  # - it is calculated once per `RedisQueudLocks::Client` instance;
137
147
  # - expects the proc object;
148
+ # - `SecureRandom.hex(8)` by default;
138
149
  config.uniq_identifier = -> { RedisQueuedLocks::Resource.calc_uniq_identity }
139
150
 
140
151
  # (default: RedisQueuedLocks::Logging::VoidLogger)
@@ -181,6 +192,8 @@ end
181
192
  - [locks](#locks---get-list-of-obtained-locks)
182
193
  - [queues](#queues---get-list-of-lock-request-queues)
183
194
  - [keys](#keys---get-list-of-taken-locks-and-queues)
195
+ - [locks_info](#locks_info---get-list-of-locks-with-their-info)
196
+ - [queues_info](#queues_info---get-list-of-queues-with-their-info)
184
197
 
185
198
  ---
186
199
 
@@ -487,8 +500,12 @@ Return:
487
500
  #### #locks - get list of obtained locks
488
501
 
489
502
  - uses redis `SCAN` under the hood;
490
- - accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
491
- - returns `Set<String>`
503
+ - accepts:
504
+ - `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
505
+ - `:with_info` - `Boolean` - `false` by default (for details see [#locks_info](#locks_info---get-list-of-locks-with-their-info));
506
+ - returns:
507
+ - `Set<String>` (for `with_info: false`);
508
+ - `Set<Hash<Symbol,Any>>` (for `with_info: true`). See `#locks_info` for details;
492
509
 
493
510
  ```ruby
494
511
  rql.locks # or rql.locks(scan_size: 123)
@@ -513,8 +530,12 @@ rql.locks # or rql.locks(scan_size: 123)
513
530
  #### #queues - get list of lock request queues
514
531
 
515
532
  - uses redis `SCAN` under the hood;
516
- - accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
517
- - returns `Set<String>`
533
+ - accepts
534
+ - `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
535
+ - `:with_info` - `Boolean` - `false` by default (for details see [queues_info](#queues_info---get-list-of-queues-with-their-info));
536
+ - returns:
537
+ - `Set<String>` (for `with_info: false`);
538
+ - `Set<Hash<Symbol,Any>>` (for `with_info: true`). See `#locks_info` for details;
518
539
 
519
540
  ```ruby
520
541
  rql.queues # or rql.queues(scan_size: 123)
@@ -539,8 +560,9 @@ rql.queues # or rql.queues(scan_size: 123)
539
560
  #### #keys - get list of taken locks and queues
540
561
 
541
562
  - uses redis `SCAN` under the hood;
542
- - accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
543
- - returns `Set<String>`
563
+ - accepts:
564
+ `:scan_size` - `Integer` - (`config[:key_extraction_batch_size]` by default);
565
+ - returns: `Set<String>`
544
566
 
545
567
  ```ruby
546
568
  rql.keys # or rql.keys(scan_size: 123)
@@ -566,6 +588,66 @@ rql.keys # or rql.keys(scan_size: 123)
566
588
 
567
589
  ---
568
590
 
591
+ #### #locks_info - get list of locks with their info
592
+
593
+ - uses redis `SCAN` under the hod;
594
+ - accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
595
+ - returns `Set<Hash<Symbol,Any>>` (see [#lock_info](#lock_info) and examples below for details).
596
+ - contained data: `{ lock: String, status: Symbol, info: Hash<String,Any> }`;
597
+ - `:lock` - `String` - lock key in Redis;
598
+ - `:status` - `Symbol`- `:released` or `:alive`. The lock may become relased durign the lock info extractio process;
599
+ - `:info` - `Hash<String,Any>` - lock data stored in the lock key in Redis. See [#lock_info](#lock_info) for details;
600
+
601
+ ```ruby
602
+ rql.locks_info # or rql.locks_info(scan_size: 123)
603
+
604
+ # =>
605
+ => #<Set:
606
+ {{:lock=>"rql:lock:some-lock-123",
607
+ :status=>:alive,
608
+ :info=>{
609
+ "acq_id"=>"rql:acq:41478/4320/4340/4360/848818f09d8c3420",
610
+ "ts"=>1711607112.670343,
611
+ "ini_ttl"=>15000,
612
+ "rem_ttl"=>13998}},
613
+ {:lock=>"rql:lock:some-lock-456",
614
+ :status=>:released,
615
+ :info=>{
616
+ "acq_id"=>"rql:acq:41478/4500/4520/4360/848818f09d8c3420",
617
+ "ts"=>1711607112.67106,
618
+ "ini_ttl"=>15000,
619
+ "rem_ttl"=>13999}}}>
620
+ ```
621
+
622
+ ---
623
+
624
+ #### #queues_info - get list of queues with their info
625
+
626
+ - uses redis `SCAN` under the hod;
627
+ - accepts `scan_size:`/`Integer` option (`config[:key_extraction_batch_size]` by default);
628
+ - returns `Set<Hash<Symbol,Any>>` (see [#queue_info](#queue_info) and examples below for details).
629
+ - contained data: `{ queue: String, contains: Array<Hash<String,Any>> }`
630
+ - `:queue` - `String` - lock key queue in Redis;
631
+ - `:contains` - `Array<Hash<String,Any>>` - lock requests in the que with their acquier id and score.
632
+
633
+ ```ruby
634
+ rql.queues_info # or rql.qeuues_info(scan_size: 123)
635
+
636
+ => #<Set:
637
+ {{:queue=>"rql:lock_queue:some-lock-123",
638
+ :contains=>
639
+ [{"acq_id"=>"rql:acq:38529/4500/4520/4360/66093702f24a3129", "score"=>1711606640.540842},
640
+ {"acq_id"=>"rql:acq:38529/4580/4600/4360/66093702f24a3129", "score"=>1711606640.540906},
641
+ {"acq_id"=>"rql:acq:38529/4620/4640/4360/66093702f24a3129", "score"=>1711606640.5409632}]},
642
+ {:queue=>"rql:lock_queue:some-lock-456",
643
+ :contains=>
644
+ [{"acq_id"=>"rql:acq:38529/4380/4400/4360/66093702f24a3129", "score"=>1711606640.540722},
645
+ {"acq_id"=>"rql:acq:38529/4420/4440/4360/66093702f24a3129", "score"=>1711606640.5407748},
646
+ {"acq_id"=>"rql:acq:38529/4460/4480/4360/66093702f24a3129", "score"=>1711606640.540808}]}}>
647
+ ```
648
+
649
+ ---
650
+
569
651
  ## Instrumentation
570
652
 
571
653
  An instrumentation layer is incapsulated in `instrumenter` object stored in [config](#configuration) (`RedisQueuedLocks::Client#config[:instrumenter]`).
@@ -647,8 +729,6 @@ Detalized event semantics and payload structure:
647
729
  - better code stylization and interesting refactorings;
648
730
  - dead queue keys cleanup (empty queues);
649
731
  - statistics with UI;
650
- - support for `Dragonfly` DB backend;
651
- - support for `Garnet` DB backend;
652
732
 
653
733
  ---
654
734
 
@@ -221,7 +221,7 @@ module RedisQueuedLocks::Acquier::AcquireLock::TryToLock
221
221
  transact.call('ZREM', lock_key_queue, acquier_id)
222
222
 
223
223
  RedisQueuedLocks.debug(
224
- 'Step №4: Забираем наш текущий процесс из очереди. [ZPOPMIN]'
224
+ 'Step №4: Забираем наш текущий процесс из очереди. [ZREM]'
225
225
  )
226
226
 
227
227
  # rubocop:disable Layout/LineLength
@@ -278,8 +278,8 @@ module RedisQueuedLocks::Acquier::AcquireLock::TryToLock
278
278
  # => (!) analyze the command result and do actions with the depending on it;
279
279
  # => (*) at this moment we accept that all comamnds are completed successfully;
280
280
  # => (!) need to analyze:
281
- # 1. zpopmin should return our process (array with <acq_id> and <score>)
282
- # 2. hset should return 2 (lock key is added to the redis as a hashmap with 2 fields)
281
+ # 1. zrem shoud return ? (?)
282
+ # 2. hset should return 3 as minimum (lock key is added to the redis as a hashmap with 3 fields as minimum)
283
283
  # 3. pexpire should return 1 (expiration time is successfully applied)
284
284
 
285
285
  # Step 7.d: locked! :) let's go! => successfully acquired
@@ -3,14 +3,34 @@
3
3
  # @api private
4
4
  # @since 0.1.0
5
5
  module RedisQueuedLocks::Acquier::Locks
6
+ # @return [Hash]
7
+ #
8
+ # @api private
9
+ # @since 0.1.0
10
+ NO_LOCK_INFO = {}.freeze
11
+
6
12
  class << self
7
13
  # @param redis_client [RedisClient]
8
14
  # @option scan_size [Integer]
15
+ # @option with_info [Boolean]
16
+ # @return [Set<String>,Set<Hash<Symbol,Any>>]
17
+ #
18
+ # @api private
19
+ # @since 0.1.0
20
+ def locks(redis_client, scan_size:, with_info:)
21
+ lock_keys = scan_locks(redis_client, scan_size)
22
+ with_info ? extract_locks_info(redis_client, lock_keys) : lock_keys
23
+ end
24
+
25
+ private
26
+
27
+ # @param redis_client [RedisClient]
28
+ # @param scan_size [Integer]
9
29
  # @return [Set<String>]
10
30
  #
11
31
  # @api private
12
32
  # @since 0.1.0
13
- def locks(redis_client, scan_size:)
33
+ def scan_locks(redis_client, scan_size)
14
34
  Set.new.tap do |lock_keys|
15
35
  redis_client.scan(
16
36
  'MATCH',
@@ -22,5 +42,54 @@ module RedisQueuedLocks::Acquier::Locks
22
42
  end
23
43
  end
24
44
  end
45
+
46
+ # @param redis_client [RedisClient]
47
+ # @param lock_keys [Set<String>]
48
+ # @return [Set<Hash<Symbol,Any>>]
49
+ #
50
+ # @api private
51
+ # @since 0.1.0
52
+ # rubocop:disable Metrics/MethodLength
53
+ def extract_locks_info(redis_client, lock_keys)
54
+ # TODO: refactor with RedisQueuedLocks::Acquier::LockInfo
55
+ Set.new.tap do |seeded_locks|
56
+ # Step X: iterate each lock and extract their info
57
+ lock_keys.each do |lock_key|
58
+ # Step 1: extract lock info from redis
59
+ lock_info = redis_client.multi(watch: [lock_key]) do |transact|
60
+ transact.call('HGETALL', lock_key)
61
+ transact.call('PTTL', lock_key)
62
+ end.yield_self do |result| # Step 2: format the result
63
+ # Step 2.X: lock is released
64
+ if result == nil
65
+ {}
66
+ else
67
+ hget_cmd_res = result[0] # NOTE: HGETALL result (hash)
68
+ pttl_cmd_res = result[1] # NOTE: PTTL result (integer)
69
+
70
+ # Step 2.Y: lock is released
71
+ if hget_cmd_res == {} || pttl_cmd_res == -2 # NOTE: key does not exist
72
+ {}
73
+ else
74
+ # Step 2.Z: lock is alive => format received info + add additional rem_ttl info
75
+ hget_cmd_res.tap do |lock_data|
76
+ lock_data['ts'] = Float(lock_data['ts'])
77
+ lock_data['ini_ttl'] = Integer(lock_data['ini_ttl'])
78
+ lock_data['rem_ttl'] = ((pttl_cmd_res == -1) ? Infinity : pttl_cmd_res)
79
+ end
80
+ end
81
+ end
82
+ end
83
+
84
+ # Step 3: push the lock info to the result store
85
+ seeded_locks << {
86
+ lock: lock_key,
87
+ status: (lock_info.empty? ? :released : :alive),
88
+ info: lock_info
89
+ }
90
+ end
91
+ end
92
+ end
93
+ # rubocop:enable Metrics/MethodLength
25
94
  end
26
95
  end
@@ -4,13 +4,27 @@
4
4
  # @since 0.1.0
5
5
  module RedisQueuedLocks::Acquier::Queues
6
6
  class << self
7
+ # @param redis_client [RedisClient]
8
+ # @option scan_size [Integer]
9
+ # @option with_info [Boolean]
10
+ # @return [Set<String>,Set<Hash<Symbol,Any>>]
11
+ #
12
+ # @api private
13
+ # @since 0.1.0
14
+ def queues(redis_client, scan_size:, with_info:)
15
+ lock_queues = scan_queues(redis_client, scan_size)
16
+ with_info ? extract_queues_info(redis_client, lock_queues) : lock_queues
17
+ end
18
+
19
+ private
20
+
7
21
  # @param redis_client [RedisClient]
8
22
  # @param scan_size [Integer]
9
23
  # @return [Set<String>]
10
24
  #
11
25
  # @api private
12
26
  # @since 0.1.0
13
- def queues(redis_client, scan_size:)
27
+ def scan_queues(redis_client, scan_size)
14
28
  Set.new.tap do |lock_queues|
15
29
  redis_client.scan(
16
30
  'MATCH',
@@ -22,5 +36,41 @@ module RedisQueuedLocks::Acquier::Queues
22
36
  end
23
37
  end
24
38
  end
39
+
40
+ # @param redis_client [RedisClient]
41
+ # @param lock_queus [Set<String>]
42
+ # @return [Set<Hash<Symbol,Any>>]
43
+ #
44
+ # @api private
45
+ # @since 0.1.0
46
+ def extract_queues_info(redis_client, lock_queues)
47
+ # TODO: refactor with RedisQueuedLocks::Acquier::QueueInfo
48
+ Set.new.tap do |seeded_queues|
49
+ # Step X: iterate over each lock queue and extract their info
50
+ lock_queues.each do |lock_queue|
51
+ # Step 1: extract lock queue info from reids
52
+ queue_info = redis_client.pipelined do |pipeline|
53
+ pipeline.call('EXISTS', lock_queue)
54
+ pipeline.call('ZRANGE', lock_queue, '0', '-1', 'WITHSCORES')
55
+ end.yield_self do |result| # Step 2: format the result
56
+ exists_cmd_res = result[0]
57
+ zrange_cmd_res = result[1]
58
+
59
+ if exists_cmd_res == 1 # Step 2.X: lock queue existed during the piepline invocation
60
+ zrange_cmd_res.map { |val| { 'acq_id' => val[0], 'score' => val[1] } }
61
+ else
62
+ # Step 2.Y: lock queue did not exist during the pipeline invocation
63
+ []
64
+ end
65
+ end
66
+
67
+ # Step 3: push the lock queue info to the result store
68
+ seeded_queues << {
69
+ queue: lock_queue,
70
+ contains: queue_info
71
+ }
72
+ end
73
+ end
74
+ end
25
75
  end
26
76
  end
@@ -280,21 +280,68 @@ class RedisQueuedLocks::Client
280
280
  end
281
281
 
282
282
  # @option scan_size [Integer]
283
- # @return [Set<String>]
283
+ # The batch of scanned keys for Redis'es SCAN command.
284
+ # @option with_info [Boolean]
285
+ # Extract lock info or not. If you want to extract a lock info too you have to consider
286
+ # that during info extration the lock key may expire. The keys extraction (SCAN) without
287
+ # any info extraction is doing first.
288
+ # Possible options:
289
+ # - `true` => returns a set of hashes that represents the lock state: <lock:, status:, info:>
290
+ # - :lock (String) - the lock key in Redis database;
291
+ # - :status (Symbol) - lock key state in redis. possible values:
292
+ # - :released - the lock is expired/released during the info extraction;
293
+ # - :alive - the lock still obtained;
294
+ # - :info (Hash) - see #lock_info method for detals;
295
+ # - `false` => returns a set of strings that represents an active locks
296
+ # at the moment of Redis'es SCAN;
297
+ # @return [Set<String>,Set<Hash<Symbol,Any>>]
284
298
  #
285
299
  # @api public
286
300
  # @since 0.1.0
287
- def locks(scan_size: config[:key_extraction_batch_size])
288
- RedisQueuedLocks::Acquier::Locks.locks(redis_client, scan_size:)
301
+ def locks(scan_size: config[:key_extraction_batch_size], with_info: false)
302
+ RedisQueuedLocks::Acquier::Locks.locks(redis_client, scan_size:, with_info:)
289
303
  end
290
304
 
305
+ # Extracts lock keys with their info. See #locks(with_info: true) for details.
306
+ #
291
307
  # @option scan_size [Integer]
292
- # @return [Set<String>]
308
+ # @return [Set<Hash<String,Any>>]
309
+ #
310
+ # @api public
311
+ # @since 0.1.0
312
+ def locks_info(scan_size: config[:key_extraction_batch_size])
313
+ locks(scan_size:, with_info: true)
314
+ end
315
+
316
+ # @option scan_size [Integer]
317
+ # The batch of scanned keys for Redis'es SCAN command.
318
+ # @option with_info [Boolean]
319
+ # Extract lock qeue info or not. If you want to extract a lock queue info too you have to
320
+ # consider that during info extration the lock queue may become empty. The queue key extraction
321
+ # (SCAN) without queue info extraction is doing first.
322
+ # Possible options:
323
+ # - `true` => returns a set of hashes that represents the queue state: <queue:, containing:>
324
+ # - :queue (String) - the lock queue key in Redis database;
325
+ # - :contains (Array<Hash<String,Any>>) - queue state in redis. see #queue_info for details;
326
+ # - `false` => returns a set of strings that represents an active queues
327
+ # at the moment of Redis'es SCAN;
328
+ # @return [Set<String>,String<Hash<Symbol,Any>>]
329
+ #
330
+ # @api public
331
+ # @since 0.1.0
332
+ def queues(scan_size: config[:key_extraction_batch_size], with_info: false)
333
+ RedisQueuedLocks::Acquier::Queues.queues(redis_client, scan_size:, with_info:)
334
+ end
335
+
336
+ # Extracts lock queues with their info. See #queues(with_info: true) for details.
337
+ #
338
+ # @option scan_size [Integer]
339
+ # @return [Set<Hash<Symbol,Any>>]
293
340
  #
294
341
  # @api public
295
342
  # @since 0.1.0
296
- def queues(scan_size: config[:key_extraction_batch_size])
297
- RedisQueuedLocks::Acquier::Queues.queues(redis_client, scan_size:)
343
+ def queues_info(scan_size: config[:key_extraction_batch_size])
344
+ queues(scan_size:, with_info: true)
298
345
  end
299
346
 
300
347
  # @option scan_size [Integer]
@@ -5,6 +5,6 @@ module RedisQueuedLocks
5
5
  #
6
6
  # @api public
7
7
  # @since 0.0.1
8
- # @version 0.0.35
9
- VERSION = '0.0.35'
8
+ # @version 0.0.36
9
+ VERSION = '0.0.36'
10
10
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: redis_queued_locks
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.35
4
+ version: 0.0.36
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rustam Ibragimov
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2024-03-26 00:00:00.000000000 Z
11
+ date: 2024-03-28 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: redis-client