de-dupe 0.0.1 → 0.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +21 -0
- data/lib/de_dupe/dataset.rb +17 -0
- data/lib/de_dupe/version.rb +1 -1
- metadata +1 -8
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 5d57f4361b02f7ba63b32d39529770fa09f73fc07fbaf957104865e488f7b99b
|
|
4
|
+
data.tar.gz: 8adee9fd090575a21a093786c68914590e66728b390d11959213d877cde93e6c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d119da0edcc25fb75d547b68f60502bd6f3c2ca93c2a4596eeb33c1be25612bce9e8ddb9024816d2c9f9eef4480056398d5d1c1c85a5fba25ead5a2a487e3782
|
|
7
|
+
data.tar.gz: 812f18dbe25170e65f475d320dc45368dd19b42d86075d534fd316e74118367de41ac629778f0ec7d81ee8268c3ccbea404703fc6bf2edf2c7eca6ffe482bb38
|
data/README.md
CHANGED
|
@@ -134,6 +134,10 @@ end
|
|
|
134
134
|
locked = dataset.locked_members("user-1", "user-2", "user-3")
|
|
135
135
|
# => ["user-1", "user-2", "user-3"]
|
|
136
136
|
|
|
137
|
+
# Check the size of the dataset
|
|
138
|
+
size = dataset.size
|
|
139
|
+
# => 3
|
|
140
|
+
|
|
137
141
|
# Check which IDs are unlocked
|
|
138
142
|
unlocked = dataset.unlocked_members("user-1", "user-2", "user-3")
|
|
139
143
|
# => []
|
|
@@ -151,6 +155,11 @@ dataset.flush
|
|
|
151
155
|
|
|
152
156
|
# Manually clean up expired entries
|
|
153
157
|
dataset.flush_expired_members
|
|
158
|
+
|
|
159
|
+
# Iterate over all members
|
|
160
|
+
dataset.members do |member_id|
|
|
161
|
+
puts "Processing: #{member_id}"
|
|
162
|
+
end
|
|
154
163
|
```
|
|
155
164
|
|
|
156
165
|
## API Reference
|
|
@@ -224,6 +233,10 @@ Return array of IDs that are currently locked (not expired).
|
|
|
224
233
|
|
|
225
234
|
Return array of IDs that are unlocked (don't exist or expired).
|
|
226
235
|
|
|
236
|
+
#### `size`
|
|
237
|
+
|
|
238
|
+
Return the number of entries in the dataset.
|
|
239
|
+
|
|
227
240
|
#### `flush`
|
|
228
241
|
|
|
229
242
|
Remove all entries from the dataset.
|
|
@@ -232,6 +245,14 @@ Remove all entries from the dataset.
|
|
|
232
245
|
|
|
233
246
|
Remove all expired entries from the dataset (automatically called during `acquire`).
|
|
234
247
|
|
|
248
|
+
#### `members(&block)`
|
|
249
|
+
|
|
250
|
+
Iterate over all non-expired members in the dataset. Automatically flushes expired members before iterating.
|
|
251
|
+
|
|
252
|
+
- Yields each member to the block
|
|
253
|
+
- Returns an enumerator if no block is given
|
|
254
|
+
- Only includes non-expired members
|
|
255
|
+
|
|
235
256
|
## Examples
|
|
236
257
|
|
|
237
258
|
### Preventing Duplicate Job Execution
|
data/lib/de_dupe/dataset.rb
CHANGED
|
@@ -74,6 +74,23 @@ module DeDupe
|
|
|
74
74
|
end
|
|
75
75
|
end
|
|
76
76
|
|
|
77
|
+
def size(flush_expired: true)
|
|
78
|
+
flush_expired_members if flush_expired
|
|
79
|
+
redis_pool.with do |conn|
|
|
80
|
+
conn.zcard(lock_key)
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def members(&block)
|
|
85
|
+
flush_expired_members
|
|
86
|
+
members = redis_pool.with do |conn|
|
|
87
|
+
conn.zrange(lock_key, 0, -1, with_scores: false)
|
|
88
|
+
end
|
|
89
|
+
return members.to_enum unless block
|
|
90
|
+
|
|
91
|
+
members.each(&block)
|
|
92
|
+
end
|
|
93
|
+
|
|
77
94
|
def flush
|
|
78
95
|
redis_pool.with do |conn|
|
|
79
96
|
conn.del(lock_key) > 0
|
data/lib/de_dupe/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: de-dupe
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.0.
|
|
4
|
+
version: 0.0.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Marcos G. Zimmermann
|
|
@@ -56,13 +56,6 @@ description: |
|
|
|
56
56
|
DeDupe is a Ruby gem for distributed deduplication and locking using Redis Sorted Sets.
|
|
57
57
|
It provides a simple and efficient way to prevent duplicate execution of tasks across
|
|
58
58
|
multiple processes or servers, with automatic TTL-based expiration and cleanup.
|
|
59
|
-
|
|
60
|
-
Features:
|
|
61
|
-
- Distributed locking across multiple processes/servers
|
|
62
|
-
- TTL-based automatic expiration
|
|
63
|
-
- Automatic cleanup of expired entries
|
|
64
|
-
- Multiple ID management within namespaces
|
|
65
|
-
- Simple API with block support
|
|
66
59
|
email:
|
|
67
60
|
- mgzmaster@gmail.com
|
|
68
61
|
executables: []
|