me-redis 0.1.0 → 0.1.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: a72bf4245ee1c3efe20256d83c4999e0002d78ea
4
- data.tar.gz: 02acdfc1108cc6a3e06b04885b901fa1b63e7c95
2
+ SHA256:
3
+ metadata.gz: 42d0534ff1e6462c1ada7d35f277b4eae482285d19df0678542cc9e21f78d5f5
4
+ data.tar.gz: 56b06010bba70a0694f1ecd6d6b90147dff50af4281bc6a4993b9985ad6ccf2b
5
5
  SHA512:
6
- metadata.gz: cc66f942a6f49e9c86678ee7f92fdd2dbac60edd77df7b2a7feecde527769a0629da6f06b2d52bd7f5e9c1d9d9ac96a5905bc3ee17658e75d90336d31b5978e4
7
- data.tar.gz: 5e699108252163cda89cfd604189343ad38c91a4dc4908d0aaefc890690415109ac6aa8d144a4846b350d20ffaf4e79c09292405fc69c3dffa30731ef6db7db9
6
+ metadata.gz: af8c922ef45eaf8a89a39bf840e92115c21f2fca4886add3d9b4a219d5e8ce8bb989be4dd7f3b20cee3170c43978ad9d71260140092d12dd6bc0161de767c010
7
+ data.tar.gz: e5882c9e5e905a8e457668b569140d0ec678eb81eb6de0c54249e073ca2f8dc7abdc42dce10613a2aaeaa53325194f4fcaaf947ec897b7bedf43093aedaad831
data/.travis.yml CHANGED
@@ -1,4 +1,7 @@
1
1
  language: ruby
2
2
  rvm:
3
- - 2.1.6
4
- before_install: gem install bundler -v 1.10.6
3
+ - 2.4.4
4
+ - 2.5.1
5
+
6
+ services:
7
+ - redis-server
data/CHANGELOG.md ADDED
@@ -0,0 +1,8 @@
1
+ # 0.1.2
2
+ * add AwsConfigBlocker extension. Prepending MeRedis successor with this module
3
+ prevents error while using the MeRedis with AWS ElasticCache (AWS blocks all config calls).
4
+
5
+ * added docker-compose for local testing comfort
6
+
7
+ # 0.1.1
8
+ * add MockRedis extension
data/Dockerfile ADDED
@@ -0,0 +1,6 @@
1
+ FROM ruby:2.5
2
+
3
+ WORKDIR me-redis
4
+
5
+ ADD . /me-redis/
6
+ RUN bundle install
data/README.md CHANGED
@@ -5,21 +5,19 @@ Me - Memory Efficient
5
5
  This gem is delivering memory optimizations for Redis with slightest code changes.
6
6
 
7
7
  To understand optimizations and how to use them
8
- I suggest you to read my paper this topic: https://medium.com/p/61076c7da4c
8
+ I suggest you to read my paper: https://medium.com/@leshchuk/zipem-all-61076c7da4c
9
9
 
10
- #Features:
10
+ Compatibility: ruby >= 2.4, redis gem >= 3.0.4
11
+
12
+ ## Features:
11
13
   
12
- * seamless integration with code already in use, hardest integration possible:
13
- add me_ prefix to some of your methods ( me_ methods implement hash memory optimization ).
14
- It's all in MeRedis configuration, not your current code. 
14
+ * seamless integration with code already in use. **It's all in MeRedis configuration, not your current code.** 
15
15
 
16
- * hash key/value optimization with seamless code changes,
17
- you can replace set('object:id', value) with me_set( 'object:id', value)
18
- and free 90 byte for each ['object:id', value] pair. 
16
+ * hash key/value optimization with seamless code changes, just replace set('object:id', value) with me_set( 'object:id', value) and free upto 90 bytes for each ['object:id', value] pair. 
19
17
 
20
18
  * zips user-friendly key crumbs according to configuration, i.e. converts for example user:id to u:id
21
19
 
22
- * zip integer parts of a keys with base62 encoding. Since all keys in redis are always strings, than we don't care for integers parts base, and by using base62 encoding we can 1.8 times shorten integer crumbs of keys 
20
+ * zip integer parts of keys with base62 encoding. Since all keys in redis are always strings, than we don't care for integer parts base, and by using base62 encoding we can shorten them upto 1.8 times shorter  
23
21
 
24
22
  * respects pipelined and multi, properly works with Futures. 
25
23
 
@@ -55,7 +53,9 @@ Or install it yourself as:
55
53
  1) less memory on redis side is better than less performance on ruby side
56
54
  2) more result with less code changes,
57
55
  i.e. overriding native redis methods with proper configuration basis is
58
- preferred over mixin new methods
56
+ preferred over mixin new methods.
57
+
58
+ I wanted magic to happend: mix all kind of optimizations without making 3 call of different methods.
59
59
 
60
60
  MeRedis based on three general optimization ideas:
61
61
  * shorten keys
@@ -73,7 +73,7 @@ way to deal with them all is call include upon MeRedis:
73
73
  redis = Redis.new
74
74
  ```
75
75
 
76
- If you want to keep a clear Redis class, you can do this way:
76
+ If you want to keep a clear Redis class, you can do it this way:
77
77
 
78
78
  ```ruby
79
79
  me_redis = Class.new(Redis).include(MeRedis).configure({...}).new
@@ -94,7 +94,22 @@ If you want to include them separately look at MeRedis.included method:
94
94
 
95
95
  This is the right chain of prepending/including, so just remove unnecessary module.
96
96
 
97
- ###Base use
97
+ ### AWS ElasticCache usage
98
+
99
+ AWS blocks config call with exception. To prevent this behaviour prepend MeRedis successor
100
+ with AwsConfigBlocker module:
101
+
102
+ ```ruby
103
+ Class.new(Redis)
104
+ .include(MeRedis)
105
+ .prepend( MeRedis::AwsConfigBlocker )
106
+ .configure({
107
+ # some config
108
+ }).new
109
+ ```
110
+
111
+
112
+ ### Base use
98
113
 
99
114
  ```ruby
100
115
  redis = Redis.new
@@ -133,9 +148,9 @@ This is the right chain of prepending/including, so just remove unnecessary modu
133
148
 
134
149
  As you can see you can get a result with smallest or even none code changes!
135
150
 
136
- All the ideas is to move complexity to config.
151
+ The idea is to move complexity to config.
137
152
 
138
- ###Config
153
+ ### Config
139
154
  ```ruby
140
155
 
141
156
  Redis.include(MeRedis).configure( hash_max_ziplist_entries: 512 )
@@ -151,9 +166,9 @@ All the ideas is to move complexity to config.
151
166
  :hash_max_ziplist_value
152
167
 
153
168
  # array or hash or string/sym of keys crumbs to zip,
154
- # if a hash given it used as is,
169
+ # when a hash is given it used as is,
155
170
  # otherwise MeRedis tries to construct hash by using first char from each key given
156
- # + integer in base62 starting from 1 for subsequent appearence of a crumbs starting with same chars
171
+ # + integer in base62 starting from 1 for subsequent appearence of a crumbs starting with same char
157
172
  :zip_crumbs
158
173
 
159
174
  # set to true if you want to zip ALL integers in keys to base62 form
@@ -183,7 +198,7 @@ All the ideas is to move complexity to config.
183
198
  # any kind of object which responds to compress/decompress methods
184
199
  :default_compressor
185
200
  ```
186
- ###Config examples
201
+ ### Config examples
187
202
 
188
203
  ```ruby
189
204
 
@@ -250,8 +265,8 @@ Redis.configure(
250
265
  integers_to_base62: true,
251
266
  zip_crumbs: %i[user card card_preview organization], # -> { user: :u, card: :c, card_preview: :c1, organization: :o }
252
267
  compress_namespaces: {
253
- /o:[a-zA-Z\d]+:card_preview/ => MeRedis::ZipValues::ZlibCompressor,
254
- [:user, :card].map{|crumb| /o:[a-zA-Z\d]+:#{crumb}/ } => ActiveRecordJSONCompressor
268
+ /o:[a-zA-Z\d]+:c1/ => MeRedis::ZipValues::ZlibCompressor,
269
+ [:u, :c].map{|crumb| /o:[a-zA-Z\d]+:#{crumb}/ } => ActiveRecordJSONCompressor
255
270
  }
256
271
  )
257
272
 
@@ -276,20 +291,24 @@ Redis.configure(
276
291
  compress_namespaces: %i[user card card_preview]
277
292
  )
278
293
  ```
294
+ ### Config best practices
279
295
  Now I may suggest some best practices for MeRedis configure:
280
296
 
281
297
  * explicit crumbs schema is preferable over implicit
282
- * if you are going lazy, and use implicit schemas, than avoid keys shuffling,
298
+ * if you are going lazy, and use implicit schemas, than avoid keys reshuffling,
283
299
  cause it messes with your cache
284
- * better to configure hash-max-ziplist-* in MeRedis.configure than elsewhere.
285
- * use in persistent Redis-based system with extreme caution
300
+ * better to configure hash-max-ziplist-* in MeRedis.configure than
301
+ elsewhere. And don't forget - this will reset them globally
302
+ for this Redis server!
303
+ * use MeRedis in persistent Redis-based system with extreme caution!
304
+
286
305
 
287
306
 
288
- #Custom Compressors
307
+ # Custom Compressors
289
308
 
290
309
  MeRedis allow you to compress values through different compressor.
291
310
  Here is an example of custom compressor for ActiveRecord objects,
292
- I use to test compression ratio against plain compression of to_json.
311
+ I use it to test compression ratio against plain compression of to_json. It takes ~40% less memory than Zlib.deflate(object.to_json).
293
312
 
294
313
  ```ruby
295
314
 
@@ -299,7 +318,7 @@ module ActiveRecordJSONCompressor
299
318
  # like User: { first_name: 1, last_name: 2 ... },
300
319
  # than your cache will be safer on schema changes.
301
320
  COMPRESSOR_SCHEMAS = [User, HTag].map{|mdl|
302
- [mdl.to_s, mdl.column_names.each_with_index.map{ |el, i| [el, (20 + i).to_base62] }.to_h]
321
+ [mdl.to_s, mdl.column_names.each_with_index.map{ |el, i| [el, i.to_base62] }.to_h]
303
322
  }.to_h.with_indifferent_access
304
323
 
305
324
  REVERSE_COMPRESSOR_SCHEMA = COMPRESSOR_SCHEMAS.dup.transform_values(&:invert)
@@ -326,20 +345,21 @@ end
326
345
 
327
346
  ```
328
347
 
329
- #Hot migration
330
- MeRedis deliver additional module for hot migration to KeyZipping and ZipToHash.
331
- We don't need one in generally for base implementation of ZipValues cause
332
- its getter methods fallbacks to value.
348
+ # Hot migration
349
+ MeRedis deliver additional module MeRedisHotMigrator for hot migration to KeyZipping and ZipToHash.
333
350
 
334
- ###Features
335
- * mget hget hgetall get exists type getset - fallbacks for key_zipping
336
- * me_get me_mget - fallbacks for hash zipping
351
+ We don't need one in generally for base implementation of ZipValues module cause
352
+ its getter methods fallbacks to values.
353
+
354
+ ### Features
355
+ * mget hget hgetall get exists type getset - fallbacks to original methods for key_zipping
356
+ * me_get me_mget - fallbacks to original methods for hash zipping
337
357
  * partially respects pipelining and multi
338
358
  * protecting you from accidentally do many to less many migration
339
359
  and from ZipToHash migration without key zipping (
340
360
  though it's impossible to hot migrate from 'user:100' to 'user:1', 'B',
341
361
  because of same namespace 'user' for flat key/value pair and hashes,
342
- you'll definetely get an error )
362
+ you'll definetely get an error from Redis )
343
363
  * reverse migration methods
344
364
 
345
365
  ```ruby
@@ -350,42 +370,59 @@ its getter methods fallbacks to value.
350
370
  usr_1_cache = redis.me_get('user:1')
351
371
 
352
372
  all_user_keys = redis.keys('user*')
373
+ # it doesn't delete all_user_keys!!! they still all in the Redis
353
374
  redis.migrate_to_hash_representation( all_user_keys )
354
-
355
375
  usr_1_cache == redis.me_get('user:1') # true
356
376
 
377
+ redis.del( *all_user_keys )
378
+ # now you a ready to measure memory bonus you've got
379
+ ap redis.info(:memory)
380
+
381
+ usr_1_cache == redis.me_get('user:1') # true
382
+
357
383
  redis.reverse_from_hash_representation!( all_user_keys )
358
384
 
359
385
  usr_1_cache == redis.me_get('user:1') # true
360
-
361
386
  ```
362
387
 
363
388
  For persistent store use with extreme caution!!
364
- Backup, test, test, user test and after you are sure than you may migrate.
389
+ Backup, test, test, user test and after you are completely sure than you may migrate.
365
390
 
366
- Try not to stuck with it because doing double amount of actions,
367
- do BG deploy of code, run migration in parallel, replace MeRedisHotMigrator with MeRedis
368
- do BG deploy and you are done.
391
+ Try not to stuck with MeRedisHotMigrator because it doing double amount of actions:
392
+
393
+ 1. do BG deploy of code with MeRedisHotMigrator,
394
+ 2. run migration in parallel,
395
+ 3. replace MeRedisHotMigrator with MeRedis
396
+ 4. do BG deploy
397
+
398
+ and you are done.
399
+
400
+ # Debug and broken expectations
401
+
402
+ 1. Check if modules you using override appropriate method ( if not make issue :) )
403
+ 2. Check the order of prepending / including modules ( look into MeRedis.included to see how things must be done )
404
+ 3. Call ap Redis.me_config and check it over your expectation,
405
+ mistakes are usually in config. Don't forget - namespaces inside regexp must be in zipped form ( integers became base62 integers strings and crumbs are zipped )!
406
+ 4. Check hash-max-size-value, if your average data size is greater
407
+ than hash-max-size-value than hash optimization will not bring resource savings.
408
+ 5. Check hash-max-size-entries if it small than hash effect is less than expected
409
+ 5. Trace through code and inspect zip_key and zip? output.
369
410
 
370
- #Limitations
411
+ # Limitations
371
412
 
372
- ###Me_* methods limitation
413
+ ### Me_* methods limitation
373
414
 
374
415
  Some of me_methods like me_mget/me_mset/me_getset
375
- are imitations for corresponded base methods behaviour through
416
+ are imitations for corresponded Redis base methods behaviour through
376
417
  pipeline and transactions. So inside pipelined call it may not
377
418
  deliver a completely equal behaviour.
378
419
 
379
- me_mget has an additional double me_mget_p in case you need to use it with futures.
420
+ me_mget has a method double me_mget_p in case you need to use it with futures.
380
421
 
381
- ###ZipKeys and ZipValues
422
+ ### ZipKeys and ZipValues
382
423
  As I already mention if you want to use custom prefix regex
383
424
  for zipping values than it must be constructed with a crumbs substitutions,
384
425
  not the original crumb, see config example.
385
-
386
- ```ruby
387
-
388
- ```
389
426
 
390
427
  ## Development
391
428
 
@@ -393,6 +430,31 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
393
430
 
394
431
  To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
395
432
 
433
+ ## Testing
434
+
435
+ ```bash
436
+ docker-compose build
437
+ docker-compose run test
438
+ ```
439
+
440
+ ### MockRedis
441
+
442
+ If you are using in mock_redis gem for testing purpose and see some test
443
+ breakage me-redis has a MockRedis extended class.
444
+ Its imitate single DB for all instances of MockRedis class
445
+ and add some methods MeRedis rely upon. Its not a complete replacement
446
+ for Redis class, but you can try it out.
447
+
448
+
449
+ ### RedisSafety
450
+ MeRedis tests itself against RedisSafety class which is a Redis descendant class.
451
+ All instances of RedisSafety ensure that Redis DB is empty otherwise they will raise an error.
452
+ This is a precaution measure to prevent you from accidentally run MeRedis test again production DB.
453
+
454
+ ```
455
+ rake test REDIS_TEST_DB=5
456
+ ```
457
+
396
458
  ## Contributing
397
459
 
398
460
  Bug reports and pull requests are welcome on GitHub at https://github.com/alekseyl/me-redis.
@@ -404,5 +466,6 @@ The gem is available as open source under the terms of the [MIT License](http://
404
466
 
405
467
  ## ToDo List
406
468
 
469
+ * add warning option for hash-max-ziplist-value overthrowing
407
470
  * add keys method
408
- * refactor readme
471
+ * refactor readme
@@ -0,0 +1,17 @@
1
+ version: "3.7"
2
+ services:
3
+ redis:
4
+ image: redis
5
+ ports:
6
+ - "6379"
7
+
8
+ test:
9
+ build: .
10
+ image: me-redis-test
11
+ command: bundle exec rake test
12
+ volumes:
13
+ - '.:/me-redis'
14
+ depends_on:
15
+ - redis
16
+ environment:
17
+ REDIS_URL: redis://redis:6379
data/lib/me_redis.rb CHANGED
@@ -5,7 +5,7 @@ require 'me_redis/zip_values'
5
5
  require 'me_redis/me_redis_hot_migrator'
6
6
  require 'me_redis/integer'
7
7
  require 'me_redis/hash'
8
-
8
+ require 'me_redis/aws_config_blocker'
9
9
  require 'zlib'
10
10
 
11
11
  # Main ideas:
@@ -140,28 +140,27 @@ module MeRedis
140
140
  end
141
141
 
142
142
  def prepare_compressors
143
-
144
143
  me_config.default_compressor ||= MeRedis::ZipValues::EmptyCompressor
145
144
 
146
145
  me_config.compress_namespaces = case me_config.compress_namespaces
147
- when Array
148
- me_config.compress_namespaces.map{|ns| [replace_ns(ns), me_config.default_compressor] }.to_h
149
- when String, Symbol, Regexp
150
- { replace_ns( me_config.compress_namespaces ) => me_config.default_compressor }
151
- when Hash
152
- me_config.compress_namespaces.inject({}) do |sum, (name_space, compressor)|
153
- name_space.is_a?( Array ) ?
154
- sum.merge!( name_space.map{ |ns| [replace_ns( ns), compressor] }.to_h )
155
- : sum[replace_ns(name_space)] = compressor
156
- sum
157
- end
158
- else
159
- raise ArgumentError.new(<<~NS_ERR) if me_config.compress_namespaces
146
+ when Array
147
+ me_config.compress_namespaces.map{|ns| [replace_ns(ns), me_config.default_compressor] }.to_h
148
+ when String, Symbol, Regexp
149
+ { replace_ns( me_config.compress_namespaces ) => me_config.default_compressor }
150
+ when Hash
151
+ me_config.compress_namespaces.inject({}) do |sum, (name_space, compressor)|
152
+ name_space.is_a?( Array ) ?
153
+ sum.merge!( name_space.map{ |ns| [replace_ns( ns), compressor] }.to_h )
154
+ : sum[replace_ns(name_space)] = compressor
155
+ sum
156
+ end
157
+ else
158
+ raise ArgumentError.new(<<~NS_ERR) if me_config.compress_namespaces
160
159
  Wrong class for compress_namespaces, expected Array,
161
- Hash, String or Symbol! Got: #{me_config.compress_namespaces.class.to_s}
162
- NS_ERR
163
- {}
164
- end
160
+ Hash, String or Symbol! Got: #{me_config.compress_namespaces.class.to_s}
161
+ NS_ERR
162
+ {}
163
+ end
165
164
 
166
165
  zip_ns_finder
167
166
  end
@@ -0,0 +1,30 @@
1
+ require 'ap'
2
+ module MeRedis
3
+ # how to use:
4
+ # Redis.include( MeRedis::AwsConfigBlocker )
5
+ module AwsConfigBlocker
6
+
7
+ def config( *args, &block )
8
+ print <<AWS_MSG
9
+
10
+ \e[0;33;31;1m!!!!!!!!! MeRedis AWS CONFIG BLOCKER WARNING!!!!!! \e[0;33;31;0m
11
+ You introduced AwsConfigBlocker into the ancestors chain, that means that you intend to skip Redis config call,
12
+ because AWS does not support config get/set calls by throwing an exception.
13
+ AwsConfigBlocker will block config call from reaching your Redis server!
14
+
15
+ You are calling config with arguments:
16
+
17
+ AWS_MSG
18
+
19
+ ap args
20
+
21
+ print <<END_MSG
22
+
23
+
24
+ Don't forget to setup redis param group with exact values, or you might face unexpected optimization degradation.
25
+ You can find more details in the README section.
26
+ END_MSG
27
+ {}
28
+ end
29
+ end
30
+ end
data/lib/me_redis/hash.rb CHANGED
@@ -1,3 +1,4 @@
1
+ # ruby 2.3 support
1
2
  class Hash
2
3
  # Returns a new hash with all keys converted using the +block+ operation.
3
4
  #
@@ -26,4 +27,5 @@ class Hash
26
27
  end
27
28
  self
28
29
  end
30
+
29
31
  end unless Hash.method_defined?(:transform_keys)
@@ -1,3 +1,3 @@
1
1
  module MeRedis
2
- VERSION = "0.1.0"
2
+ VERSION = "0.1.2"
3
3
  end
@@ -28,7 +28,6 @@ module MeRedis
28
28
  def hmget( key, *args ); super( zip_key(key), *args ) end
29
29
  #---- Hash methods END --------------------
30
30
 
31
-
32
31
  def incr( key ); super( zip_key(key) ) end
33
32
  def get( key ); super( zip_key(key) ) end
34
33
 
@@ -30,8 +30,8 @@ module MeRedis
30
30
  end
31
31
 
32
32
  def self.included(base)
33
- base.extend( MeRedis::ClassMethods )
34
- base.prepend( PrependMethods )
33
+ base.extend(MeRedis::ClassMethods)
34
+ base.prepend(PrependMethods)
35
35
  end
36
36
 
37
37
  def me_del( *keys )
@@ -1,6 +1,6 @@
1
1
  module MeRedis
2
- # todo warn in development that gzipped size iz bigger than strict
3
- # use prepend for classes or extend on instances
2
+ # todo warn in development when gzipped size iz bigger than strict
3
+ # use prepend for classes
4
4
  module ZipValues
5
5
  module FutureUnzip
6
6
  def set_transformation(&block)
@@ -10,7 +10,7 @@ module MeRedis
10
10
  @old_transformation = @transformation
11
11
  @transformation = -> (vl) {
12
12
  if @old_transformation
13
- @old_transformation.call( block.call(vl, self) )
13
+ @old_transformation.call(block.call(vl, self))
14
14
  else
15
15
  block.call(vl, self)
16
16
  end
@@ -19,50 +19,9 @@ module MeRedis
19
19
  end
20
20
 
21
21
  # patch futures we need only when we are returning values, usual setters returns OK
22
- COMMANDS = %i[ incr incrby hincrby get hget getset mget hgetall ].map{|cmd| [cmd, true]}.to_h
22
+ COMMANDS = %i[incr incrby hincrby get hget getset mget hgetall].map{ |cmd| [cmd, true]}.to_h
23
23
  end
24
24
 
25
- module PrependMethods
26
- def pipelined( &block )
27
- super do |redis|
28
- block.call(redis)
29
- _patch_futures(@client)
30
- end
31
- end
32
-
33
- def multi( &block )
34
- super do |redis|
35
- block.call(redis)
36
- _patch_futures(@client)
37
- end
38
- end
39
-
40
- def _patch_futures(client)
41
- client.futures.each do |ftr|
42
-
43
- ftr.set_transformation do |vl|
44
- if vl && FutureUnzip::COMMANDS[ftr._command[0]]
45
- # we only dealing here with GET methods, so it could be hash getters or get/mget
46
- keys = ftr._command[0][0] == 'h' ? ftr._command[1,1] : ftr._command[1..-1]
47
- if ftr._command[0] == :mget
48
- vl.each_with_index.map{ |v, i| zip?(keys[i]) ? self.class.get_compressor_for_key(keys[i]).decompress( v ) : v }
49
- elsif zip?(keys[0])
50
- compressor = self.class.get_compressor_for_key(keys[0])
51
- # on hash commands it could be an array
52
- vl.is_a?(Array) ? vl.map!{|v| compressor.decompress( v ) } : compressor.decompress(vl)
53
- else
54
- vl
55
- end
56
- else
57
- vl
58
- end
59
- end
60
-
61
- end
62
- end
63
- end
64
-
65
-
66
25
  module ZlibCompressor
67
26
  def self.compress(value); Zlib.deflate(value.to_s ) end
68
27
 
@@ -78,23 +37,53 @@ module MeRedis
78
37
  def self.decompress(value); value end
79
38
  end
80
39
 
81
- # for the global gzipping
82
40
  def self.prepended(base)
83
41
  base::Future.prepend(FutureUnzip)
84
- base.prepend(PrependMethods)
85
42
 
86
- base.extend( MeRedis::ClassMethods )
87
- base.me_config.default_compressor = MeRedis::ZipValues::ZlibCompressor
43
+ base.extend(MeRedis::ClassMethods)
44
+
45
+ base.me_config.default_compressor = ::MeRedis::ZipValues::ZlibCompressor
88
46
  end
89
- # for object extending
90
- def self.included(base)
91
- base::Future.prepend(FutureUnzip)
92
- base.prepend(PrependMethods)
93
47
 
94
- base.extend( MeRedis::ClassMethods )
95
- base.me_config.default_compressor = MeRedis::ZipValues::ZlibCompressor
48
+ def pipelined(&block)
49
+ super do |redis|
50
+ block.call(redis)
51
+ _patch_futures(@client)
52
+ end
53
+ end
54
+
55
+ def multi(&block)
56
+ super do |redis|
57
+ block.call(redis)
58
+ _patch_futures(@client)
59
+ end
60
+ end
61
+
62
+ def _patch_futures(client)
63
+ client.futures.each do |ftr|
64
+
65
+ ftr.set_transformation do |vl|
66
+ if vl && FutureUnzip::COMMANDS[ftr._command[0]]
67
+ # we only dealing here with GET methods, so it could be hash getters or get/mget
68
+ keys = ftr._command[0][0] == 'h' ? ftr._command[1, 1] : ftr._command[1..-1]
69
+ if ftr._command[0] == :mget
70
+ vl.each_with_index.map{ |v, i| zip?(keys[i]) ? self.class.get_compressor_for_key(keys[i]).decompress( v ) : v }
71
+ elsif zip?(keys[0])
72
+ compressor = self.class.get_compressor_for_key(keys[0])
73
+ # on hash commands it could be an array
74
+ vl.is_a?(Array) ? vl.map!{ |v| compressor.decompress(v) } : compressor.decompress(vl)
75
+ else
76
+ vl
77
+ end
78
+ else
79
+ vl
80
+ end
81
+ end
82
+
83
+ end
96
84
  end
97
85
 
86
+
98
87
  def zip_value(value, key )
99
88
  zip?(key) ? self.class.get_compressor_for_key(key).compress( value ) : value
100
89
  end
data/me-redis.gemspec CHANGED
@@ -29,8 +29,11 @@ Gem::Specification.new do |spec|
29
29
 
30
30
  spec.add_dependency 'redis', '>= 3.0'
31
31
  spec.add_dependency 'base62-rb'
32
+ spec.add_dependency "awesome_print"
32
33
 
33
34
  spec.add_development_dependency "bundler", "~> 1.16"
34
35
  spec.add_development_dependency "rake", "~> 10.0"
35
36
  spec.add_development_dependency "minitest"
37
+ spec.add_development_dependency "pry-byebug"
38
+
36
39
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: me-redis
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - alekseyl
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2018-05-23 00:00:00.000000000 Z
11
+ date: 2019-06-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: redis
@@ -38,6 +38,20 @@ dependencies:
38
38
  - - ">="
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: awesome_print
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :runtime
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
41
55
  - !ruby/object:Gem::Dependency
42
56
  name: bundler
43
57
  requirement: !ruby/object:Gem::Requirement
@@ -80,6 +94,20 @@ dependencies:
80
94
  - - ">="
81
95
  - !ruby/object:Gem::Version
82
96
  version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: pry-byebug
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
83
111
  description: Enable to zip keys, zip values and replace simple storage key/value pairs
84
112
  with hash storing
85
113
  email:
@@ -89,18 +117,18 @@ extensions: []
89
117
  extra_rdoc_files: []
90
118
  files:
91
119
  - ".gitignore"
92
- - ".idea/me-redis.iml"
93
- - ".idea/misc.xml"
94
- - ".idea/modules.xml"
95
- - ".idea/workspace.xml"
96
120
  - ".travis.yml"
121
+ - CHANGELOG.md
122
+ - Dockerfile
97
123
  - Gemfile
98
124
  - LICENSE.txt
99
125
  - README.md
100
126
  - Rakefile
101
127
  - bin/console
102
128
  - bin/setup
129
+ - docker-compose.yml
103
130
  - lib/me_redis.rb
131
+ - lib/me_redis/aws_config_blocker.rb
104
132
  - lib/me_redis/hash.rb
105
133
  - lib/me_redis/integer.rb
106
134
  - lib/me_redis/me_redis_hot_migrator.rb
@@ -130,7 +158,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
130
158
  version: '0'
131
159
  requirements: []
132
160
  rubyforge_project:
133
- rubygems_version: 2.6.11
161
+ rubygems_version: 2.7.9
134
162
  signing_key:
135
163
  specification_version: 4
136
164
  summary: Memory efficient redis extention