readthis 0.8.1 → 1.0.0.pre.beta
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +24 -3
- data/README.md +22 -4
- data/benchmarks/composing.rb +43 -0
- data/benchmarks/generic.rb +30 -0
- data/benchmarks/marshalling.rb +6 -8
- data/benchmarks/multi.rb +0 -4
- data/benchmarks/parsing.rb +42 -0
- data/benchmarks/profile.rb +1 -1
- data/lib/readthis.rb +6 -0
- data/lib/readthis/cache.rb +6 -5
- data/lib/readthis/entity.rb +121 -28
- data/lib/readthis/serializers.rb +100 -0
- data/lib/readthis/version.rb +1 -1
- data/spec/readthis/cache_spec.rb +37 -4
- data/spec/readthis/entity_spec.rb +79 -12
- data/spec/readthis/serializers_spec.rb +72 -0
- data/spec/readthis_spec.rb +9 -0
- metadata +13 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b0b93b4402db2eaaff687eb60a40296bb618dd4e
|
4
|
+
data.tar.gz: 81f108e00b2b7dfe0ce92d9209c1160e3cc59a6f
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d8e81b60017904895db2c2d2666f9f9d2fdad7fa2178c6bbca617abc0d49e5d8934c3128e0f8dba1014c5b30f02d2101fcf3102d2ecee441d5f43f9be9a35282
|
7
|
+
data.tar.gz: 1b8c28d2a8fdcff75d38bc565dceaf14fb442a1d14ca2c547c53e315d5d6eae08f35f96dbf3f97e268f71774ab27eb9afa5faf4266a1f3becf8af7303c92eee9
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,24 @@
|
|
1
|
+
## v1.0.0-beta
|
2
|
+
|
3
|
+
- Breaking: This change is necessary for the consistency and portability of
|
4
|
+
values going forward. All entities are now written with a set of option flags
|
5
|
+
as the initial byte. This flag is later used to determine whether the entity
|
6
|
+
was compressed and what was used to marshal it. There are a number of
|
7
|
+
advantages to this approach, consistency and reliability being the most
|
8
|
+
important. See [readthis#17][pull-17] for additional background.
|
9
|
+
- Added: Per-entity options can be passed through to any cache method that
|
10
|
+
writes a value (`write`, `fetch`, etc). For example, this allows certain
|
11
|
+
entities to be cached as JSON while all other entities are cached using
|
12
|
+
Marshal. Thanks to @fabn.
|
13
|
+
- Fixed: A hash containing the cache key is passed as the payload for
|
14
|
+
`ActiveSupport::Notifications` instrumentation, rather than the key directly.
|
15
|
+
This moves the implementation in-line with the tests for the code, and
|
16
|
+
prevents errors from being masked when an error occurs inside an instrumented
|
17
|
+
block. [readthis#20][pull-20]. Discovered by @banister and fixed by @workmad3.
|
18
|
+
|
19
|
+
[pull-17]: https://github.com/sorentwo/readthis/pull/17
|
20
|
+
[pull-20]: https://github.com/sorentwo/readthis/pull/20
|
21
|
+
|
1
22
|
## v0.8.1 2015-09-04
|
2
23
|
|
3
24
|
- Changed: `Readthis::Cache` now has an accessor for the options that were
|
@@ -13,9 +34,9 @@
|
|
13
34
|
when it can the value looks to be compressed, falling back to the initial
|
14
35
|
value when decompression fails. See [readthis#13][issue-13] for details.
|
15
36
|
|
16
|
-
[issue-13]: https://github.com/sorentwo/readthis/
|
17
|
-
[issue-15]: https://github.com/sorentwo/readthis/
|
18
|
-
[issue-16]: https://github.com/sorentwo/readthis/
|
37
|
+
[issue-13]: https://github.com/sorentwo/readthis/issues/13
|
38
|
+
[issue-15]: https://github.com/sorentwo/readthis/issues/15
|
39
|
+
[issue-16]: https://github.com/sorentwo/readthis/issues/16
|
19
40
|
|
20
41
|
## v0.8.0 2015-08-26
|
21
42
|
|
data/README.md
CHANGED
@@ -102,11 +102,11 @@ Readthis uses Ruby's `Marshal` module for dumping and loading all values by
|
|
102
102
|
default. This isn't always the fastest option, and depending on your use case it
|
103
103
|
may be desirable to use a faster but less flexible marshaller.
|
104
104
|
|
105
|
-
|
105
|
+
By default Readthis knows about 3 different serializers for marshalling:
|
106
106
|
|
107
|
-
|
108
|
-
|
109
|
-
|
107
|
+
* Marshal
|
108
|
+
* JSON
|
109
|
+
* Passthrough
|
110
110
|
|
111
111
|
If all cached data can safely be represented as a string then use the
|
112
112
|
pass-through marshaller:
|
@@ -115,6 +115,24 @@ pass-through marshaller:
|
|
115
115
|
Readthis::Cache.new(marshal: Readthis::Passthrough)
|
116
116
|
```
|
117
117
|
|
118
|
+
You can introduce up to four additional marshals by configuring `serializers` on
|
119
|
+
the Readthis module. For example, if you wanted to use Oj for JSON marshalling,
|
120
|
+
it is extremely fast, but supports limited types:
|
121
|
+
|
122
|
+
```ruby
|
123
|
+
Readthis.serializers << Oj
|
124
|
+
|
125
|
+
# Freeze the serializers to ensure they aren't changed at runtime.
|
126
|
+
Readthis.serializers.freeze!
|
127
|
+
|
128
|
+
Readthis::Cache.new(marshal: Oj)
|
129
|
+
```
|
130
|
+
|
131
|
+
Be aware that the order in which you add serializers matters. Serializers are
|
132
|
+
sticky and a flag is stored with each cached value. If you subsequently go to
|
133
|
+
deserialize values and haven't configured the same serializers in the same order
|
134
|
+
your application will raise errors.
|
135
|
+
|
118
136
|
## Differences From ActiveSupport::Cache
|
119
137
|
|
120
138
|
Readthis supports all of standard cache methods except for the following:
|
@@ -0,0 +1,43 @@
|
|
1
|
+
require 'benchmark/ips'
|
2
|
+
|
3
|
+
def compose_a(marshal, compress)
|
4
|
+
prefix = ''
|
5
|
+
prefix << 'R|'.freeze
|
6
|
+
prefix << marshal.name.ljust(24)
|
7
|
+
prefix << (compress ? '1'.freeze : '0'.freeze)
|
8
|
+
prefix << 1
|
9
|
+
prefix << '|R'.freeze
|
10
|
+
end
|
11
|
+
|
12
|
+
def compose_b(marshal, compress)
|
13
|
+
"R|#{marshal.name.ljust(24)}#{compress ? '1'.freeze : '0'.freeze}1|R"
|
14
|
+
end
|
15
|
+
|
16
|
+
def compose_c(marshal, compress)
|
17
|
+
name = marshal.name.ljust(24)
|
18
|
+
comp = compress ? '1'.freeze : '0'.freeze
|
19
|
+
|
20
|
+
"R|#{name}#{comp}1|R"
|
21
|
+
end
|
22
|
+
|
23
|
+
SERIALIZER_FLAG = { Marshal => 0x1 }.freeze
|
24
|
+
COMPRESSED_FLAG = 0x8
|
25
|
+
|
26
|
+
# | 0000 | 0 | 000 |
|
27
|
+
# four unused bits, # 1 compression bit, 3 bits for serializer, allow up to 8
|
28
|
+
# different marshalers
|
29
|
+
def compose_d(marshal, compress)
|
30
|
+
flags = SERIALIZER_FLAG[marshal]
|
31
|
+
flags |= COMPRESSED_FLAG if compress
|
32
|
+
|
33
|
+
[flags].pack('C')
|
34
|
+
end
|
35
|
+
|
36
|
+
Benchmark.ips do |x|
|
37
|
+
x.report('a') { compose_a(Marshal, true) }
|
38
|
+
x.report('b') { compose_b(Marshal, true) }
|
39
|
+
x.report('c') { compose_c(Marshal, true) }
|
40
|
+
x.report('d') { compose_d(Marshal, true) }
|
41
|
+
|
42
|
+
x.compare!
|
43
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
require 'benchmark/ips'
|
2
|
+
require 'json'
|
3
|
+
require 'readthis'
|
4
|
+
|
5
|
+
READTHIS = Readthis::Cache.new(
|
6
|
+
expires_in: 120,
|
7
|
+
marshal: JSON,
|
8
|
+
compress: true
|
9
|
+
)
|
10
|
+
|
11
|
+
def write_key(key)
|
12
|
+
READTHIS.write(key, key.to_s * 2048)
|
13
|
+
end
|
14
|
+
|
15
|
+
KEYS = (1..1_000).to_a
|
16
|
+
KEYS.each { |key| write_key(key) }
|
17
|
+
|
18
|
+
Benchmark.ips do |x|
|
19
|
+
x.report 'readthis:write' do
|
20
|
+
write_key(KEYS.sample)
|
21
|
+
end
|
22
|
+
|
23
|
+
x.report 'readthis:read' do
|
24
|
+
READTHIS.read(KEYS.sample)
|
25
|
+
end
|
26
|
+
|
27
|
+
x.report 'readthis:read_multi' do
|
28
|
+
READTHIS.read(KEYS.sample(30))
|
29
|
+
end
|
30
|
+
end
|
data/benchmarks/marshalling.rb
CHANGED
@@ -9,18 +9,17 @@ require 'msgpack'
|
|
9
9
|
require 'readthis'
|
10
10
|
require 'readthis/passthrough'
|
11
11
|
|
12
|
-
|
12
|
+
Readthis.serializers << Oj
|
13
|
+
Readthis.serializers << MessagePack
|
13
14
|
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
readthis_ruby = Readthis::Cache.new(OPTIONS.merge(marshal: Marshal))
|
15
|
+
readthis_oj = Readthis::Cache.new(marshal: Oj)
|
16
|
+
readthis_msgpack = Readthis::Cache.new(marshal: MessagePack)
|
17
|
+
readthis_json = Readthis::Cache.new(marshal: JSON)
|
18
|
+
readthis_ruby = Readthis::Cache.new(marshal: Marshal)
|
19
19
|
|
20
20
|
HASH = ('a'..'z').each_with_object({}) { |key, memo| memo[key] = key }
|
21
21
|
|
22
22
|
Benchmark.ips do |x|
|
23
|
-
x.report('pass:hash:dump') { readthis_pass.write('pass', HASH) }
|
24
23
|
x.report('oj:hash:dump') { readthis_oj.write('oj', HASH) }
|
25
24
|
x.report('json:hash:dump') { readthis_json.write('json', HASH) }
|
26
25
|
x.report('msgpack:hash:dump') { readthis_msgpack.write('msgpack', HASH) }
|
@@ -30,7 +29,6 @@ Benchmark.ips do |x|
|
|
30
29
|
end
|
31
30
|
|
32
31
|
Benchmark.ips do |x|
|
33
|
-
x.report('pass:hash:load') { readthis_pass.read('pass') }
|
34
32
|
x.report('oj:hash:load') { readthis_oj.read('oj') }
|
35
33
|
x.report('json:hash:load') { readthis_json.read('json') }
|
36
34
|
x.report('msgpack:hash:load') { readthis_msgpack.read('msgpack') }
|
data/benchmarks/multi.rb
CHANGED
@@ -0,0 +1,42 @@
|
|
1
|
+
require 'benchmark/ips'
|
2
|
+
|
3
|
+
def parse_a(string)
|
4
|
+
marshal = string[2, 12].strip
|
5
|
+
compress = string[15] == '1'.freeze
|
6
|
+
|
7
|
+
[Kernel.const_get(marshal), compress, string[18..-1]]
|
8
|
+
end
|
9
|
+
|
10
|
+
def parse_b(marked)
|
11
|
+
prefix = marked[0, 32].scrub('*'.freeze)[/R\|(.*)\|R/, 1]
|
12
|
+
offset = prefix.size + 4
|
13
|
+
|
14
|
+
marshal, c_name, _ = prefix.split(' '.freeze)
|
15
|
+
|
16
|
+
compress = c_name == 'true'.freeze
|
17
|
+
|
18
|
+
[Kernel.const_get(marshal), compress, marked[offset..-1]]
|
19
|
+
end
|
20
|
+
|
21
|
+
SERIALIZERS = { Marshal => 0x1 }.freeze
|
22
|
+
DESERIALIZERS = SERIALIZERS.invert.freeze
|
23
|
+
COMPRESSED_FLAG = 0x8
|
24
|
+
MARSHAL_FLAG = 0x3
|
25
|
+
BINARY_FLAG = [SERIALIZERS[Marshal] | COMPRESSED_FLAG].pack('C')
|
26
|
+
|
27
|
+
def parse_c(binary_string)
|
28
|
+
flags = binary_string[0].unpack('C').first
|
29
|
+
|
30
|
+
[DESERIALIZERS[flags & MARSHAL_FLAG], flags & COMPRESSED_FLAG, binary_string[1..-1]]
|
31
|
+
end
|
32
|
+
|
33
|
+
STR = 'R|Marshal 0|Rafdlkadfjadfj asdlkfjasdlfkj asdlfkjdasflkjadsflkjadslkjfadslkjfasdlkjfadlskjf laksdjflkajsdflkjadsflkadjsfladskjf laksjflakdjfalsdkjfadlskjf laksdjflkajdsflk j'
|
34
|
+
STR2 = BINARY_FLAG << 'Rafdlkadfjadfj asdlkfjasdlfkj asdlfkjdasflkjadsflkjadslkjfadslkjfasdlkjfadlskjf laksdjflkajsdflkjadsflkadjsfladskjf laksjflakdjfalsdkjfadlskjf laksdjflkajdsflk j'
|
35
|
+
|
36
|
+
Benchmark.ips do |x|
|
37
|
+
x.report('a') { parse_a(STR) }
|
38
|
+
x.report('b') { parse_b(STR) }
|
39
|
+
x.report('c') { parse_c(STR2) }
|
40
|
+
|
41
|
+
x.compare!
|
42
|
+
end
|
data/benchmarks/profile.rb
CHANGED
data/lib/readthis.rb
CHANGED
data/lib/readthis/cache.rb
CHANGED
@@ -171,7 +171,7 @@ module Readthis
|
|
171
171
|
# cache.increment('counter', 2) # => 3
|
172
172
|
#
|
173
173
|
def increment(key, amount = 1, options = {})
|
174
|
-
invoke(:
|
174
|
+
invoke(:increment, key) do |store|
|
175
175
|
alter(key, amount, options)
|
176
176
|
end
|
177
177
|
end
|
@@ -310,7 +310,7 @@ module Readthis
|
|
310
310
|
# @example
|
311
311
|
#
|
312
312
|
# cache.clear #=> 'OK'
|
313
|
-
def clear(
|
313
|
+
def clear(_options = nil)
|
314
314
|
invoke(:clear, '*', &:flushdb)
|
315
315
|
end
|
316
316
|
|
@@ -318,11 +318,12 @@ module Readthis
|
|
318
318
|
|
319
319
|
def write_entity(key, value, store, options)
|
320
320
|
namespaced = namespaced_key(key, options)
|
321
|
+
dumped = entity.dump(value, options)
|
321
322
|
|
322
323
|
if expiration = options[:expires_in]
|
323
|
-
store.setex(namespaced, expiration.to_i,
|
324
|
+
store.setex(namespaced, expiration.to_i, dumped)
|
324
325
|
else
|
325
|
-
store.set(namespaced,
|
326
|
+
store.set(namespaced, dumped)
|
326
327
|
end
|
327
328
|
end
|
328
329
|
|
@@ -339,7 +340,7 @@ module Readthis
|
|
339
340
|
name = "cache_#{operation}.active_support"
|
340
341
|
payload = { key: key }
|
341
342
|
|
342
|
-
self.class.notifications.instrument(name,
|
343
|
+
self.class.notifications.instrument(name, payload) { yield(payload) }
|
343
344
|
end
|
344
345
|
|
345
346
|
def invoke(operation, key, &block)
|
data/lib/readthis/entity.rb
CHANGED
@@ -2,51 +2,144 @@ require 'zlib'
|
|
2
2
|
|
3
3
|
module Readthis
|
4
4
|
class Entity
|
5
|
-
|
6
|
-
|
5
|
+
DEFAULT_OPTIONS = {
|
6
|
+
compress: false,
|
7
|
+
marshal: Marshal,
|
8
|
+
threshold: 8 * 1024
|
9
|
+
}.freeze
|
7
10
|
|
8
|
-
|
11
|
+
COMPRESSED_FLAG = 0x8
|
12
|
+
MARSHAL_FLAG = 0x3
|
9
13
|
|
14
|
+
# Creates a Readthis::Entity with default options. Each option can be
|
15
|
+
# overridden later when entities are being dumped.
|
16
|
+
#
|
17
|
+
# Options are sticky, meaning that whatever is used when dumping will
|
18
|
+
# automatically be used again when loading, regardless of how current
|
19
|
+
# options are set.
|
20
|
+
#
|
21
|
+
# @option [Boolean] :compress (false) Enable or disable automatic compression
|
22
|
+
# @option [Module] :marshal (Marshal) Any module that responds to `dump` and `load`
|
23
|
+
# @option [Number] :threshold (8k) The size a string must be for compression
|
24
|
+
#
|
10
25
|
def initialize(options = {})
|
11
|
-
@
|
12
|
-
@compression = options.fetch(:compress, false)
|
13
|
-
@threshold = options.fetch(:threshold, DEFAULT_THRESHOLD)
|
26
|
+
@options = DEFAULT_OPTIONS.merge(options)
|
14
27
|
end
|
15
28
|
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
29
|
+
# Output a value prepared for cache storage. Passed options will override
|
30
|
+
# whatever has been specified for the instance.
|
31
|
+
#
|
32
|
+
# @param [String] String to dump
|
33
|
+
# @option [Boolean] :compress Enable or disable automatic compression
|
34
|
+
# @option [Module] :marshal Any module that responds to `dump` and `load`
|
35
|
+
# @option [Number] :threshold The size a string must be for compression
|
36
|
+
# @return [String] The prepared, possibly compressed, string
|
37
|
+
#
|
38
|
+
# @example Dumping a value using defaults
|
39
|
+
#
|
40
|
+
# entity.dump(string)
|
41
|
+
#
|
42
|
+
# @example Dumping a value with overrides
|
43
|
+
#
|
44
|
+
# entity.dump(string, compress: false)
|
45
|
+
#
|
46
|
+
def dump(value, options = {})
|
47
|
+
compress = with_fallback(options, :compress)
|
48
|
+
marshal = with_fallback(options, :marshal)
|
49
|
+
threshold = with_fallback(options, :threshold)
|
50
|
+
|
51
|
+
dumped = deflate(marshal.dump(value), compress, threshold)
|
52
|
+
|
53
|
+
compose(dumped, marshal, compress)
|
22
54
|
end
|
23
55
|
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
56
|
+
# Parse a dumped value using the embedded options.
|
57
|
+
#
|
58
|
+
# @param [String] Option embedded string to load
|
59
|
+
# @return [String] The original dumped string, restored
|
60
|
+
#
|
61
|
+
# @example
|
62
|
+
#
|
63
|
+
# entity.load(dumped)
|
64
|
+
#
|
65
|
+
def load(string)
|
66
|
+
marshal, compress, value = decompose(string)
|
67
|
+
|
68
|
+
marshal.load(inflate(value, compress))
|
69
|
+
rescue TypeError, NoMethodError
|
70
|
+
string
|
32
71
|
end
|
33
72
|
|
34
|
-
|
35
|
-
|
73
|
+
# Composes a single byte comprised of the chosen serializer and compression
|
74
|
+
# options. The byte is formatted as:
|
75
|
+
#
|
76
|
+
# | 0000 | 0 | 000 |
|
77
|
+
#
|
78
|
+
# Where there are four unused bits, 1 compression bit, and 3 bits for the
|
79
|
+
# serializer. This allows up to 8 different serializers for marshaling.
|
80
|
+
#
|
81
|
+
# @param [String] String to prefix with flags
|
82
|
+
# @param [Module] The marshal module to be used
|
83
|
+
# @param [Boolean] Flag determining whether the value is compressed
|
84
|
+
# @return [String] The original string with a single byte prefixed
|
85
|
+
#
|
86
|
+
# @example Compose an option embedded string
|
87
|
+
#
|
88
|
+
# entity.compose(string, Marshal, false) => 0x1 + string
|
89
|
+
# entity.compose(string, JSON, true) => 0x10 + string
|
90
|
+
#
|
91
|
+
def compose(value, marshal, compress)
|
92
|
+
flags = serializers.assoc(marshal)
|
93
|
+
flags |= COMPRESSED_FLAG if compress
|
94
|
+
|
95
|
+
value.prepend([flags].pack('C'))
|
36
96
|
end
|
37
97
|
|
38
|
-
|
39
|
-
|
98
|
+
# Decompose an option embedded string into marshal, compression and value.
|
99
|
+
#
|
100
|
+
# @param [String] Option embedded string to
|
101
|
+
# @return [Array<Module, Boolean, String>] An array comprised of the
|
102
|
+
# marshal, compression flag, and the base string.
|
103
|
+
#
|
104
|
+
def decompose(string)
|
105
|
+
flags = string[0].unpack('C').first
|
106
|
+
|
107
|
+
if flags < 16
|
108
|
+
marshal = serializers.rassoc(flags & MARSHAL_FLAG)
|
109
|
+
compress = (flags & COMPRESSED_FLAG) != 0
|
110
|
+
|
111
|
+
[marshal, compress, string[1..-1]]
|
112
|
+
else
|
113
|
+
[@options[:marshal], @options[:compress], string]
|
114
|
+
end
|
40
115
|
end
|
41
116
|
|
42
117
|
private
|
43
118
|
|
44
|
-
def
|
45
|
-
|
119
|
+
def deflate(value, compress, threshold)
|
120
|
+
if compress && value.bytesize >= threshold
|
121
|
+
Zlib::Deflate.deflate(value)
|
122
|
+
else
|
123
|
+
value
|
124
|
+
end
|
125
|
+
end
|
126
|
+
|
127
|
+
def inflate(value, decompress)
|
128
|
+
if decompress
|
129
|
+
Zlib::Inflate.inflate(value)
|
130
|
+
else
|
131
|
+
value
|
132
|
+
end
|
133
|
+
rescue Zlib::Error
|
134
|
+
value
|
135
|
+
end
|
136
|
+
|
137
|
+
def serializers
|
138
|
+
Readthis.serializers
|
46
139
|
end
|
47
140
|
|
48
|
-
def
|
49
|
-
|
141
|
+
def with_fallback(options, key)
|
142
|
+
options.key?(key) ? options[key] : @options[key]
|
50
143
|
end
|
51
144
|
end
|
52
145
|
end
|
@@ -0,0 +1,100 @@
|
|
1
|
+
require 'json'
|
2
|
+
require 'readthis/passthrough'
|
3
|
+
|
4
|
+
module Readthis
|
5
|
+
SerializersFrozenError = Class.new(Exception)
|
6
|
+
SerializersLimitError = Class.new(Exception)
|
7
|
+
|
8
|
+
class Serializers
|
9
|
+
BASE_SERIALIZERS = {
|
10
|
+
Marshal => 0x1,
|
11
|
+
Passthrough => 0x2,
|
12
|
+
JSON => 0x3
|
13
|
+
}.freeze
|
14
|
+
|
15
|
+
SERIALIZER_LIMIT = 7
|
16
|
+
|
17
|
+
attr_reader :serializers, :inverted
|
18
|
+
|
19
|
+
# Creates a new Readthis::Serializers entity. No configuration is expected
|
20
|
+
# during initialization.
|
21
|
+
#
|
22
|
+
def initialize
|
23
|
+
reset!
|
24
|
+
end
|
25
|
+
|
26
|
+
# Append a new serializer. Up to 7 total serializers may be configured for
|
27
|
+
# any single application be configured for any single application. This
|
28
|
+
# limit is based on the number of bytes available in the option flag.
|
29
|
+
#
|
30
|
+
# @param [Module] Any object that responds to `dump` and `load`
|
31
|
+
# @return [self] Returns itself for possible chaining
|
32
|
+
#
|
33
|
+
# @example
|
34
|
+
#
|
35
|
+
# serializers = Readthis::Serializers.new
|
36
|
+
# serializers << Oj
|
37
|
+
#
|
38
|
+
def <<(serializer)
|
39
|
+
case
|
40
|
+
when serializers.frozen?
|
41
|
+
raise SerializersFrozenError
|
42
|
+
when serializers.length > SERIALIZER_LIMIT
|
43
|
+
raise SerializersLimitError
|
44
|
+
else
|
45
|
+
@serializers[serializer] = flags.max.succ
|
46
|
+
@inverted = @serializers.invert
|
47
|
+
end
|
48
|
+
|
49
|
+
self
|
50
|
+
end
|
51
|
+
|
52
|
+
# Freeze the serializers hash, preventing modification.
|
53
|
+
#
|
54
|
+
def freeze!
|
55
|
+
serializers.freeze
|
56
|
+
end
|
57
|
+
|
58
|
+
# Reset the instance back to the default state. Useful for cleanup during
|
59
|
+
# testing.
|
60
|
+
#
|
61
|
+
def reset!
|
62
|
+
@serializers = BASE_SERIALIZERS.dup
|
63
|
+
@inverted = @serializers.invert
|
64
|
+
end
|
65
|
+
|
66
|
+
# Find a flag by the marshal object.
|
67
|
+
#
|
68
|
+
# @param [Object] Look up a flag by object
|
69
|
+
# @return [Number] Corresponding flag for the marshal object
|
70
|
+
#
|
71
|
+
# @example
|
72
|
+
#
|
73
|
+
# serializers.assoc(Marshal) #=> 1
|
74
|
+
#
|
75
|
+
def assoc(marshal)
|
76
|
+
serializers[marshal]
|
77
|
+
end
|
78
|
+
|
79
|
+
# Find a marshal object by flag value.
|
80
|
+
#
|
81
|
+
# @param [Number] Flag to look up the marshal object by
|
82
|
+
# @return [Module] The marshal object
|
83
|
+
#
|
84
|
+
def rassoc(flag)
|
85
|
+
inverted[flag]
|
86
|
+
end
|
87
|
+
|
88
|
+
# The current list of marshal objects.
|
89
|
+
#
|
90
|
+
def marshals
|
91
|
+
serializers.keys
|
92
|
+
end
|
93
|
+
|
94
|
+
# The current list of flags.
|
95
|
+
#
|
96
|
+
def flags
|
97
|
+
serializers.values
|
98
|
+
end
|
99
|
+
end
|
100
|
+
end
|
data/lib/readthis/version.rb
CHANGED
data/spec/readthis/cache_spec.rb
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
require 'readthis
|
1
|
+
require 'readthis'
|
2
2
|
|
3
3
|
RSpec.describe Readthis::Cache do
|
4
4
|
let(:cache) { Readthis::Cache.new }
|
@@ -82,19 +82,52 @@ RSpec.describe Readthis::Cache do
|
|
82
82
|
end
|
83
83
|
end
|
84
84
|
|
85
|
+
describe 'serializers' do
|
86
|
+
after do
|
87
|
+
Readthis.serializers.reset!
|
88
|
+
end
|
89
|
+
|
90
|
+
it 'uses globally configured serializers' do
|
91
|
+
custom = Class.new do
|
92
|
+
def self.dump(_)
|
93
|
+
'dumped'
|
94
|
+
end
|
95
|
+
|
96
|
+
def self.load(_)
|
97
|
+
'dumped'
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
Readthis.serializers << custom
|
102
|
+
|
103
|
+
cache.write('customized', 'overwrite me', marshal: custom)
|
104
|
+
|
105
|
+
expect(cache.read('customized')).to include('dumped')
|
106
|
+
end
|
107
|
+
end
|
108
|
+
|
85
109
|
describe 'compression' do
|
86
|
-
it '
|
110
|
+
it 'roundtrips entries when compression is enabled' do
|
87
111
|
com_cache = Readthis::Cache.new(compress: true, compression_threshold: 8)
|
88
112
|
raw_cache = Readthis::Cache.new
|
89
113
|
value = 'enough text that it should be compressed'
|
90
114
|
|
91
115
|
com_cache.write('compressed', value)
|
92
116
|
|
93
|
-
expect(raw_cache.read('compressed')).not_to eq(value)
|
94
117
|
expect(com_cache.read('compressed')).to eq(value)
|
118
|
+
expect(raw_cache.read('compressed')).to eq(value)
|
119
|
+
end
|
120
|
+
|
121
|
+
it 'roundtrips entries with option overrides' do
|
122
|
+
cache = Readthis::Cache.new(compress: false)
|
123
|
+
value = 'enough text that it should be compressed'
|
124
|
+
|
125
|
+
cache.write('comp-round', value, marshal: JSON, compress: true, threshold: 8)
|
126
|
+
|
127
|
+
expect(cache.read('comp-round')).to eq(value)
|
95
128
|
end
|
96
129
|
|
97
|
-
it '
|
130
|
+
it 'roundtrips bulk entries when compression is enabled' do
|
98
131
|
cache = Readthis::Cache.new(compress: true, compression_threshold: 8)
|
99
132
|
value = 'also enough text to compress'
|
100
133
|
|
@@ -1,4 +1,4 @@
|
|
1
|
-
require 'readthis
|
1
|
+
require 'readthis'
|
2
2
|
require 'json'
|
3
3
|
|
4
4
|
RSpec.describe Readthis::Entity do
|
@@ -7,14 +7,21 @@ RSpec.describe Readthis::Entity do
|
|
7
7
|
string = 'some string'
|
8
8
|
entity = Readthis::Entity.new
|
9
9
|
|
10
|
-
expect(entity.dump(string)).to
|
10
|
+
expect(entity.dump(string)).to include(Marshal.dump(string))
|
11
11
|
end
|
12
12
|
|
13
13
|
it 'marshals using a custom marshaller' do
|
14
14
|
string = 'some string'
|
15
15
|
entity = Readthis::Entity.new(marshal: JSON)
|
16
16
|
|
17
|
-
expect(entity.dump(string)).to
|
17
|
+
expect(entity.dump(string)).to include(JSON.dump(string))
|
18
|
+
end
|
19
|
+
|
20
|
+
it 'overrides the marshaller' do
|
21
|
+
string = 'still some string'
|
22
|
+
entity = Readthis::Entity.new
|
23
|
+
|
24
|
+
expect(entity.dump(string, marshal: JSON)).to include(JSON.dump(string))
|
18
25
|
end
|
19
26
|
|
20
27
|
it 'applies compression when enabled' do
|
@@ -39,6 +46,22 @@ RSpec.describe Readthis::Entity do
|
|
39
46
|
expect(entity.load(string)).to eq(string)
|
40
47
|
end
|
41
48
|
|
49
|
+
it 'overrides the compression threshold' do
|
50
|
+
string = 'a' * 8
|
51
|
+
entity = Readthis::Entity.new(compress: true, threshold: 2)
|
52
|
+
dumped = entity.dump(string)
|
53
|
+
|
54
|
+
expect(entity.dump(string, threshold: 100)).not_to eq(dumped)
|
55
|
+
end
|
56
|
+
|
57
|
+
it 'overrides the compression option' do
|
58
|
+
string = 'a' * 8
|
59
|
+
entity = Readthis::Entity.new(compress: true, threshold: 2)
|
60
|
+
dumped = entity.dump(string)
|
61
|
+
|
62
|
+
expect(entity.dump(string, compress: false)).not_to eq(dumped)
|
63
|
+
end
|
64
|
+
|
42
65
|
it 'safely roundtrips nil values' do
|
43
66
|
entity = Readthis::Entity.new
|
44
67
|
|
@@ -49,26 +72,27 @@ RSpec.describe Readthis::Entity do
|
|
49
72
|
describe '#load' do
|
50
73
|
it 'unmarshals a value' do
|
51
74
|
object = { a: 1, b: '2' }
|
52
|
-
dumped = Marshal.dump(object)
|
53
75
|
entity = Readthis::Entity.new
|
76
|
+
dumped = entity.dump(object)
|
54
77
|
|
55
78
|
expect(entity.load(dumped)).to eq(object)
|
56
79
|
end
|
57
80
|
|
58
81
|
it 'uncompresses when compression is enabled' do
|
59
82
|
string = 'another one of those huge strings'
|
60
|
-
entity = Readthis::Entity.new(compress: true, threshold:
|
61
|
-
dumped =
|
62
|
-
|
63
|
-
compressed = entity.compress(dumped)
|
83
|
+
entity = Readthis::Entity.new(compress: true, threshold: 4)
|
84
|
+
dumped = entity.dump(dumped)
|
64
85
|
|
65
|
-
expect(entity.load(
|
86
|
+
expect(entity.load(dumped)).not_to eq(string)
|
66
87
|
end
|
67
88
|
|
68
|
-
it '
|
69
|
-
|
89
|
+
it 'uses the dumped value to define load options' do
|
90
|
+
value = [1, 2, 3]
|
91
|
+
custom = Readthis::Entity.new(marshal: JSON, compress: true)
|
92
|
+
general = Readthis::Entity.new(marshal: Marshal, compress: false)
|
93
|
+
dumped = custom.dump(value)
|
70
94
|
|
71
|
-
expect(
|
95
|
+
expect(general.load(dumped)).to eq(value)
|
72
96
|
end
|
73
97
|
|
74
98
|
it 'passes through the value when it fails to marshal' do
|
@@ -84,4 +108,47 @@ RSpec.describe Readthis::Entity do
|
|
84
108
|
expect { entity.load(dumped) }.not_to raise_error
|
85
109
|
end
|
86
110
|
end
|
111
|
+
|
112
|
+
describe '#compose' do
|
113
|
+
it 'prepends the string with a formatted marker' do
|
114
|
+
string = 'the quick brown fox'
|
115
|
+
marked = Readthis::Entity.new.compose(string, Marshal, true)
|
116
|
+
|
117
|
+
expect(marked[0]).not_to eq('t')
|
118
|
+
end
|
119
|
+
end
|
120
|
+
|
121
|
+
describe '#decompose' do
|
122
|
+
it 'returns extracted options and values' do
|
123
|
+
string = 'the quick brown fox'
|
124
|
+
entity = Readthis::Entity.new
|
125
|
+
marked = entity.compose(string.dup, JSON, true)
|
126
|
+
|
127
|
+
marshal, compress, value = entity.decompose(marked)
|
128
|
+
|
129
|
+
expect(marshal).to eq(JSON)
|
130
|
+
expect(compress).to eq(true)
|
131
|
+
expect(value).to eq(string)
|
132
|
+
end
|
133
|
+
|
134
|
+
it 'can reconstruct longer qualified module names' do
|
135
|
+
string = 'a' * 30
|
136
|
+
entity = Readthis::Entity.new
|
137
|
+
marked = entity.compose(string, Readthis::Passthrough, false)
|
138
|
+
|
139
|
+
marshal, _, _ = entity.decompose(marked)
|
140
|
+
|
141
|
+
expect(marshal).to eq(Readthis::Passthrough)
|
142
|
+
end
|
143
|
+
|
144
|
+
it 'returns the original string without a marker' do
|
145
|
+
string = 'the quick brown fox'
|
146
|
+
entity = Readthis::Entity.new
|
147
|
+
marshal, compress, value = entity.decompose(string)
|
148
|
+
|
149
|
+
expect(marshal).to eq(Marshal)
|
150
|
+
expect(compress).to eq(false)
|
151
|
+
expect(value).to eq(string)
|
152
|
+
end
|
153
|
+
end
|
87
154
|
end
|
@@ -0,0 +1,72 @@
|
|
1
|
+
require 'readthis/serializers'
|
2
|
+
|
3
|
+
RSpec.describe Readthis::Serializers do
|
4
|
+
CustomSerializer = Class.new
|
5
|
+
AnotherSerializer = Class.new
|
6
|
+
|
7
|
+
describe '#<<' do
|
8
|
+
it 'appends new serializers' do
|
9
|
+
serializers = Readthis::Serializers.new
|
10
|
+
|
11
|
+
serializers << CustomSerializer
|
12
|
+
|
13
|
+
expect(serializers.marshals).to include(CustomSerializer)
|
14
|
+
expect(serializers.flags).to eq((1..4).to_a)
|
15
|
+
end
|
16
|
+
|
17
|
+
it 'increments flags' do
|
18
|
+
serializers = Readthis::Serializers.new
|
19
|
+
serializers << CustomSerializer
|
20
|
+
serializers << AnotherSerializer
|
21
|
+
|
22
|
+
expect(serializers.flags).to eq((1..5).to_a)
|
23
|
+
end
|
24
|
+
|
25
|
+
it 'prevents more than seven serializers' do
|
26
|
+
serializers = Readthis::Serializers.new
|
27
|
+
|
28
|
+
expect {
|
29
|
+
10.times { serializers << Class.new }
|
30
|
+
}.to raise_error(Readthis::SerializersLimitError)
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
describe '#assoc' do
|
35
|
+
it 'looks up serializers by module' do
|
36
|
+
serializers = Readthis::Serializers.new
|
37
|
+
|
38
|
+
expect(serializers.assoc(Marshal)).to eq(0x1)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
describe '#rassoc' do
|
43
|
+
it 'inverts the current set of serializers' do
|
44
|
+
serializers = Readthis::Serializers.new
|
45
|
+
|
46
|
+
expect(serializers.rassoc(1)).to eq(Marshal)
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
describe '#freeze!' do
|
51
|
+
it 'does now allow appending after freeze' do
|
52
|
+
serializers = Readthis::Serializers.new
|
53
|
+
|
54
|
+
serializers.freeze!
|
55
|
+
|
56
|
+
expect {
|
57
|
+
serializers << CustomSerializer
|
58
|
+
}.to raise_error(Readthis::SerializersFrozenError)
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
describe '#reset!' do
|
63
|
+
it 'reverts back to the original set of serializers' do
|
64
|
+
serializers = Readthis::Serializers.new
|
65
|
+
|
66
|
+
serializers << Class.new
|
67
|
+
serializers.reset!
|
68
|
+
|
69
|
+
expect(serializers.marshals.length).to eq(3)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: readthis
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.0.pre.beta
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Parker Selbert
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-09-
|
11
|
+
date: 2015-09-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: redis
|
@@ -111,11 +111,14 @@ files:
|
|
111
111
|
- PERFORMANCE.md
|
112
112
|
- README.md
|
113
113
|
- Rakefile
|
114
|
+
- benchmarks/composing.rb
|
114
115
|
- benchmarks/compressed.rb
|
115
116
|
- benchmarks/driver.rb
|
117
|
+
- benchmarks/generic.rb
|
116
118
|
- benchmarks/marshalling.rb
|
117
119
|
- benchmarks/memory.rb
|
118
120
|
- benchmarks/multi.rb
|
121
|
+
- benchmarks/parsing.rb
|
119
122
|
- benchmarks/profile.rb
|
120
123
|
- bin/rspec
|
121
124
|
- lib/active_support/cache/readthis_store.rb
|
@@ -125,6 +128,7 @@ files:
|
|
125
128
|
- lib/readthis/expanders.rb
|
126
129
|
- lib/readthis/notifications.rb
|
127
130
|
- lib/readthis/passthrough.rb
|
131
|
+
- lib/readthis/serializers.rb
|
128
132
|
- lib/readthis/version.rb
|
129
133
|
- readthis.gemspec
|
130
134
|
- spec/readthis/cache_spec.rb
|
@@ -132,6 +136,8 @@ files:
|
|
132
136
|
- spec/readthis/expanders_spec.rb
|
133
137
|
- spec/readthis/notifications_spec.rb
|
134
138
|
- spec/readthis/passthrough_spec.rb
|
139
|
+
- spec/readthis/serializers_spec.rb
|
140
|
+
- spec/readthis_spec.rb
|
135
141
|
- spec/spec_helper.rb
|
136
142
|
homepage: https://github.com/sorentwo/readthis
|
137
143
|
licenses:
|
@@ -148,12 +154,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
148
154
|
version: '0'
|
149
155
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
150
156
|
requirements:
|
151
|
-
- - "
|
157
|
+
- - ">"
|
152
158
|
- !ruby/object:Gem::Version
|
153
|
-
version:
|
159
|
+
version: 1.3.1
|
154
160
|
requirements: []
|
155
161
|
rubyforge_project:
|
156
|
-
rubygems_version: 2.4.5
|
162
|
+
rubygems_version: 2.4.5.1
|
157
163
|
signing_key:
|
158
164
|
specification_version: 4
|
159
165
|
summary: Pooled active support compliant caching with redis
|
@@ -163,4 +169,6 @@ test_files:
|
|
163
169
|
- spec/readthis/expanders_spec.rb
|
164
170
|
- spec/readthis/notifications_spec.rb
|
165
171
|
- spec/readthis/passthrough_spec.rb
|
172
|
+
- spec/readthis/serializers_spec.rb
|
173
|
+
- spec/readthis_spec.rb
|
166
174
|
- spec/spec_helper.rb
|