mneme 0.5.2 → 0.6.0
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +4 -0
- data/config.rb +2 -0
- data/lib/mneme.rb +2 -3
- data/mneme.gemspec +1 -1
- metadata +1 -1
data/README.md
CHANGED
@@ -23,6 +23,8 @@ For more details: [Mneme: Scalable Duplicate Filtering Service](http://www.igvit
|
|
23
23
|
config['hashes'] = 7 # number of times each key will be hashed
|
24
24
|
config['seed'] = 30 # seed value for the hash function
|
25
25
|
|
26
|
+
config['pool'] = 2 # number of concurrent Redis connections
|
27
|
+
|
26
28
|
To learn more about Bloom filter configuration: [Scalable Datasets: Bloom Filters in Ruby](http://www.igvita.com/2008/12/27/scalable-datasets-bloom-filters-in-ruby/)
|
27
29
|
|
28
30
|
## Launching mneme
|
@@ -49,6 +51,8 @@ That's it! You now have a mneme web service running on port 9000. Let's try quer
|
|
49
51
|
- The speed of storing a new key is: *O(number of BF hashes) - aka, O(1)*
|
50
52
|
- The speed of retrieving a key is: *O(number of filters * number of BF hashes) - aka, O(1)*
|
51
53
|
|
54
|
+
- Sample ab benchmarks for single key lookup: [https://gist.github.com/895326](https://gist.github.com/895326)
|
55
|
+
|
52
56
|
Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positives are possible, but false negatives are not. Because we are using Redis as a backend, in-memory store for the filters, there is some extra overhead. Sample memory requirements:
|
53
57
|
|
54
58
|
- 1.0% error rate for 1M items, 10 bits/item: 2.5 mb
|
data/config.rb
CHANGED
data/lib/mneme.rb
CHANGED
@@ -12,8 +12,6 @@ class Mneme < Goliath::API
|
|
12
12
|
include Mnemosyne::Helper
|
13
13
|
plugin Mnemosyne::Sweeper
|
14
14
|
|
15
|
-
use ::Rack::Reloader, 0 if Goliath.dev?
|
16
|
-
|
17
15
|
use Goliath::Rack::Params
|
18
16
|
use Goliath::Rack::DefaultMimeType
|
19
17
|
use Goliath::Rack::Formatters::JSON
|
@@ -85,7 +83,8 @@ class Mneme < Goliath::API
|
|
85
83
|
hashes: config['hashes']
|
86
84
|
}
|
87
85
|
|
88
|
-
|
86
|
+
pool = config['pool'] || 1
|
87
|
+
env[Goliath::Constants::CONFIG][period] = EventMachine::Synchrony::ConnectionPool.new(size: pool) do
|
89
88
|
BloomFilter::Redis.new(opts)
|
90
89
|
end
|
91
90
|
end
|
data/mneme.gemspec
CHANGED