mneme 0.5.2 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +4 -0
- data/config.rb +2 -0
- data/lib/mneme.rb +2 -3
- data/mneme.gemspec +1 -1
- metadata +1 -1
data/README.md
CHANGED
@@ -23,6 +23,8 @@ For more details: [Mneme: Scalable Duplicate Filtering Service](http://www.igvit
|
|
23
23
|
config['hashes'] = 7 # number of times each key will be hashed
|
24
24
|
config['seed'] = 30 # seed value for the hash function
|
25
25
|
|
26
|
+
config['pool'] = 2 # number of concurrent Redis connections
|
27
|
+
|
26
28
|
To learn more about Bloom filter configuration: [Scalable Datasets: Bloom Filters in Ruby](http://www.igvita.com/2008/12/27/scalable-datasets-bloom-filters-in-ruby/)
|
27
29
|
|
28
30
|
## Launching mneme
|
@@ -49,6 +51,8 @@ That's it! You now have a mneme web service running on port 9000. Let's try quer
|
|
49
51
|
- The speed of storing a new key is: *O(number of BF hashes) - aka, O(1)*
|
50
52
|
- The speed of retrieving a key is: *O(number of filters * number of BF hashes) - aka, O(1)*
|
51
53
|
|
54
|
+
- Sample ab benchmarks for single key lookup: [https://gist.github.com/895326](https://gist.github.com/895326)
|
55
|
+
|
52
56
|
Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positives are possible, but false negatives are not. Because we are using Redis as a backend, in-memory store for the filters, there is some extra overhead. Sample memory requirements:
|
53
57
|
|
54
58
|
- 1.0% error rate for 1M items, 10 bits/item: 2.5 mb
|
data/config.rb
CHANGED
data/lib/mneme.rb
CHANGED
@@ -12,8 +12,6 @@ class Mneme < Goliath::API
|
|
12
12
|
include Mnemosyne::Helper
|
13
13
|
plugin Mnemosyne::Sweeper
|
14
14
|
|
15
|
-
use ::Rack::Reloader, 0 if Goliath.dev?
|
16
|
-
|
17
15
|
use Goliath::Rack::Params
|
18
16
|
use Goliath::Rack::DefaultMimeType
|
19
17
|
use Goliath::Rack::Formatters::JSON
|
@@ -85,7 +83,8 @@ class Mneme < Goliath::API
|
|
85
83
|
hashes: config['hashes']
|
86
84
|
}
|
87
85
|
|
88
|
-
|
86
|
+
pool = config['pool'] || 1
|
87
|
+
env[Goliath::Constants::CONFIG][period] = EventMachine::Synchrony::ConnectionPool.new(size: pool) do
|
89
88
|
BloomFilter::Redis.new(opts)
|
90
89
|
end
|
91
90
|
end
|
data/mneme.gemspec
CHANGED