mneme 0.5.2 → 0.6.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (5) hide show
  1. data/README.md +4 -0
  2. data/config.rb +2 -0
  3. data/lib/mneme.rb +2 -3
  4. data/mneme.gemspec +1 -1
  5. metadata +1 -1
data/README.md CHANGED
@@ -23,6 +23,8 @@ For more details: [Mneme: Scalable Duplicate Filtering Service](http://www.igvit
23
23
  config['hashes'] = 7 # number of times each key will be hashed
24
24
  config['seed'] = 30 # seed value for the hash function
25
25
 
26
+ config['pool'] = 2 # number of concurrent Redis connections
27
+
26
28
  To learn more about Bloom filter configuration: [Scalable Datasets: Bloom Filters in Ruby](http://www.igvita.com/2008/12/27/scalable-datasets-bloom-filters-in-ruby/)
27
29
 
28
30
  ## Launching mneme
@@ -49,6 +51,8 @@ That's it! You now have a mneme web service running on port 9000. Let's try quer
49
51
  - The speed of storing a new key is: *O(number of BF hashes) - aka, O(1)*
50
52
  - The speed of retrieving a key is: *O(number of filters * number of BF hashes) - aka, O(1)*
51
53
 
54
+ - Sample ab benchmarks for single key lookup: [https://gist.github.com/895326](https://gist.github.com/895326)
55
+
52
56
  Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positives are possible, but false negatives are not. Because we are using Redis as a backend, in-memory store for the filters, there is some extra overhead. Sample memory requirements:
53
57
 
54
58
  - 1.0% error rate for 1M items, 10 bits/item: 2.5 mb
data/config.rb CHANGED
@@ -6,3 +6,5 @@ config['size'] = 1000
6
6
  config['bits'] = 10
7
7
  config['hashes'] = 7
8
8
  config['seed'] = 30
9
+
10
+ config['pool'] = 2
@@ -12,8 +12,6 @@ class Mneme < Goliath::API
12
12
  include Mnemosyne::Helper
13
13
  plugin Mnemosyne::Sweeper
14
14
 
15
- use ::Rack::Reloader, 0 if Goliath.dev?
16
-
17
15
  use Goliath::Rack::Params
18
16
  use Goliath::Rack::DefaultMimeType
19
17
  use Goliath::Rack::Formatters::JSON
@@ -85,7 +83,8 @@ class Mneme < Goliath::API
85
83
  hashes: config['hashes']
86
84
  }
87
85
 
88
- env[Goliath::Constants::CONFIG][period] = EventMachine::Synchrony::ConnectionPool.new(size: 1) do
86
+ pool = config['pool'] || 1
87
+ env[Goliath::Constants::CONFIG][period] = EventMachine::Synchrony::ConnectionPool.new(size: pool) do
89
88
  BloomFilter::Redis.new(opts)
90
89
  end
91
90
  end
@@ -3,7 +3,7 @@ $:.push File.expand_path("../lib", __FILE__)
3
3
 
4
4
  Gem::Specification.new do |s|
5
5
  s.name = "mneme"
6
- s.version = "0.5.2"
6
+ s.version = "0.6.0"
7
7
  s.platform = Gem::Platform::RUBY
8
8
  s.authors = ["Ilya Grigorik"]
9
9
  s.email = ["ilya@igvita.com"]
metadata CHANGED
@@ -2,7 +2,7 @@
2
2
  name: mneme
3
3
  version: !ruby/object:Gem::Version
4
4
  prerelease:
5
- version: 0.5.2
5
+ version: 0.6.0
6
6
  platform: ruby
7
7
  authors:
8
8
  - Ilya Grigorik