joshbuddy-tokyo_cache_cow 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
data/README.rdoc ADDED
@@ -0,0 +1,88 @@
1
+ = Tokyo Cache Cow
2
+
3
+ Tokyo Cache Cow is MemCache protocol speaking cache server. It offers the ability to delete keys based on a substring.
4
+
5
+ == Motivation
6
+
7
+ Cache sweepers in rails do not currently operate with memcache because the memcache server itself does not support key matching removal operations. After reading http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/ and seeing the performance characteristics of that database, I thought I'd give it a go. Event Machine does the heavy lifting on the network end. Performance is currently comparable to memcached.
8
+
9
+ == Prerequisites
10
+
11
+ You'll need the eventmachine gem installed. As well, you'll have to install Tokyo Cabinet itself (available at http://tokyocabinet.sourceforge.net/index.html) and the Tokyo Cabinet Ruby bindings (available at http://tokyocabinet.sourceforge.net/rubypkg/)
12
+
13
+ == Example (using the rails client under script/console)
14
+
15
+ Lets write four keys: <i>other_key</i>, <i>test_key</i>, <i>test_key2</i> and <i>test_key3</i>.
16
+
17
+ >> Rails.cache.write('other_key', 'other_value')
18
+ => true
19
+ >> Rails.cache.write('test_key', 'test_value')
20
+ => true
21
+ >> Rails.cache.write('test_key2', 'test_value2')
22
+ => true
23
+ >> Rails.cache.write('test_key3', 'test_value3')
24
+ => true
25
+
26
+ Read back <i>test_key</i> and make sure life is still good.
27
+
28
+ >> Rails.cache.read('test_key2')
29
+ => "test_value2"
30
+
31
+ But lets delete <i>test_key2</i> for fun.
32
+
33
+ >> Rails.cache.delete('test_key2')
34
+ => true
35
+
36
+ Confirm that <i>test_key2</i> is really gone.
37
+
38
+ >> Rails.cache.read('test_key2')
39
+ => nil
40
+
41
+ .. but our other keys (namely, <i>test_key</i>) are just fine, thank you.
42
+
43
+ >> Rails.cache.read('test_key')
44
+ => "test_value"
45
+
46
+ .. lets nuke *EVERYTHING* with <i>test_key</i> in it though.
47
+
48
+ >> Rails.cache.delete_matched('test_key')
49
+ => true
50
+
51
+ Now <i>test_key</i> and <i>test_key3</i> are both nuked.
52
+
53
+ >> Rails.cache.read('test_key')
54
+ => nil
55
+ >> Rails.cache.read('test_key3')
56
+ => nil
57
+
58
+ But <i>other_key</i> is still peachy.
59
+
60
+ >> Rails.cache.read('other_key')
61
+ => "other_value"
62
+
63
+
64
+ == Usage
65
+
66
+ === Server
67
+
68
+
69
+ >> ruby -rubygems runner.rb --help
70
+
71
+ --------------
72
+
73
+ Usage: runner.rb [options]
74
+ -p, --port[OPTIONAL] Port (default: 11211)
75
+ -h, --host[OPTIONAL] Host (default: 0.0.0.0)
76
+ --help Show this help message.
77
+
78
+ === Client
79
+
80
+ Any standard memcache client will do, however, there is a special initialize for Rails to enable delete_matched functionality within the built-in memcache client there. To install this into rails:
81
+
82
+ script/plugin install git://github.com/joshbuddy/tokyo-cache-cow.git
83
+
84
+ == Caveats
85
+
86
+ Right now there is no compression on disk. Things could get big, but Tokyo Cabinet does support various compression schemes, so exposing that to the runner should be trivial. Potentially performance could degrade after time, but I have yet to seriously investigate if thats the case.
87
+
88
+ Feel the moo.
data/Rakefile ADDED
@@ -0,0 +1,32 @@
1
+ require 'rubygems'
2
+ require 'lib/tokyo_cache_cow'
3
+
4
+ begin
5
+ require 'jeweler'
6
+ Jeweler::Tasks.new do |s|
7
+ s.name = "tokyo_cache_cow"
8
+ s.description = s.summary = ""
9
+ s.email = "joshbuddy@gmail.com"
10
+ s.homepage = "http://github.com/joshbuddy/tokyo_cache_cow"
11
+ s.authors = ["Joshua Hull"]
12
+ s.files = FileList["[A-Z]*", "{lib,spec,rails,bin}/**/*"]
13
+ s.executables = ['tokyo_cache_cow']
14
+ end
15
+ rescue LoadError
16
+ puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
17
+ end
18
+
19
+ require 'spec'
20
+ require 'spec/rake/spectask'
21
+
22
+ task :spec => 'spec:all'
23
+ namespace(:spec) do
24
+ Spec::Rake::SpecTask.new(:all) do |t|
25
+ t.spec_opts ||= []
26
+ t.spec_opts << "-rubygems"
27
+ t.spec_opts << "--options" << "spec/spec.opts"
28
+ t.spec_files = FileList['spec/**/*_spec.rb']
29
+ end
30
+
31
+ end
32
+
data/VERSION.yml ADDED
@@ -0,0 +1,4 @@
1
+ ---
2
+ :patch: 1
3
+ :major: 0
4
+ :minor: 0
@@ -0,0 +1,47 @@
1
+ require 'eventmachine'
2
+ require 'optparse'
3
+
4
+ $:.unshift File.join(File.dirname(__FILE__), '..')
5
+ require 'lib/tokyo_cache_cow'
6
+
7
+ options = {:port => '11211', :host => '0.0.0.0', :provider => 'tokyo_cabinet', :file => '/tmp/cache'}
8
+ OptionParser.new do |opts|
9
+ opts.banner = "Usage: runner.rb [options]"
10
+
11
+ opts.on("-p[OPTIONAL]", "--port", "Port (default: #{options[:port]})") do |v|
12
+ options[:port] = v
13
+ end
14
+
15
+ opts.on("-h[OPTIONAL]", "--host", "Host (default: #{options[:host]})") do |v|
16
+ options[:host] = v
17
+ end
18
+
19
+ opts.on("-c[OPTIONAL]", "--cache-provider", "Cache provider (default: #{options[:provider]})") do |v|
20
+ options[:provider] = v
21
+ end
22
+
23
+ opts.on("-f[OPTIONAL]", "--file", "File (default: #{options[:file]})") do |v|
24
+ options[:file] = v
25
+ end
26
+
27
+ opts.on_tail("-h", "--help", "Show this help message.") { puts opts; exit }
28
+
29
+ end.parse!
30
+
31
+ trap("INT") { EM.stop; puts "moooooooo ya later" }
32
+
33
+ if File.exists?(File.join(File.dirname(__FILE__), options[:provider]))
34
+ require File.join(File.dirname(__FILE__), options[:provider])
35
+ cache = $cache
36
+ else
37
+ require 'lib/tokyo_cache_cow/cache'
38
+ cache = TokyoCacheCow::Cache.const_get("#{options[:provider]}_memcache".split('_').map{|e| e.capitalize}.join.to_sym).new(options[:file])
39
+ end
40
+
41
+ puts "Starting the tokyo cache cow"
42
+ EM.run do
43
+ EM.start_server(options[:host], options[:port], TokyoCacheCow::Server) do |c|
44
+ c.cache = cache
45
+ end
46
+ end
47
+
@@ -0,0 +1,15 @@
1
+ class TokyoCacheCow
2
+ class Cache
3
+ class Base
4
+
5
+ def process_time(time)
6
+ time = case time
7
+ when 0, nil: 0
8
+ when 1..2592000: (Time.now.to_i + time.to_i)
9
+ else time
10
+ end
11
+ end
12
+
13
+ end
14
+ end
15
+ end
@@ -0,0 +1,111 @@
1
+ require 'yaml'
2
+ require 'fileutils'
3
+ require 'cgi'
4
+ class TokyoCacheCow
5
+ class Cache
6
+ class FileMemcache < Base
7
+
8
+ def process_time(time)
9
+ time = case time
10
+ when 0, nil: 0
11
+ when 1..2592000: (Time.now.to_i + time.to_i)
12
+ else time
13
+ end
14
+ end
15
+
16
+ def initialize(path)
17
+ @path = path
18
+ FileUtils.rm_rf(@path)
19
+ FileUtils.mkdir_p(@path)
20
+ end
21
+
22
+ def time_expired?(time)
23
+ time.to_i == 0 ? false : time < Time.now.to_i
24
+ end
25
+
26
+ def generate_data_hash(value, options)
27
+ {
28
+ :value => value,
29
+ :expires => process_time(options[:expires] || 0),
30
+ :flags => options[:flags] || 0
31
+ }
32
+ end
33
+
34
+ def add(key, value, options = {})
35
+ if (data = get_raw(key)) && !time_expired?(data[:expired])
36
+ nil
37
+ else
38
+ set(key, value, options) and true
39
+ end
40
+ end
41
+
42
+ def delete_match(key)
43
+ FileUtils.rm Dir.glob(File.join(@path, "*#{CGI.escape(key)}*"))
44
+ end
45
+
46
+ def replace(key, value, options = {})
47
+ set(key, value, options) if File.exists?(path_for_key(key))
48
+ end
49
+
50
+ def append(key, val)
51
+ if data = get(key)
52
+ data[:value] << val
53
+ set_raw(key, data)
54
+ true
55
+ else
56
+ false
57
+ end
58
+ end
59
+
60
+ def incr(key, value)
61
+ if data = get(key)
62
+ new_value = data[:value].to_i + value
63
+ set(key, new_value.to_s, :expires => data[:expires], :flags => data[:flags])
64
+ new_value
65
+ end
66
+ end
67
+
68
+ def decr(key, value)
69
+ incr(key, -value)
70
+ end
71
+
72
+ def flush_all
73
+ FileUtils.rm(Dir.glob(File.join(@path, '*'))) and true
74
+ end
75
+
76
+ def delete(key, expires = nil)
77
+ FileUtils.rm(Dir.glob(path_for_key(key))) and true
78
+ end
79
+
80
+ def path_for_key(key)
81
+ File.join(@path, CGI.escape(key))
82
+ end
83
+
84
+ def get_raw(key)
85
+ File.exists?(path_for_key(key)) ? YAML::load( File.open( path_for_key(key) ) ) : nil
86
+ end
87
+
88
+ def set_raw(key, data)
89
+ File.open(path_for_key(key), 'w') do |out|
90
+ YAML.dump(data, out)
91
+ end
92
+ end
93
+
94
+ def get(key)
95
+ if data = get_raw(key)
96
+ if time_expired?(data[:expires])
97
+ delete(key)
98
+ nil
99
+ else
100
+ data
101
+ end
102
+ end
103
+ end
104
+
105
+ def set(key, value, options = {})
106
+ set_raw(key, generate_data_hash(value, options))
107
+ end
108
+
109
+ end
110
+ end
111
+ end
@@ -0,0 +1,92 @@
1
+ class TokyoCacheCow
2
+ class Cache
3
+ class HashMemcache < Base
4
+
5
+ def process_time(time)
6
+ time = case time
7
+ when 0, nil: 0
8
+ when 1..2592000: (Time.now.to_i + time.to_i)
9
+ else time
10
+ end
11
+ end
12
+
13
+ def initialize(file)
14
+ @cache = {}
15
+ end
16
+
17
+ def time_expired?(time)
18
+ time.to_i == 0 ? false : time < Time.now.to_i
19
+ end
20
+
21
+ def generate_data_hash(value, options)
22
+ {
23
+ :value => value,
24
+ :expires => process_time(options[:expires] || 0),
25
+ :flags => options[:flags] || 0
26
+ }
27
+ end
28
+
29
+ def add(key, value, options = {})
30
+ if (data = @cache[key]) && !time_expired?(data[:expired])
31
+ nil
32
+ else
33
+ set(key, value, options) and true
34
+ end
35
+ end
36
+
37
+ def delete_match(key)
38
+ @cache.delete_if{ |key, value| key.index(key) }
39
+ end
40
+
41
+ def replace(key, value, options = {})
42
+ set(key, value, options) if @cache.key?(key)
43
+ end
44
+
45
+ def append(key, val)
46
+ if data = get(key)
47
+ data[:value] << val
48
+ @cache[key] = data
49
+ true
50
+ else
51
+ false
52
+ end
53
+ end
54
+
55
+ def incr(key, value)
56
+ if data = get(key)
57
+ new_value = data[:value].to_i + value
58
+ set(key, new_value.to_s, :expires => data[:expires], :flags => data[:flags])
59
+ new_value
60
+ end
61
+ end
62
+
63
+ def decr(key, value)
64
+ incr(key, -value)
65
+ end
66
+
67
+ def flush_all
68
+ @cache.clear and true
69
+ end
70
+
71
+ def delete(key, expires = nil)
72
+ @cache.delete(key)
73
+ end
74
+
75
+ def get(key)
76
+ if data = @cache[key]
77
+ if time_expired?(data[:expires])
78
+ @cache.delete(key)
79
+ nil
80
+ else
81
+ data
82
+ end
83
+ end
84
+ end
85
+
86
+ def set(key, value, options = {})
87
+ @cache[key] = generate_data_hash(value, options)
88
+ end
89
+
90
+ end
91
+ end
92
+ end
@@ -0,0 +1,119 @@
1
+ require 'tokyocabinet'
2
+
3
+ class TokyoCacheCow
4
+ class Cache
5
+ class TokyoCabinetMemcache < Base
6
+
7
+ include TokyoCabinet
8
+
9
+ def process_time(time)
10
+ time = case time_i = Integer(time)
11
+ when 0: '0'
12
+ when 1..2592000: (Time.now.to_i + time_i).to_s
13
+ else time
14
+ end
15
+ end
16
+
17
+ def flush_all
18
+ @cache.vanish
19
+ end
20
+
21
+ def get(key, cas = nil)
22
+ if (data = @cache.get(key)) && data['expired']
23
+ nil
24
+ elsif data
25
+ expires = data['expires'] && data['expires'].to_i
26
+ flags = data['flags'] && data['flags'].to_i
27
+ if expires != 0 && expires < Time.now.to_i
28
+ delete(key)
29
+ nil
30
+ else
31
+ { :value => data['value'], :expires => expires, :flags => flags }
32
+ end
33
+ end
34
+ end
35
+
36
+ def incr(key, value)
37
+ if data = get(key)
38
+ new_value = data[:value].to_i + value
39
+ set(key, new_value.to_s, :expires => data[:expires], :flags => data[:flags])
40
+ new_value
41
+ end
42
+ end
43
+
44
+ def decr(key, value)
45
+ incr(key, -value)
46
+ end
47
+
48
+ def append(key, val)
49
+ if data = get(key)
50
+ data[:value] << val
51
+ set(key, data[:value], :expires => data[:expires], :flags => data[:flags])
52
+ true
53
+ else
54
+ false
55
+ end
56
+ end
57
+
58
+ def prepend(key, val)
59
+ if data = @cache.get(key)
60
+ data[:data][0,0] = val
61
+ put(key, data[:value], :expires => data[:expires], :flags => data[:flags])
62
+ true
63
+ else
64
+ false
65
+ end
66
+ end
67
+
68
+ def generate_data_hash(value, options)
69
+ expires = options[:expires] && options[:expires].to_s || '0'
70
+ flags = options[:flags] && options[:flags].to_s || '0'
71
+ { 'value' => value, 'expires' => process_time(expires), 'flags' => flags }
72
+ end
73
+
74
+ def time_expired?(time)
75
+ time == '0' ? false : time.to_i < Time.now.to_i
76
+ end
77
+
78
+
79
+ def set(key, value, options = {})
80
+ @cache.put(key, generate_data_hash(value, options))
81
+ end
82
+
83
+ def add(key, value, options = {})
84
+ if data = @cache.get(key)
85
+ time_expired?(data[:expired]) ?
86
+ nil : @cache.putkeep(key, generate_data_hash(value, options))
87
+ else
88
+ @cache.putkeep(key, generate_data_hash(value, options))
89
+ end
90
+ end
91
+
92
+ def replace(key, value, options = {})
93
+ get(key) ? @cache.put(key, generate_data_hash(value, options)) : nil
94
+ end
95
+
96
+ def delete(key, opts = {})
97
+ if opts[:expires] && opts[:expires] != 0
98
+ @cache.put(key, {'expired' => process_time(opts[:expires])})
99
+ else
100
+ @cache.out(key)
101
+ end
102
+ end
103
+
104
+ def delete_match(match)
105
+ q = TDBQRY.new(@cache)
106
+ q.addcond('', TDBQRY::QCSTRINC, match)
107
+ q.search
108
+ q.searchout
109
+ end
110
+
111
+ def initialize(file)
112
+ @cache = TDB::new # hash database
113
+ @cache.open(file, TDB::OWRITER | TDB::OCREAT | TDB::OTRUNC)
114
+ @cache.setxmsiz(500_000_000)
115
+ end
116
+
117
+ end
118
+ end
119
+ end
@@ -0,0 +1,11 @@
1
+ $:.unshift(File.dirname(__FILE__))
2
+
3
+ class TokyoCacheCow
4
+ class Cache
5
+ self.autoload :Base, 'cache/base'
6
+ self.autoload :FileMemcache, 'cache/file_memcache'
7
+ self.autoload :TokyoCabinetMemcache, 'cache/tokyo_cabinet_memcache'
8
+ self.autoload :HashMemcache, 'cache/hash_memcache'
9
+ end
10
+ end
11
+
@@ -0,0 +1,13 @@
1
+ class TokyoCacheCow
2
+
3
+ autoload :TokyoCabinetMemcache, 'lib/tokyo_cache_cow/tokyo_cabinet_memcache'
4
+
5
+ class Providers
6
+
7
+ def self.provide_cache
8
+ #require 'lib/tokyo_cache_cow/tokyo_cabinet_memcache'
9
+ @@cache ||= TokyoCacheCow::TokyoCabinetMemcache.new('/tmp/tcc')
10
+ end
11
+
12
+ end
13
+ end
@@ -0,0 +1,176 @@
1
+ require 'strscan'
2
+ require 'eventmachine'
3
+
4
+ class TokyoCacheCow
5
+ class Server < EventMachine::Connection
6
+
7
+ Terminator = "\r\n"
8
+
9
+ #set
10
+ SetCommand = /(\S+) +(\d+) +(\d+) +(\d+)( +noreply)?/
11
+ CasCommand = /(\S+) +(\d+) +(\d+) +(\d+) +(\d+)( +noreply)?/
12
+
13
+ StoredReply = "STORED\r\n"
14
+ NotStoredReply = "NOT_STORED\r\n"
15
+ ExistsReply = "EXISTS\r\n"
16
+ NotFoundReply = "NOT_FOUND\r\n"
17
+
18
+ GetValueReply = "VALUE %s %d %d\r\n"
19
+ CasValueReply = "VALUE %d %d %d %d\r\n"
20
+ EndReply = "END\r\n"
21
+
22
+ #delete
23
+ DeleteCommand = /(\S+) *(noreply)?/
24
+ DeleteWithTimeoutCommand = /(\S+) +(\d+) *(noreply)?/
25
+
26
+ DeletedReply = "DELETED\r\n"
27
+ NotDeletedReply = "NOT_DELETED\r\n"
28
+
29
+ #delete_match
30
+ DeleteMatchCommand = /(\S+)( +noreply)?/
31
+
32
+ #Increment/Decrement
33
+ IncrementDecrementCommand = /(\S+) +(\d+)( +noreply)?/
34
+
35
+ ValueReply = "%d\r\n"
36
+
37
+ #errors
38
+ OK = "OK\r\n"
39
+ Error = "ERROR\r\n"
40
+ ClientError = "CLIENT_ERROR %s\r\n"
41
+ ServerError = "SERVER_ERROR %s\r\n"
42
+
43
+ TerminatorRegex = /\r\n/
44
+
45
+ attr_accessor :cache
46
+
47
+ def send_client_error(message = "invalid arguments")
48
+ send_data(ClientError % message.to_s)
49
+ end
50
+
51
+ def send_server_error(message = "there was a problem")
52
+ send_data(ServerError % message.to_s)
53
+ end
54
+
55
+ def validate_key(key)
56
+ if key.nil?
57
+ send_data(ClientError % "key cannot be blank")
58
+ elsif key && key.index(' ')
59
+ send_data(ClientError % "key cannot contain spaces")
60
+ nil
61
+ elsif key.size > 250
62
+ send_data(ClientError % "key must be less than 250 characters")
63
+ nil
64
+ else
65
+ key
66
+ end
67
+ end
68
+
69
+ def receive_data(data)
70
+ ss = StringScanner.new(data)
71
+
72
+ while part = ss.scan_until(TerminatorRegex)
73
+ begin
74
+ command_argument_separator_index = part.index(/\s/)
75
+ command = part[0, command_argument_separator_index]
76
+ args = part[command_argument_separator_index + 1, part.size - command_argument_separator_index - 3]
77
+ case command
78
+ when 'get', 'gets'
79
+ keys = args.split(/\s+/)
80
+ keys.each do |k|
81
+ next unless validate_key(k)
82
+ if data = @cache.get(k)
83
+ if command == 'get'
84
+ send_data(GetValueReply % [k, data[:flags], data[:value].size])
85
+ else
86
+ send_data(CasValueReply % [k, data[:flags], data[:value].size, data[:value].hash])
87
+ end
88
+ send_data(data[:value])
89
+ send_data(Terminator)
90
+ end
91
+ end
92
+ send_data(EndReply)
93
+ when 'set'
94
+ SetCommand.match(args) or (send_client_error and next)
95
+ (key, flags, expires, bytes, noreply) = [$1, Integer($2), Integer($3), Integer($4), !$5.nil?]
96
+ next unless validate_key(key)
97
+ send_data(@cache.set(key, ss.rest[0, bytes.to_i], :flags => flags, :expires => expires) ?
98
+ StoredReply : NotStoredReply)
99
+ ss.pos += bytes + 2
100
+ when 'add'
101
+ SetCommand.match(args)
102
+ (key, flags, expires, bytes, noreply) = [$1, $2.to_i, $3.to_i, $4, !$5.nil?]
103
+ send_data(@cache.add(key, ss.rest[0, bytes.to_i], :flags => flags, :expires => expires) ?
104
+ StoredReply : NotStoredReply)
105
+ ss.pos += bytes + 2
106
+ when 'replace'
107
+ SetCommand.match(args)
108
+ (key, flags, expires, bytes, noreply) = [$1, $2.to_i, $3.to_i, $4, !$5.nil?]
109
+ send_data(@cache.replace(key, ss.rest[0, bytes.to_i], :flags => flags, :expires => expires) ?
110
+ StoredReply : NotStoredReply)
111
+ ss.pos += bytes + 2
112
+ when 'append'
113
+ SetCommand.match(args)
114
+ (key, flags, expires, bytes, noreply) = [$1, $2.to_i, $3.to_i, $4, !$5.nil?]
115
+ send_data(@cache.append(key, ss.rest[0, bytes.to_i], :flags => flags, :expires => expires) ?
116
+ StoredReply : NotStoredReply)
117
+ ss.pos += bytes + 2
118
+ when 'prepend'
119
+ SetCommand.match(args)
120
+ (key, flags, expires, bytes, noreply) = [$1, $2.to_i, $3.to_i, $4, !$5.nil?]
121
+ send_data(@cache.prepend(key, ss.rest[0, bytes.to_i], :flags => flags, :expires => expires) ?
122
+ StoredReply : NotStoredReply)
123
+ ss.pos += bytes + 2
124
+ when 'cas'
125
+ # do something
126
+ when 'delete'
127
+ case args
128
+ when DeleteWithTimeoutCommand
129
+ (key, timeout, noreply) = [$1.chomp, $2, !$3.nil?]
130
+ next unless validate_key(key)
131
+ send_data(@cache.delete_expire(key, timeout) ?
132
+ DeletedReply : NotDeletedReply)
133
+ when DeleteCommand
134
+ (key, noreply) = [$1.chomp, !$2.nil?]
135
+ next unless validate_key(key)
136
+ send_data @cache.delete(key) ?
137
+ DeletedReply : NotDeletedReply
138
+ end
139
+ when 'delete_match'
140
+ DeleteMatchCommand.match(args)
141
+ (key, noreply) = [$1.chomp, !$2.nil?]
142
+ next unless validate_key(key)
143
+ @cache.delete_match(key)
144
+ send_data(DeletedReply)
145
+ when 'incr', 'decr'
146
+ IncrementDecrementCommand.match(args)
147
+ (key, value, noreply) = [$1, $2.to_i, !$3.nil?]
148
+ next unless validate_key(key)
149
+ send_data(if d = @cache.get(key)
150
+ value = -value if command == 'decr'
151
+ d['data'] = (val = (d['data'].to_i + value)).to_s
152
+ @cache.put(key, d)
153
+ ValueReply % val
154
+ else
155
+ NotFoundReply
156
+ end)
157
+ when 'stats'
158
+ send_data(Error)
159
+ when 'flush_all'
160
+ send_data(@cache.flush_all ? OK : Error)
161
+ when 'version'
162
+ send_data(Error)
163
+ when 'quit'
164
+ close_connection_after_writing
165
+ else
166
+ send_data(Error)
167
+ end
168
+ rescue
169
+ send_server_error($!)
170
+ end
171
+ end
172
+
173
+ end
174
+
175
+ end
176
+ end
@@ -0,0 +1,4 @@
1
+ $:.unshift File.dirname(__FILE__)
2
+
3
+ require 'tokyo_cache_cow/server'
4
+ require 'tokyo_cache_cow/providers'
data/rails/init.rb ADDED
@@ -0,0 +1,43 @@
1
+ module ::ActiveSupport
2
+ module Cache
3
+ class MemCacheStore < Store
4
+
5
+ def delete_matched(matcher, options = nil) # :nodoc:
6
+ super
7
+ response = @data.delete_match(matcher)
8
+ response == Response::DELETED
9
+ rescue MemCache::MemCacheError => e
10
+ logger.error("MemCacheError (#{e}): #{e.message}")
11
+ false
12
+ end
13
+
14
+ end
15
+ end
16
+ end
17
+
18
+ class ::MemCache
19
+
20
+ def delete_match(key)
21
+ @mutex.lock if @multithread
22
+
23
+ raise MemCacheError, "No active servers" unless active?
24
+ cache_key = make_cache_key key
25
+ server = get_server_for_key cache_key
26
+
27
+ sock = server.socket
28
+ raise MemCacheError, "No connection to server" if sock.nil?
29
+
30
+ begin
31
+ sock.write "delete_match #{cache_key}\r\n"
32
+ result = sock.gets
33
+ raise_on_error_response! result
34
+ result
35
+ rescue SocketError, SystemCallError, IOError => err
36
+ server.close
37
+ raise MemCacheError, err.message
38
+ end
39
+ ensure
40
+ @mutex.unlock if @multithread
41
+ end
42
+
43
+ end
@@ -0,0 +1,153 @@
1
+ require 'benchmark'
2
+ require 'lib/tokyo_cache_cow/cache'
3
+
4
+ {
5
+ 'tokyo cabinet memcache' => TokyoCacheCow::Cache::TokyoCabinetMemcache.new('/tmp/tcc1'),
6
+ 'hash memcache' => TokyoCacheCow::Cache::HashMemcache.new(nil),
7
+ 'file memcache' => TokyoCacheCow::Cache::FileMemcache.new('/tmp/filecache')
8
+ }.each do |name, cache|
9
+
10
+ describe name do
11
+
12
+ before(:each) do
13
+ cache.flush_all.should == true
14
+ end
15
+
16
+ it "should add" do
17
+ cache.get('added_key').should == nil
18
+ cache.add('added_key','zig').should == true
19
+ cache.get('added_key')[:value].should == 'zig'
20
+ cache.add('added_key','ziglar')
21
+ cache.get('added_key')[:value].should == 'zig'
22
+ end
23
+
24
+ it "should put & get" do
25
+ 100.times do |i|
26
+ cache.set("/blog/show/#{i}","this is a big ol' blog post!!! #{i}")
27
+ end
28
+
29
+ 100.times do |i|
30
+ cache.get("/blog/show/#{i}")[:value].should == "this is a big ol' blog post!!! #{i}"
31
+ end
32
+ end
33
+
34
+ it "should delete" do
35
+ cache.set("key-set-123","you should never see me")
36
+ cache.get("key-set-123")[:value].should == "you should never see me"
37
+ cache.delete("key-set-123")
38
+ cache.get("key-set-123").should == nil
39
+ end
40
+
41
+ it "should delete (with expiry)" do
42
+ cache.set('delete-with-expiry', 'hillbillies')
43
+ cache.get('delete-with-expiry')[:value].should == 'hillbillies'
44
+ cache.delete('delete-with-expiry', :expires => 3)
45
+ cache.get('delete-with-expiry').should == nil
46
+ cache.replace('delete-with-expiry', 'more hillbillies')
47
+ cache.get('delete-with-expiry').should == nil
48
+ sleep(5)
49
+ cache.get('delete-with-expiry').should == nil
50
+ cache.set('delete-with-expiry', 'more hillbillies')
51
+ cache.get('delete-with-expiry')[:value].should == 'more hillbillies'
52
+ end
53
+
54
+ it "should delete (with expiry) and set again" do
55
+ cache.set('delete-with-expiry', 'hillbillies')
56
+ cache.get('delete-with-expiry')[:value].should == 'hillbillies'
57
+ cache.delete('delete-with-expiry', 3)
58
+ cache.get('delete-with-expiry').should == nil
59
+ cache.set('delete-with-expiry', 'more hillbillies')
60
+ cache.get('delete-with-expiry')[:value].should == 'more hillbillies'
61
+ sleep(5)
62
+ cache.get('delete-with-expiry')[:value].should == 'more hillbillies'
63
+ end
64
+
65
+ it "should delete_match" do
66
+ 100.times do
67
+ cache.set("asd/qwe/zxc/10","you should never see me")
68
+ cache.set("asd/qwe/zxc/20","you should never see me")
69
+ cache.set("asd/qwe/zxc/30","you should never see me")
70
+ cache.set("asd/qwe/zxc/40","you should never see me")
71
+ cache.set("asd/qwe/zxc/11","you should never see me")
72
+ cache.set("asd/qwe/zxc/21","you should never see me")
73
+ cache.set("asd/qwe/zxc/31","you should never see me")
74
+ cache.set("asd/qwe/zxc/41","you should never see me")
75
+ cache.set("asd/qwe/zxc/12","you should never see me")
76
+ cache.set("asd/qwe/zxc/22","you should never see me")
77
+ cache.set("asd/qwe/zxc/32","you should never see me")
78
+ cache.set("asd/qwe/zxc/42","you should never see me")
79
+ cache.set("asd/qwe/zxc/101","you should never see me")
80
+ cache.set("asd/qwe/zxc/201","you should never see me")
81
+ cache.set("asd/qwe/zxc/301","you should never see me")
82
+ cache.set("asd/qwe/zxc/401","you should never see me")
83
+ cache.set("asd/qwe/zxc/111","you should never see me")
84
+ cache.set("asd/qwe/zxc/211","you should never see me")
85
+ cache.set("asd/qwe/zxc/311","you should never see me")
86
+ cache.set("asd/qwe/zxc/411","you should never see me")
87
+ cache.set("asd/qwe/zxc/121","you should never see me")
88
+ cache.set("asd/qwe/zxc/221","you should never see me")
89
+ cache.set("asd/qwe/zxc/321","you should never see me")
90
+ cache.set("asd/qwe/zxc/421","you should never see me")
91
+ cache.delete_match("asd/qwe/zxc")
92
+ cache.get("asd/qwe/zxc/40").should == nil
93
+ cache.get("asd/qwe/zxc/30").should == nil
94
+ cache.get("asd/qwe/zxc/20").should == nil
95
+ cache.get("asd/qwe/zxc/10").should == nil
96
+ cache.get("asd/qwe/zxc/41").should == nil
97
+ cache.get("asd/qwe/zxc/31").should == nil
98
+ cache.get("asd/qwe/zxc/21").should == nil
99
+ cache.get("asd/qwe/zxc/11").should == nil
100
+ cache.get("asd/qwe/zxc/42").should == nil
101
+ cache.get("asd/qwe/zxc/32").should == nil
102
+ cache.get("asd/qwe/zxc/22").should == nil
103
+ cache.get("asd/qwe/zxc/12").should == nil
104
+ cache.get("asd/qwe/zxc/401").should == nil
105
+ cache.get("asd/qwe/zxc/301").should == nil
106
+ cache.get("asd/qwe/zxc/201").should == nil
107
+ cache.get("asd/qwe/zxc/101").should == nil
108
+ cache.get("asd/qwe/zxc/411").should == nil
109
+ cache.get("asd/qwe/zxc/311").should == nil
110
+ cache.get("asd/qwe/zxc/211").should == nil
111
+ cache.get("asd/qwe/zxc/111").should == nil
112
+ cache.get("asd/qwe/zxc/421").should == nil
113
+ cache.get("asd/qwe/zxc/321").should == nil
114
+ cache.get("asd/qwe/zxc/221").should == nil
115
+ cache.get("asd/qwe/zxc/121").should == nil
116
+ end
117
+ end
118
+
119
+ it "should expire" do
120
+ cache.set("expiring key","you should never see me", :expires => 1)
121
+ sleep(3)
122
+ cache.get("expiring key").should == nil
123
+ end
124
+
125
+ it "should replace" do
126
+ cache.replace("replacing-key", "newkey")
127
+ cache.get("replacing-key").should == nil
128
+ cache.set("replacing-key", "oldkey")
129
+ cache.replace("replacing-key", "newkey")
130
+ cache.get("replacing-key")[:value].should == 'newkey'
131
+ end
132
+
133
+ it "should append" do
134
+ cache.set("appending-key", "test1")
135
+ cache.get("appending-key")[:value].should == "test1"
136
+ cache.append("appending-key", "test2")
137
+ cache.get("appending-key")[:value].should == "test1test2"
138
+ end
139
+
140
+ it "should incr" do
141
+ cache.set("incr-key", 123)
142
+ cache.incr("incr-key", 20).should == 143
143
+ cache.get("incr-key")[:value].should == '143'
144
+ end
145
+
146
+ it "should decr" do
147
+ cache.set("decr-key", 123)
148
+ cache.decr("decr-key", 20).should == 103
149
+ cache.get("decr-key")[:value].should == '103'
150
+ end
151
+
152
+ end
153
+ end
@@ -0,0 +1,24 @@
1
+ require 'rubygems'
2
+ require 'memcached'
3
+ require 'benchmark'
4
+
5
+ cache = Memcached.new('127.0.0.1:11211')
6
+
7
+ describe 'memcache server' do
8
+
9
+ before(:each) do
10
+ cache.flush
11
+ end
12
+
13
+ it "should get & set" do
14
+ cache.set('asd', "qweqweasd")
15
+ cache.get('asd').should == "qweqweasd"
16
+ end
17
+
18
+ it "should delete" do
19
+ cache.set('asd', 'qwe')
20
+ cache.delete('asd')
21
+ proc {cache.get('asd')}.should raise_error Memcached::NotFound
22
+ end
23
+
24
+ end
data/spec/spec.opts ADDED
@@ -0,0 +1,8 @@
1
+ -rubygems
2
+ --colour
3
+ --format
4
+ specdoc
5
+ --loadby
6
+ mtime
7
+ --reverse
8
+ --backtrace
metadata ADDED
@@ -0,0 +1,71 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: joshbuddy-tokyo_cache_cow
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ platform: ruby
6
+ authors:
7
+ - Joshua Hull
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+
12
+ date: 2009-04-21 00:00:00 -07:00
13
+ default_executable: tokyo_cache_cow
14
+ dependencies: []
15
+
16
+ description: ""
17
+ email: joshbuddy@gmail.com
18
+ executables:
19
+ - tokyo_cache_cow
20
+ extensions: []
21
+
22
+ extra_rdoc_files:
23
+ - README.rdoc
24
+ files:
25
+ - Rakefile
26
+ - README.rdoc
27
+ - VERSION.yml
28
+ - lib/tokyo_cache_cow
29
+ - lib/tokyo_cache_cow/cache
30
+ - lib/tokyo_cache_cow/cache/base.rb
31
+ - lib/tokyo_cache_cow/cache/file_memcache.rb
32
+ - lib/tokyo_cache_cow/cache/hash_memcache.rb
33
+ - lib/tokyo_cache_cow/cache/tokyo_cabinet_memcache.rb
34
+ - lib/tokyo_cache_cow/cache.rb
35
+ - lib/tokyo_cache_cow/providers.rb
36
+ - lib/tokyo_cache_cow/server.rb
37
+ - lib/tokyo_cache_cow.rb
38
+ - spec/cache_spec.rb
39
+ - spec/server_spec.rb
40
+ - spec/spec.opts
41
+ - rails/init.rb
42
+ - bin/tokyo_cache_cow
43
+ has_rdoc: true
44
+ homepage: http://github.com/joshbuddy/tokyo_cache_cow
45
+ post_install_message:
46
+ rdoc_options:
47
+ - --inline-source
48
+ - --charset=UTF-8
49
+ require_paths:
50
+ - lib
51
+ required_ruby_version: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: "0"
56
+ version:
57
+ required_rubygems_version: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: "0"
62
+ version:
63
+ requirements: []
64
+
65
+ rubyforge_project:
66
+ rubygems_version: 1.2.0
67
+ signing_key:
68
+ specification_version: 2
69
+ summary: ""
70
+ test_files: []
71
+