fluent-plugin-redis-store 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 0c2db458d39cf72f93460102e6421b9d1b270bbc
4
+ data.tar.gz: c801090f87a8d00bc142016ed871d7a04a4f847b
5
+ SHA512:
6
+ metadata.gz: 8aa1ec479a471ac96a86632b90ed6560447a632579768cd13b7d0bffc0fef8ac9e91ed65bdf67aa1f7f6ab2ac4fa952c63d2ea5fbb08279c9e21a902d976825b
7
+ data.tar.gz: b51ea99f33bd87b5510b967dc22ab21c92540a355af8814cbc62b02812acc50258de0fa9a762b3c9c0cd23e7f87f39978ced799d2f53446b25e5ba71dffc6c1d
data/AUTHORS ADDED
@@ -0,0 +1,2 @@
1
+ moaikids
2
+ HANAI Tohru aka pokehanai
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
@@ -0,0 +1,157 @@
1
+ Redis Output Plugin For fluentd
2
+ ===============================
3
+ [Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
4
+
5
+ [Fluentd]: http://fluentd.org/
6
+ [Redis]: http://redis.io/
7
+
8
+ Background
9
+ ----------
10
+
11
+ This is folked project from [fluent-plugin-redisstore][].
12
+
13
+ [fluent-plugin-redisstore]: https://github.com/moaikids/fluent-plugin-redisstore
14
+
15
+ Features
16
+ --------
17
+
18
+ #### Supported Redis commands
19
+
20
+ Currently the plugin supports following Redis commands:
21
+
22
+ - **set** by `string` type (of the plugin)
23
+ - **lpush**/**rpush** by `list` type
24
+ - **sadd** by `set` type
25
+ - **zadd** by `zset` type
26
+ - **publish** by `publish` type
27
+
28
+ #### Supported _value_ format
29
+
30
+ - plain(as is)
31
+ - JSON
32
+ - [MessagePack](http://msgpack.org/)
33
+
34
+ #### _key_ string for Redis storage
35
+
36
+ Redis commands require _key_ and _value_.
37
+ For _key_, the plugin supports either way;
38
+
39
+ 1. Specify a fixed key.
40
+ You can do this simply using `key` option in td-agent configuration file.
41
+
42
+ ```apache
43
+ type redis_store
44
+ key userdata
45
+ ```
46
+
47
+ 2. Lookup a key string in every event data by a lookup path.
48
+ If event data have structured data like
49
+
50
+ ```javascript
51
+ { "user": { "name": "Kei" } }
52
+ ```
53
+
54
+ and you want to use each name of user, you can use `key_path` option.
55
+
56
+ ```apache
57
+ type redis_store
58
+ key_path user.name
59
+ ```
60
+
61
+ With the above data, `Kei` will be a _key_.
62
+
63
+ In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
64
+
65
+ ```apache
66
+ type redis_store
67
+ key_path user.name
68
+ key_prefix ouruser.
69
+ key_suffix .accesslog
70
+ ```
71
+
72
+ With the previous data, _key_ will be `outuser.Kei.accesslog`.
73
+
74
+ #### _value_ data for Redis storage
75
+
76
+ To determine what _value_ in every event data to be srtored, you have two options;
77
+
78
+ 1. Store extracted data in event data, by a lookup path with `value_path` option.
79
+ It works like `key_path`.
80
+ 2. Store whole data.
81
+ This is default behavior. To do it, simply omit `value_path` option.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-redis-store
87
+
88
+ Configuration
89
+ -------------
90
+
91
+ ### Redis connection
92
+
93
+ | Key | Type | Required? | Default | Description |
94
+ | :---- | :----- | :---------- | :----------------------- | :------------ |
95
+ | `url` | string | Optional | redis://127.0.0.1:6379/0 | URL to Redis server |
96
+ | `port` | int | Optional | 6379 | port number of Redis server |
97
+ | `password` | string | Optional | | password for Redis connection |
98
+ | `path` | string | Optional | | To connect via Unix socket, try '/tmp/redis.sock' |
99
+ | `db` | int | Optional | 0 | DB number of Redis |
100
+ | `timeout` | float | Optional | 5.0 | connection timeout in seconds |
101
+
102
+ ### common options for storages
103
+
104
+ | Key | Type | Default | Description |
105
+ | :---- | :----- | :----------------------- | :------------ |
106
+ | `key` | string | | Fixed _key_ used to store(publish) in Redis |
107
+ | `key_path` | string | | path to lookup for _key_ in the event data |
108
+ | `key_prefix` | string | | prefix of _key_ |
109
+ | `key_suffix` | string | | suffix of _key_ |
110
+ | `value_path` | string | (whole event data) | path to lookup for _value_ in the event data |
111
+ | `store_type` | string | zset | `string`/`list`/`set`/`zset`/`publish` |
112
+ | `format_type` | string | plain | format type for _value_ (`plain`/`json`/`msgpack`) |
113
+ | `key_expire` | int | -1 | If set, the key will be expired in specified seconds |
114
+
115
+ Note: either `key` or `key_path` is required.
116
+
117
+ ### `string` storage specific options
118
+
119
+ | Key | Type | Default | Description |
120
+ | :---- | :----- | :----------------------- | :------------ |
121
+ | `type` | string | | Fixed _key_ used to store(publish) in Redis |
122
+ No more options than common options.
123
+
124
+ ### `list` storage specific options
125
+
126
+ | Key | Type | Default | Description |
127
+ | :---- | :----- | :----------------------- | :------------ |
128
+ | `order` | string | asc | `asc`: **rpush**, `desc`: **lpush** |
129
+
130
+ ### `set` storage specific options
131
+
132
+ No more options than common options.
133
+
134
+ ### `zset` storage specific options
135
+
136
+ | Key | Type | Default | Description |
137
+ | :---- | :----- | :----------------------- | :------------ |
138
+ | `score_path` | string | (_time_ of log event) | path to lookup for _score_ in the event data |
139
+ | `value_expire` | int | | value expiration in seconds |
140
+
141
+ If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
142
+ based on *timestamp* and it deletes expired _members_ every after new event data arrives.
143
+
144
+ ### `publish` storage specific options
145
+
146
+ No more options than common options.
147
+
148
+
149
+ Copyright
150
+ ---------
151
+
152
+ Copyright (c) 2013 moaikids
153
+ Copyright (c) 2014 HANAI Tohru
154
+
155
+ License
156
+ -------
157
+ Apache License, Version 2.0
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+
4
+ require 'rake/testtask'
5
+ Rake::TestTask.new(:test) do |test|
6
+ test.libs << 'lib' << 'test'
7
+ test.pattern = 'test/**/test_*.rb'
8
+ test.verbose = true
9
+ end
10
+
11
+ task :default => :test
@@ -0,0 +1,21 @@
1
+ # -*- encoding: utf-8 -*-
2
+ Gem::Specification.new do |gem|
3
+ gem.name = "fluent-plugin-redis-store"
4
+ gem.email = "hanai@pokelabo.co.jp"
5
+ gem.version = "0.1.0"
6
+ gem.authors = ["moaikids", "HANAI Tohru aka pokehanai"]
7
+ gem.licenses = ["Apache License Version 2.0"]
8
+ gem.summary = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
9
+ gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
10
+ gem.homepage = "https://github.com/pokehanai/fluent-plugin-redis-store"
11
+
12
+ gem.files = `git ls-files`.split($\)
13
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
14
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
15
+ gem.require_paths = ["lib"]
16
+
17
+ gem.add_development_dependency "rake"
18
+ gem.add_development_dependency "test-unit"
19
+ gem.add_runtime_dependency "fluentd"
20
+ gem.add_runtime_dependency "redis"
21
+ end
@@ -0,0 +1,229 @@
1
+ module Fluent
2
+ class RedisStoreOutput < BufferedOutput
3
+ Fluent::Plugin.register_output('redis_store', self)
4
+
5
+ # redis connection
6
+ config_param :host, :string, :default => '127.0.0.1'
7
+ config_param :port, :integer, :default => 6379
8
+ config_param :path, :string, :default => nil
9
+ config_param :password, :string, :default => nil
10
+ config_param :db, :integer, :default => 0
11
+ config_param :timeout, :float, :default => 5.0
12
+
13
+ # redis command and parameters
14
+ config_param :format_type, :string, :default => 'json'
15
+ config_param :store_type, :string, :default => 'zset'
16
+ config_param :key_prefix, :string, :default => ''
17
+ config_param :key_suffix, :string, :default => ''
18
+ config_param :key, :string, :default => nil
19
+ config_param :key_path, :string, :default => nil
20
+ config_param :score_path, :string, :default => nil
21
+ config_param :value_path, :string, :default => ''
22
+ config_param :key_expire, :integer, :default => -1
23
+ config_param :value_expire, :integer, :default => -1
24
+ config_param :value_length, :integer, :default => -1
25
+ config_param :order, :string, :default => 'asc'
26
+
27
+ def initialize
28
+ super
29
+ require 'redis'
30
+ require 'msgpack'
31
+ end
32
+
33
+ def configure(conf)
34
+ super
35
+
36
+ if @key_path == nil and @key == nil
37
+ raise Fluent::ConfigError, "either key_path or key is required"
38
+ end
39
+ end
40
+
41
+ def start
42
+ super
43
+ if @path
44
+ @redis = Redis.new(:path => @path, :password => @parsword,
45
+ :timeout => @timeout, :thread_safe => true, :db => @db)
46
+ else
47
+ @redis = Redis.new(:host => @host, :port => @port, :password => @parsword,
48
+ :timeout => @timeout, :thread_safe => true, :db => @db)
49
+ end
50
+ end
51
+
52
+ def shutdown
53
+ @redis.quit
54
+ end
55
+
56
+ def format(tag, time, record)
57
+ [tag, time, record].to_msgpack
58
+ end
59
+
60
+ def write(chunk)
61
+ @redis.pipelined {
62
+ chunk.open { |io|
63
+ begin
64
+ MessagePack::Unpacker.new(io).each { |message|
65
+ begin
66
+ (tag, time, record) = message
67
+ case @store_type
68
+ when 'zset'
69
+ operation_for_zset(record, time)
70
+ when 'set'
71
+ operation_for_set(record)
72
+ when 'list'
73
+ operation_for_list(record)
74
+ when 'string'
75
+ operation_for_string(record)
76
+ when 'publish'
77
+ operation_for_publish(record)
78
+ end
79
+ rescue NoMethodError => e
80
+ puts e
81
+ end
82
+ }
83
+ rescue EOFError
84
+ # EOFError always occured when reached end of chunk.
85
+ end
86
+ }
87
+ }
88
+ end
89
+
90
+ def operation_for_zset(record, time)
91
+ key = get_key_from(record)
92
+ value = get_value_from(record)
93
+ score = get_score_from(record, time)
94
+ @redis.zadd key, score, value
95
+
96
+ set_key_expire key
97
+ if 0 < @value_expire
98
+ now = Time.now.to_i
99
+ @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
100
+ end
101
+ if 0 < @value_length
102
+ script = generate_zremrangebyrank_script(key, @value_length, @order)
103
+ @redis.eval script
104
+ end
105
+ end
106
+
107
+ def operation_for_set(record)
108
+ key = get_key_from(record)
109
+ value = get_value_from(record)
110
+ @redis.sadd key, value
111
+ set_key_expire key
112
+ end
113
+
114
+ def operation_for_list(record)
115
+ key = get_key_from(record)
116
+ value = get_value_from(record)
117
+
118
+ if @order == 'asc'
119
+ @redis.rpush key, value
120
+ else
121
+ @redis.lpush key, value
122
+ end
123
+ set_key_expire key
124
+ if 0 < @value_length
125
+ script = generate_ltrim_script(key, @value_length, @order)
126
+ @redis.eval script
127
+ end
128
+ end
129
+
130
+ def operation_for_string(record)
131
+ key = get_key_from(record)
132
+ value = get_value_from(record)
133
+ @redis.set key, value
134
+
135
+ set_key_expire key
136
+ end
137
+
138
+ def operation_for_publish(record)
139
+ key = get_key_from(record)
140
+ value = get_value_from(record)
141
+ @redis.publish key, value
142
+ end
143
+
144
+ def generate_zremrangebyrank_script(key, maxlen, order)
145
+ script = "local key = '" + key.to_s + "'\n"
146
+ script += "local maxlen = " + maxlen.to_s + "\n"
147
+ script += "local order ='" + order.to_s + "'\n"
148
+ script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
149
+ script += "if len > maxlen then\n"
150
+ script += " if order == 'asc' then\n"
151
+ script += " local l = len - maxlen\n"
152
+ script += " if l >= 0 then\n"
153
+ script += " return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
154
+ script += " end\n"
155
+ script += " else\n"
156
+ script += " return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
157
+ script += " end\n"
158
+ script += "end\n"
159
+ return script
160
+ end
161
+
162
+ def generate_ltrim_script(key, maxlen, order)
163
+ script = "local key = '" + key.to_s + "'\n"
164
+ script += "local maxlen = " + maxlen.to_s + "\n"
165
+ script += "local order ='" + order.to_s + "'\n"
166
+ script += "local len = tonumber(redis.call('LLEN', key))\n"
167
+ script += "if len > maxlen then\n"
168
+ script += " if order == 'asc' then\n"
169
+ script += " local l = len - maxlen\n"
170
+ script += " return redis.call('LTRIM', key, l, -1)\n"
171
+ script += " else\n"
172
+ script += " return redis.call('LTRIM', key, 0, maxlen - 1)\n"
173
+ script += " end\n"
174
+ script += "end\n"
175
+ return script
176
+ end
177
+
178
+ def traverse(data, key)
179
+ val = data
180
+ key.split('.').each{ |k|
181
+ if val.has_key?(k)
182
+ val = val[k]
183
+ else
184
+ return nil
185
+ end
186
+ }
187
+ return val
188
+ end
189
+
190
+ def get_key_from(record)
191
+ if @key
192
+ k = @key
193
+ else
194
+ k = traverse(record, @key_path).to_s
195
+ end
196
+ key = @key_prefix + k + @key_suffix
197
+
198
+ raise Fluent::ConfigError, "key is empty" if key == ''
199
+ key
200
+ end
201
+
202
+ def get_value_from(record)
203
+ value = traverse(record, @value_path)
204
+ case @format_type
205
+ when 'json'
206
+ value.to_json
207
+ when 'msgpack'
208
+ value.to_msgpack
209
+ else
210
+ value
211
+ end
212
+ end
213
+
214
+ def get_score_from(record, time)
215
+ if @score_path
216
+ traverse(record, @score_path)
217
+ else
218
+ time
219
+ end
220
+ end
221
+
222
+ def set_key_expire(key)
223
+ if 0 < @key_expire
224
+ @redis.expire key, @key_expire
225
+ end
226
+ end
227
+
228
+ end
229
+ end
@@ -0,0 +1,28 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ begin
4
+ Bundler.setup(:default, :development)
5
+ rescue Bundler::BundlerError => e
6
+ $stderr.puts e.message
7
+ $stderr.puts "Run `bundle install` to install missing gems"
8
+ exit e.status_code
9
+ end
10
+ require 'test/unit'
11
+
12
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
14
+ require 'fluent/test'
15
+ unless ENV.has_key?('VERBOSE')
16
+ nulllogger = Object.new
17
+ nulllogger.instance_eval {|obj|
18
+ def method_missing(method, *args)
19
+ # pass
20
+ end
21
+ }
22
+ $log = nulllogger
23
+ end
24
+
25
+ require 'fluent/plugin/out_redis_store'
26
+
27
+ class Test::Unit::TestCase
28
+ end
@@ -0,0 +1,367 @@
1
+ require 'helpers'
2
+
3
+ require 'redis'
4
+
5
+ $channel = nil
6
+ $message = nil
7
+
8
+ class Redis
9
+ def initialize(options = {})
10
+ end
11
+
12
+ def pipelined
13
+ yield self
14
+ end
15
+
16
+ def set(key, message)
17
+ $command = :set
18
+ $key = key
19
+ $message = message
20
+ end
21
+
22
+ def rpush(key, message)
23
+ $command = :rpush
24
+ $key = key
25
+ $message = message
26
+ end
27
+
28
+ def lpush(key, message)
29
+ $command = :lpush
30
+ $key = key
31
+ $message = message
32
+ end
33
+
34
+ def sadd(key, message)
35
+ $command = :sadd
36
+ $key = key
37
+ $message = message
38
+ end
39
+
40
+ def zadd(key, score, message)
41
+ $command = :zadd
42
+ $key = key
43
+ $score = score
44
+ $message = message
45
+ end
46
+
47
+ def expire(key, ttl)
48
+ $expire_key = key
49
+ $ttl = ttl
50
+ end
51
+
52
+ def publish(channel, message)
53
+ $command = :publish
54
+ $channel = channel
55
+ $message = message
56
+ end
57
+
58
+ def quit
59
+ end
60
+ end
61
+
62
+ class RedisStoreOutputTest < Test::Unit::TestCase
63
+ def setup
64
+ Fluent::Test.setup
65
+ end
66
+
67
+ def create_driver(conf)
68
+ Fluent::Test::BufferedOutputTestDriver.new(Fluent::RedisStoreOutput).configure(conf)
69
+ end
70
+
71
+ def test_configure_defaults
72
+ config = %[
73
+ key_path a
74
+ score_path b
75
+ ]
76
+ d = create_driver(config)
77
+ assert_equal "127.0.0.1", d.instance.host
78
+ assert_equal 6379, d.instance.port
79
+ assert_equal nil, d.instance.path
80
+ assert_equal nil, d.instance.password
81
+ assert_equal 0, d.instance.db
82
+ assert_equal 5.0, d.instance.timeout
83
+ assert_equal 'json', d.instance.format_type
84
+ assert_equal '', d.instance.key_prefix
85
+ assert_equal '', d.instance.key_suffix
86
+ assert_equal 'zset', d.instance.store_type
87
+ assert_equal 'a', d.instance.key_path
88
+ assert_equal nil, d.instance.key
89
+ assert_equal 'b', d.instance.score_path
90
+ assert_equal '', d.instance.value_path
91
+ assert_equal -1, d.instance.key_expire
92
+ assert_equal -1, d.instance.value_expire
93
+ assert_equal -1, d.instance.value_length
94
+ assert_equal 'asc', d.instance.order
95
+ end
96
+
97
+ def test_configure_host_port_db
98
+ config = %[
99
+ host 192.168.2.3
100
+ port 9999
101
+ password abc
102
+ db 3
103
+ timeout 7
104
+ key a
105
+ score_path b
106
+ ]
107
+ d = create_driver(config)
108
+ assert_equal "192.168.2.3", d.instance.host
109
+ assert_equal 9999, d.instance.port
110
+ assert_equal nil, d.instance.path
111
+ assert_equal 'abc', d.instance.password
112
+ assert_equal 3, d.instance.db
113
+ assert_equal 7.0, d.instance.timeout
114
+ assert_equal nil, d.instance.key_path
115
+ assert_equal 'a', d.instance.key
116
+ end
117
+
118
+ def test_configure_path
119
+ config = %[
120
+ path /tmp/foo.sock
121
+ key a
122
+ score_path b
123
+ ]
124
+ d = create_driver(config)
125
+ assert_equal "/tmp/foo.sock", d.instance.path
126
+ end
127
+
128
+ def test_configure_exception
129
+ assert_raise(Fluent::ConfigError) do
130
+ create_driver(%[])
131
+ end
132
+ end
133
+
134
+ # def test_write
135
+ # d = create_driver(CONFIG1)
136
+ #
137
+ # time = Time.parse("2011-01-02 13:14:15 UTC").to_i
138
+ # d.emit({ "foo" => "bar" }, time)
139
+ # d.run
140
+ #
141
+ # assert_equal "test", $channel
142
+ # assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
143
+ # end
144
+
145
+ def get_time
146
+ Time.parse("2011-01-02 13:14:15 UTC").to_i
147
+ end
148
+
149
+ # it should return whole message
150
+ def test_omit_value_path
151
+ config = %[
152
+ format_type plain
153
+ store_type string
154
+ key_path user
155
+ ]
156
+ d = create_driver(config)
157
+ message = {
158
+ 'user' => 'george',
159
+ 'stat' => { 'attack' => 7 }
160
+ }
161
+ $ttl = nil
162
+ d.emit(message, get_time)
163
+ d.run
164
+
165
+ assert_equal "george", $key
166
+ assert_equal message, $message
167
+ assert_equal nil, $ttl
168
+ end
169
+
170
+ def test_key_value_paths
171
+ config = %[
172
+ format_type plain
173
+ store_type string
174
+ key_path user.name
175
+ value_path stat.attack
176
+ key_expire 3
177
+ ]
178
+ d = create_driver(config)
179
+ message = {
180
+ 'user' => { 'name' => 'george' },
181
+ 'stat' => { 'attack' => 7 }
182
+ }
183
+ $ttl = nil
184
+ d.emit(message, get_time)
185
+ d.run
186
+
187
+ assert_equal "george", $key
188
+ assert_equal 7, $message
189
+ assert_equal 3, $ttl
190
+ end
191
+
192
+ def test_json
193
+ config = %[
194
+ format_type json
195
+ store_type string
196
+ key_path user
197
+ ]
198
+ d = create_driver(config)
199
+ message = {
200
+ 'user' => 'george',
201
+ 'stat' => { 'attack' => 7 }
202
+ }
203
+ $ttl = nil
204
+ d.emit(message, get_time)
205
+ d.run
206
+
207
+ assert_equal "george", $key
208
+ assert_equal message.to_json, $message
209
+ end
210
+
211
+ def test_msgpack
212
+ config = %[
213
+ format_type msgpack
214
+ store_type string
215
+ key_path user
216
+ ]
217
+ d = create_driver(config)
218
+ message = {
219
+ 'user' => 'george',
220
+ 'stat' => { 'attack' => 7 }
221
+ }
222
+ $ttl = nil
223
+ d.emit(message, get_time)
224
+ d.run
225
+
226
+ assert_equal "george", $key
227
+ assert_equal message.to_msgpack, $message
228
+ end
229
+
230
+ def test_list_asc
231
+ config = %[
232
+ format_type plain
233
+ store_type list
234
+ key_path user
235
+ ]
236
+ d = create_driver(config)
237
+ message = {
238
+ 'user' => 'george',
239
+ 'stat' => { 'attack' => 7 }
240
+ }
241
+ d.emit(message, get_time)
242
+ d.run
243
+
244
+ assert_equal :rpush, $command
245
+ assert_equal "george", $key
246
+ assert_equal message, $message
247
+ end
248
+
249
+ def test_list_desc
250
+ config = %[
251
+ format_type plain
252
+ store_type list
253
+ key_path user
254
+ order desc
255
+ ]
256
+ d = create_driver(config)
257
+ message = {
258
+ 'user' => 'george',
259
+ 'stat' => { 'attack' => 7 }
260
+ }
261
+ d.emit(message, get_time)
262
+ d.run
263
+
264
+ assert_equal :lpush, $command
265
+ assert_equal "george", $key
266
+ assert_equal message, $message
267
+ end
268
+
269
+ def test_set
270
+ config = %[
271
+ format_type plain
272
+ store_type set
273
+ key_path user
274
+ order desc
275
+ ]
276
+ d = create_driver(config)
277
+ message = {
278
+ 'user' => 'george',
279
+ 'stat' => { 'attack' => 7 }
280
+ }
281
+ d.emit(message, get_time)
282
+ d.run
283
+
284
+ assert_equal :sadd, $command
285
+ assert_equal "george", $key
286
+ assert_equal message, $message
287
+ end
288
+
289
+ def test_zset
290
+ config = %[
291
+ format_type plain
292
+ store_type zset
293
+ key_path user
294
+ score_path result
295
+ ]
296
+
297
+ d = create_driver(config)
298
+ message = {
299
+ 'user' => 'george',
300
+ 'stat' => { 'attack' => 7 },
301
+ 'result' => 81
302
+ }
303
+ d.emit(message, get_time)
304
+ d.run
305
+
306
+ assert_equal :zadd, $command
307
+ assert_equal "george", $key
308
+ assert_equal 81, $score
309
+ assert_equal message, $message
310
+ end
311
+
312
+ def test_zset_with_no_score_path
313
+ config = %[
314
+ format_type plain
315
+ store_type zset
316
+ key_path user
317
+ ]
318
+
319
+ d = create_driver(config)
320
+ message = {
321
+ 'user' => 'george',
322
+ 'stat' => { 'attack' => 7 },
323
+ 'result' => 81
324
+ }
325
+ d.emit(message, get_time)
326
+ d.run
327
+
328
+ assert_equal :zadd, $command
329
+ assert_equal "george", $key
330
+ assert_equal get_time, $score
331
+ assert_equal message, $message
332
+ end
333
+
334
+ def test_publish
335
+ config = %[
336
+ format_type plain
337
+ store_type publish
338
+ key_path user
339
+ ]
340
+
341
+ d = create_driver(config)
342
+ message = {
343
+ 'user' => 'george'
344
+ }
345
+ d.emit(message, get_time)
346
+ d.run
347
+
348
+ assert_equal :publish, $command
349
+ assert_equal "george", $channel
350
+ assert_equal message, $message
351
+ end
352
+
353
+ def test_empty_key
354
+ config = %[
355
+ format_type plain
356
+ store_type string
357
+ key_path none
358
+ ]
359
+
360
+ d = create_driver(config)
361
+ message = { 'user' => 'george' }
362
+ d.emit(message, get_time)
363
+ assert_raise(Fluent::ConfigError) do
364
+ d.run
365
+ end
366
+ end
367
+ end
metadata ADDED
@@ -0,0 +1,110 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-redis-store
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - moaikids
8
+ - HANAI Tohru aka pokehanai
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2014-05-24 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - '>='
19
+ - !ruby/object:Gem::Version
20
+ version: '0'
21
+ type: :development
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - '>='
26
+ - !ruby/object:Gem::Version
27
+ version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: test-unit
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - '>='
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :development
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - '>='
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
42
+ - !ruby/object:Gem::Dependency
43
+ name: fluentd
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - '>='
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ type: :runtime
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - '>='
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
56
+ - !ruby/object:Gem::Dependency
57
+ name: redis
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - '>='
61
+ - !ruby/object:Gem::Version
62
+ version: '0'
63
+ type: :runtime
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - '>='
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
71
+ email: hanai@pokelabo.co.jp
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - AUTHORS
77
+ - Gemfile
78
+ - README.md
79
+ - Rakefile
80
+ - fluent-plugin-redis-store.gemspec
81
+ - lib/fluent/plugin/out_redis_store.rb
82
+ - test/helpers.rb
83
+ - test/plugin/test_out_redis_publish.rb
84
+ homepage: https://github.com/pokehanai/fluent-plugin-redis-store
85
+ licenses:
86
+ - Apache License Version 2.0
87
+ metadata: {}
88
+ post_install_message:
89
+ rdoc_options: []
90
+ require_paths:
91
+ - lib
92
+ required_ruby_version: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - '>='
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ required_rubygems_version: !ruby/object:Gem::Requirement
98
+ requirements:
99
+ - - '>='
100
+ - !ruby/object:Gem::Version
101
+ version: '0'
102
+ requirements: []
103
+ rubyforge_project:
104
+ rubygems_version: 2.0.14
105
+ signing_key:
106
+ specification_version: 4
107
+ summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
108
+ test_files:
109
+ - test/helpers.rb
110
+ - test/plugin/test_out_redis_publish.rb