fluent-plugin-redis-store-wejick 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 527ccf44f03adce04f4d258ebfd141682f458214
4
+ data.tar.gz: 2dffb41df7987498176f9c1f1d96fdfa28482c37
5
+ SHA512:
6
+ metadata.gz: 2996eac1fe39a32fdc986cf4ff5caada8841a272cfaa3df2ae90a84116678d4fdef3a77ca9c1afce46cb00c23530287ceed85ac38f6675fa1ab2776a3dcfdfb6
7
+ data.tar.gz: 498b42ca521cecda815e0e60e5a71821a601205bc70c67ecf101182fcdeece3746e94c6d4721dc22ede23473ad5aec47f17d8e143962d49087d6c6730d91b68b
data/AUTHORS ADDED
@@ -0,0 +1,2 @@
1
+ moaikids
2
+ HANAI Tohru aka pokehanai
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
data/README.md ADDED
@@ -0,0 +1,158 @@
1
+ Redis Output Plugin For fluentd
2
+ ===============================
3
+ [Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
4
+
5
+ [Fluentd]: http://fluentd.org/
6
+ [Redis]: http://redis.io/
7
+
8
+ Background
9
+ ----------
10
+
11
+ This is folked project from [fluent-plugin-redisstore][].
12
+
13
+ [fluent-plugin-redisstore]: https://github.com/moaikids/fluent-plugin-redisstore
14
+
15
+ Features
16
+ --------
17
+
18
+ #### Supported Redis commands
19
+
20
+ Currently the plugin supports following Redis commands:
21
+
22
+ - **set** by `string` type (of the plugin)
23
+ - **lpush**/**rpush** by `list` type
24
+ - **sadd** by `set` type
25
+ - **zadd** by `zset` type
26
+ - **publish** by `publish` type
27
+
28
+ #### Supported _value_ format
29
+
30
+ - plain(as is)
31
+ - JSON
32
+ - [MessagePack](http://msgpack.org/)
33
+
34
+ #### _key_ string for Redis storage
35
+
36
+ Redis commands require _key_ and _value_.
37
+ For _key_, the plugin supports either way;
38
+
39
+ 1. Specify a fixed key.
40
+ You can do this simply using `key` option in td-agent configuration file.
41
+
42
+ ```apache
43
+ type redis_store
44
+ key userdata
45
+ ```
46
+
47
+ 2. Lookup a key string in every event data by a lookup path.
48
+ If event data have structured data like
49
+
50
+ ```javascript
51
+ { "user": { "name": "Kei" } }
52
+ ```
53
+
54
+ and you want to use each name of user, you can use `key_path` option.
55
+
56
+ ```apache
57
+ type redis_store
58
+ key_path user.name
59
+ ```
60
+
61
+ With the above data, `Kei` will be a _key_.
62
+
63
+ In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
64
+
65
+ ```apache
66
+ type redis_store
67
+ key_path user.name
68
+ key_prefix ouruser.
69
+ key_suffix .accesslog
70
+ ```
71
+
72
+ With the previous data, _key_ will be `outuser.Kei.accesslog`.
73
+
74
+ #### _value_ data for Redis storage
75
+
76
+ To determine what _value_ in every event data to be srtored, you have two options;
77
+
78
+ 1. Store extracted data in event data, by a lookup path with `value_path` option.
79
+ It works like `key_path`.
80
+ 2. Store whole data.
81
+ This is default behavior. To do it, simply omit `value_path` option.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-redis-store
87
+
88
+ Configuration
89
+ -------------
90
+
91
+ ### Redis connection
92
+
93
+ | Key | Type | Required? | Default | Description |
94
+ | :---- | :----- | :---------- | :----------------------- | :------------ |
95
+ | `host` | string | Optional | 127.0.0.1 | host name of Redis server |
96
+ | `port` | int | Optional | 6379 | port number of Redis server |
97
+ | `password` | string | Optional | | password for Redis connection |
98
+ | `path` | string | Optional | | To connect via Unix socket, try '/tmp/redis.sock' |
99
+ | `db` | int | Optional | 0 | DB number of Redis |
100
+ | `timeout` | float | Optional | 5.0 | connection timeout in seconds |
101
+
102
+ ### common options for storages
103
+
104
+ | Key | Type | Default | Description |
105
+ | :---- | :----- | :----------------------- | :------------ |
106
+ | `key` | string | | Fixed _key_ used to store(publish) in Redis |
107
+ | `key_path` | string | | path to lookup for _key_ in the event data |
108
+ | `key_prefix` | string | | prefix of _key_ |
109
+ | `key_suffix` | string | | suffix of _key_ |
110
+ | `value_path` | string | (whole event data) | path to lookup for _value_ in the event data |
111
+ | `store_type` | string | zset | `string`/`list`/`set`/`zset`/`publish` |
112
+ | `format_type` | string | plain | format type for _value_ (`plain`/`json`/`msgpack`) |
113
+ | `key_expire` | int | -1 | If set, the key will be expired in specified seconds |
114
+ | `flush_interval` | time | 1 | Time interval which events will be flushed to Redis |
115
+ | `prevent_duplicate` | (0,1) | 0 | Prevent duplicated value in one key (list)
116
+ Note: either `key` or `key_path` is required.
117
+
118
+ ### `string` storage specific options
119
+
120
+ | Key | Type | Default | Description |
121
+ | :---- | :----- | :----------------------- | :------------ |
122
+ | `type` | string | | Fixed _key_ used to store(publish) in Redis |
123
+ No more options than common options.
124
+
125
+ ### `list` storage specific options
126
+
127
+ | Key | Type | Default | Description |
128
+ | :---- | :----- | :----------------------- | :------------ |
129
+ | `order` | string | asc | `asc`: **rpush**, `desc`: **lpush** |
130
+
131
+ ### `set` storage specific options
132
+
133
+ No more options than common options.
134
+
135
+ ### `zset` storage specific options
136
+
137
+ | Key | Type | Default | Description |
138
+ | :---- | :----- | :----------------------- | :------------ |
139
+ | `score_path` | string | (_time_ of log event) | path to lookup for _score_ in the event data |
140
+ | `value_expire` | int | | value expiration in seconds |
141
+
142
+ If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
143
+ based on *timestamp* and it deletes expired _members_ every after new event data arrives.
144
+
145
+ ### `publish` storage specific options
146
+
147
+ No more options than common options.
148
+
149
+
150
+ Copyright
151
+ ---------
152
+
153
+ Copyright (c) 2013 moaikids
154
+ Copyright (c) 2014 HANAI Tohru
155
+
156
+ License
157
+ -------
158
+ Apache License, Version 2.0
data/Rakefile ADDED
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+
4
+ require 'rake/testtask'
5
+ Rake::TestTask.new(:test) do |test|
6
+ test.libs << 'lib' << 'test'
7
+ test.pattern = 'test/**/test_*.rb'
8
+ test.verbose = true
9
+ end
10
+
11
+ task :default => :test
@@ -0,0 +1,21 @@
1
+ # -*- encoding: utf-8 -*-
2
+ Gem::Specification.new do |gem|
3
+ gem.name = "fluent-plugin-redis-store-wejick"
4
+ gem.email = "wejick@gmail.com"
5
+ gem.version = "0.0.1"
6
+ gem.authors = ["wejick", "Gian Giovani"]
7
+ gem.licenses = ["Apache License Version 2.0"]
8
+ gem.summary = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
9
+ gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
10
+ gem.homepage = "https://github.com/wejick/fluent-plugin-redis-store"
11
+
12
+ gem.files = `git ls-files`.split($\)
13
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
14
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
15
+ gem.require_paths = ["lib"]
16
+
17
+ gem.add_development_dependency "rake"
18
+ gem.add_development_dependency "test-unit"
19
+ gem.add_runtime_dependency "fluentd"
20
+ gem.add_runtime_dependency "redis"
21
+ end
@@ -0,0 +1,252 @@
1
+ module Fluent
2
+ class RedisStoreOutput < BufferedOutput
3
+ Fluent::Plugin.register_output('redis_store_wejick', self)
4
+
5
+ # redis connection
6
+ config_param :host, :string, :default => '127.0.0.1'
7
+ config_param :port, :integer, :default => 6379
8
+ config_param :path, :string, :default => nil
9
+ config_param :password, :string, :default => nil
10
+ config_param :db, :integer, :default => 0
11
+ config_param :timeout, :float, :default => 5.0
12
+
13
+ # redis command and parameters
14
+ config_param :format_type, :string, :default => 'json'
15
+ config_param :store_type, :string, :default => 'zset'
16
+ config_param :key_prefix, :string, :default => ''
17
+ config_param :key_suffix, :string, :default => ''
18
+ config_param :key, :string, :default => nil
19
+ config_param :key_path, :string, :default => nil
20
+ config_param :score_path, :string, :default => nil
21
+ config_param :value_path, :string, :default => ''
22
+ config_param :key_expire, :integer, :default => -1
23
+ config_param :value_expire, :integer, :default => -1
24
+ config_param :value_length, :integer, :default => -1
25
+ config_param :order, :string, :default => 'asc'
26
+ config_param :prevent_duplicate, :integer, :default => 0
27
+ config_param :only_alphabet,:integer, :default => 0
28
+ config_param :tidy_string, :integer, :default => 0
29
+
30
+ def initialize
31
+ super
32
+ require 'redis'
33
+ require 'msgpack'
34
+ require 'cgi'
35
+ end
36
+
37
+ def configure(conf)
38
+ super
39
+
40
+ if @key_path == nil and @key == nil
41
+ raise Fluent::ConfigError, "either key_path or key is required"
42
+ end
43
+ end
44
+
45
+ def start
46
+ super
47
+ if @path
48
+ @redis = Redis.new(:path => @path, :password => @password,
49
+ :timeout => @timeout, :thread_safe => true, :db => @db)
50
+ else
51
+ @redis = Redis.new(:host => @host, :port => @port, :password => @password,
52
+ :timeout => @timeout, :thread_safe => true, :db => @db)
53
+ end
54
+ end
55
+
56
+ def shutdown
57
+ @redis.quit
58
+ end
59
+
60
+ def format(tag, time, record)
61
+ [tag, time, record].to_msgpack
62
+ end
63
+
64
+ def write(chunk)
65
+ @redis.pipelined {
66
+ chunk.open { |io|
67
+ begin
68
+ MessagePack::Unpacker.new(io).each { |message|
69
+ begin
70
+ (tag, time, record) = message
71
+ case @store_type
72
+ when 'zset'
73
+ operation_for_zset(record, time)
74
+ when 'set'
75
+ operation_for_set(record)
76
+ when 'list'
77
+ operation_for_list(record)
78
+ when 'string'
79
+ operation_for_string(record)
80
+ when 'publish'
81
+ operation_for_publish(record)
82
+ end
83
+ rescue NoMethodError => e
84
+ puts e
85
+ end
86
+ }
87
+ rescue EOFError
88
+ # EOFError always occured when reached end of chunk.
89
+ end
90
+ }
91
+ }
92
+ end
93
+
94
+ def operation_for_zset(record, time)
95
+ key = get_key_from(record)
96
+ value = get_value_from(record)
97
+ score = get_score_from(record, time)
98
+ @redis.zadd key, score, value
99
+
100
+ set_key_expire key
101
+ if 0 < @value_expire
102
+ now = Time.now.to_i
103
+ @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
104
+ end
105
+ if 0 < @value_length
106
+ script = generate_zremrangebyrank_script(key, @value_length, @order)
107
+ @redis.eval script
108
+ end
109
+ end
110
+
111
+ def operation_for_set(record)
112
+ key = get_key_from(record)
113
+ value = get_value_from(record)
114
+ @redis.sadd key, value
115
+ set_key_expire key
116
+ end
117
+
118
+ def operation_for_list(record)
119
+ key = get_key_from(record)
120
+ value = get_value_from(record)
121
+
122
+ if 0 < @tidy_string
123
+ value = tidy_up_string(value)
124
+ end
125
+ if 0 < @only_alphabet
126
+ if ( /^[a-zA-Z0-9 ]*$/.match(value) ) != nil
127
+ else
128
+ return
129
+ end
130
+ end
131
+ if 0 < @prevent_duplicate
132
+ @redis.lrem key.to_s, 1, value.to_s
133
+ end
134
+ if @order == 'asc'
135
+ @redis.rpush key, value
136
+ else
137
+ @redis.lpush key, value
138
+ end
139
+ set_key_expire key
140
+ if 0 < @value_length
141
+ script = generate_ltrim_script(key, @value_length, @order)
142
+ @redis.eval script
143
+ end
144
+ end
145
+
146
+ def operation_for_string(record)
147
+ key = get_key_from(record)
148
+ value = get_value_from(record)
149
+ @redis.set key, value
150
+
151
+ set_key_expire key
152
+ end
153
+
154
+ def operation_for_publish(record)
155
+ key = get_key_from(record)
156
+ value = get_value_from(record)
157
+ @redis.publish key, value
158
+ end
159
+
160
+ # unescape and make it to downcase
161
+ def tidy_up_string(string)
162
+ string.downcase
163
+ string = CGI.unescape(string)
164
+ return string
165
+ end
166
+
167
+ def generate_zremrangebyrank_script(key, maxlen, order)
168
+ script = "local key = '" + key.to_s + "'\n"
169
+ script += "local maxlen = " + maxlen.to_s + "\n"
170
+ script += "local order ='" + order.to_s + "'\n"
171
+ script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
172
+ script += "if len > maxlen then\n"
173
+ script += " if order == 'asc' then\n"
174
+ script += " local l = len - maxlen\n"
175
+ script += " if l >= 0 then\n"
176
+ script += " return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
177
+ script += " end\n"
178
+ script += " else\n"
179
+ script += " return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
180
+ script += " end\n"
181
+ script += "end\n"
182
+ return script
183
+ end
184
+
185
+ def generate_ltrim_script(key, maxlen, order)
186
+ script = "local key = '" + key.to_s + "'\n"
187
+ script += "local maxlen = " + maxlen.to_s + "\n"
188
+ script += "local order ='" + order.to_s + "'\n"
189
+ script += "local len = tonumber(redis.call('LLEN', key))\n"
190
+ script += "if len > maxlen then\n"
191
+ script += " if order == 'asc' then\n"
192
+ script += " local l = len - maxlen\n"
193
+ script += " return redis.call('LTRIM', key, l, -1)\n"
194
+ script += " else\n"
195
+ script += " return redis.call('LTRIM', key, 0, maxlen - 1)\n"
196
+ script += " end\n"
197
+ script += "end\n"
198
+ return script
199
+ end
200
+
201
+ def traverse(data, key)
202
+ val = data
203
+ key.split('.').each{ |k|
204
+ if val.has_key?(k)
205
+ val = val[k]
206
+ else
207
+ return nil
208
+ end
209
+ }
210
+ return val
211
+ end
212
+
213
+ def get_key_from(record)
214
+ if @key
215
+ k = @key
216
+ else
217
+ k = traverse(record, @key_path).to_s
218
+ end
219
+ key = @key_prefix + k + @key_suffix
220
+
221
+ raise Fluent::ConfigError, "key is empty" if key == ''
222
+ key
223
+ end
224
+
225
+ def get_value_from(record)
226
+ value = traverse(record, @value_path)
227
+ case @format_type
228
+ when 'json'
229
+ value.to_json
230
+ when 'msgpack'
231
+ value.to_msgpack
232
+ else
233
+ value
234
+ end
235
+ end
236
+
237
+ def get_score_from(record, time)
238
+ if @score_path
239
+ traverse(record, @score_path)
240
+ else
241
+ time
242
+ end
243
+ end
244
+
245
+ def set_key_expire(key)
246
+ if 0 < @key_expire
247
+ @redis.expire key, @key_expire
248
+ end
249
+ end
250
+
251
+ end
252
+ end
data/test/helpers.rb ADDED
@@ -0,0 +1,28 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ begin
4
+ Bundler.setup(:default, :development)
5
+ rescue Bundler::BundlerError => e
6
+ $stderr.puts e.message
7
+ $stderr.puts "Run `bundle install` to install missing gems"
8
+ exit e.status_code
9
+ end
10
+ require 'test/unit'
11
+
12
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
14
+ require 'fluent/test'
15
+ unless ENV.has_key?('VERBOSE')
16
+ nulllogger = Object.new
17
+ nulllogger.instance_eval {|obj|
18
+ def method_missing(method, *args)
19
+ # pass
20
+ end
21
+ }
22
+ $log = nulllogger
23
+ end
24
+
25
+ require 'fluent/plugin/out_redis_store'
26
+
27
+ class Test::Unit::TestCase
28
+ end
@@ -0,0 +1,367 @@
1
+ require 'helpers'
2
+
3
+ require 'redis'
4
+
5
+ $channel = nil
6
+ $message = nil
7
+
8
+ class Redis
9
+ def initialize(options = {})
10
+ end
11
+
12
+ def pipelined
13
+ yield self
14
+ end
15
+
16
+ def set(key, message)
17
+ $command = :set
18
+ $key = key
19
+ $message = message
20
+ end
21
+
22
+ def rpush(key, message)
23
+ $command = :rpush
24
+ $key = key
25
+ $message = message
26
+ end
27
+
28
+ def lpush(key, message)
29
+ $command = :lpush
30
+ $key = key
31
+ $message = message
32
+ end
33
+
34
+ def sadd(key, message)
35
+ $command = :sadd
36
+ $key = key
37
+ $message = message
38
+ end
39
+
40
+ def zadd(key, score, message)
41
+ $command = :zadd
42
+ $key = key
43
+ $score = score
44
+ $message = message
45
+ end
46
+
47
+ def expire(key, ttl)
48
+ $expire_key = key
49
+ $ttl = ttl
50
+ end
51
+
52
+ def publish(channel, message)
53
+ $command = :publish
54
+ $channel = channel
55
+ $message = message
56
+ end
57
+
58
+ def quit
59
+ end
60
+ end
61
+
62
+ class RedisStoreOutputTest < Test::Unit::TestCase
63
+ def setup
64
+ Fluent::Test.setup
65
+ end
66
+
67
+ def create_driver(conf)
68
+ Fluent::Test::BufferedOutputTestDriver.new(Fluent::RedisStoreOutput).configure(conf)
69
+ end
70
+
71
+ def test_configure_defaults
72
+ config = %[
73
+ key_path a
74
+ score_path b
75
+ ]
76
+ d = create_driver(config)
77
+ assert_equal "127.0.0.1", d.instance.host
78
+ assert_equal 6379, d.instance.port
79
+ assert_equal nil, d.instance.path
80
+ assert_equal nil, d.instance.password
81
+ assert_equal 0, d.instance.db
82
+ assert_equal 5.0, d.instance.timeout
83
+ assert_equal 'json', d.instance.format_type
84
+ assert_equal '', d.instance.key_prefix
85
+ assert_equal '', d.instance.key_suffix
86
+ assert_equal 'zset', d.instance.store_type
87
+ assert_equal 'a', d.instance.key_path
88
+ assert_equal nil, d.instance.key
89
+ assert_equal 'b', d.instance.score_path
90
+ assert_equal '', d.instance.value_path
91
+ assert_equal -1, d.instance.key_expire
92
+ assert_equal -1, d.instance.value_expire
93
+ assert_equal -1, d.instance.value_length
94
+ assert_equal 'asc', d.instance.order
95
+ end
96
+
97
+ def test_configure_host_port_db
98
+ config = %[
99
+ host 192.168.2.3
100
+ port 9999
101
+ password abc
102
+ db 3
103
+ timeout 7
104
+ key a
105
+ score_path b
106
+ ]
107
+ d = create_driver(config)
108
+ assert_equal "192.168.2.3", d.instance.host
109
+ assert_equal 9999, d.instance.port
110
+ assert_equal nil, d.instance.path
111
+ assert_equal 'abc', d.instance.password
112
+ assert_equal 3, d.instance.db
113
+ assert_equal 7.0, d.instance.timeout
114
+ assert_equal nil, d.instance.key_path
115
+ assert_equal 'a', d.instance.key
116
+ end
117
+
118
+ def test_configure_path
119
+ config = %[
120
+ path /tmp/foo.sock
121
+ key a
122
+ score_path b
123
+ ]
124
+ d = create_driver(config)
125
+ assert_equal "/tmp/foo.sock", d.instance.path
126
+ end
127
+
128
+ def test_configure_exception
129
+ assert_raise(Fluent::ConfigError) do
130
+ create_driver(%[])
131
+ end
132
+ end
133
+
134
+ # def test_write
135
+ # d = create_driver(CONFIG1)
136
+ #
137
+ # time = Time.parse("2011-01-02 13:14:15 UTC").to_i
138
+ # d.emit({ "foo" => "bar" }, time)
139
+ # d.run
140
+ #
141
+ # assert_equal "test", $channel
142
+ # assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
143
+ # end
144
+
145
+ def get_time
146
+ Time.parse("2011-01-02 13:14:15 UTC").to_i
147
+ end
148
+
149
+ # it should return whole message
150
+ def test_omit_value_path
151
+ config = %[
152
+ format_type plain
153
+ store_type string
154
+ key_path user
155
+ ]
156
+ d = create_driver(config)
157
+ message = {
158
+ 'user' => 'george',
159
+ 'stat' => { 'attack' => 7 }
160
+ }
161
+ $ttl = nil
162
+ d.emit(message, get_time)
163
+ d.run
164
+
165
+ assert_equal "george", $key
166
+ assert_equal message, $message
167
+ assert_equal nil, $ttl
168
+ end
169
+
170
+ def test_key_value_paths
171
+ config = %[
172
+ format_type plain
173
+ store_type string
174
+ key_path user.name
175
+ value_path stat.attack
176
+ key_expire 3
177
+ ]
178
+ d = create_driver(config)
179
+ message = {
180
+ 'user' => { 'name' => 'george' },
181
+ 'stat' => { 'attack' => 7 }
182
+ }
183
+ $ttl = nil
184
+ d.emit(message, get_time)
185
+ d.run
186
+
187
+ assert_equal "george", $key
188
+ assert_equal 7, $message
189
+ assert_equal 3, $ttl
190
+ end
191
+
192
+ def test_json
193
+ config = %[
194
+ format_type json
195
+ store_type string
196
+ key_path user
197
+ ]
198
+ d = create_driver(config)
199
+ message = {
200
+ 'user' => 'george',
201
+ 'stat' => { 'attack' => 7 }
202
+ }
203
+ $ttl = nil
204
+ d.emit(message, get_time)
205
+ d.run
206
+
207
+ assert_equal "george", $key
208
+ assert_equal message.to_json, $message
209
+ end
210
+
211
+ def test_msgpack
212
+ config = %[
213
+ format_type msgpack
214
+ store_type string
215
+ key_path user
216
+ ]
217
+ d = create_driver(config)
218
+ message = {
219
+ 'user' => 'george',
220
+ 'stat' => { 'attack' => 7 }
221
+ }
222
+ $ttl = nil
223
+ d.emit(message, get_time)
224
+ d.run
225
+
226
+ assert_equal "george", $key
227
+ assert_equal message.to_msgpack, $message
228
+ end
229
+
230
+ def test_list_asc
231
+ config = %[
232
+ format_type plain
233
+ store_type list
234
+ key_path user
235
+ ]
236
+ d = create_driver(config)
237
+ message = {
238
+ 'user' => 'george',
239
+ 'stat' => { 'attack' => 7 }
240
+ }
241
+ d.emit(message, get_time)
242
+ d.run
243
+
244
+ assert_equal :rpush, $command
245
+ assert_equal "george", $key
246
+ assert_equal message, $message
247
+ end
248
+
249
+ def test_list_desc
250
+ config = %[
251
+ format_type plain
252
+ store_type list
253
+ key_path user
254
+ order desc
255
+ ]
256
+ d = create_driver(config)
257
+ message = {
258
+ 'user' => 'george',
259
+ 'stat' => { 'attack' => 7 }
260
+ }
261
+ d.emit(message, get_time)
262
+ d.run
263
+
264
+ assert_equal :lpush, $command
265
+ assert_equal "george", $key
266
+ assert_equal message, $message
267
+ end
268
+
269
+ def test_set
270
+ config = %[
271
+ format_type plain
272
+ store_type set
273
+ key_path user
274
+ order desc
275
+ ]
276
+ d = create_driver(config)
277
+ message = {
278
+ 'user' => 'george',
279
+ 'stat' => { 'attack' => 7 }
280
+ }
281
+ d.emit(message, get_time)
282
+ d.run
283
+
284
+ assert_equal :sadd, $command
285
+ assert_equal "george", $key
286
+ assert_equal message, $message
287
+ end
288
+
289
+ def test_zset
290
+ config = %[
291
+ format_type plain
292
+ store_type zset
293
+ key_path user
294
+ score_path result
295
+ ]
296
+
297
+ d = create_driver(config)
298
+ message = {
299
+ 'user' => 'george',
300
+ 'stat' => { 'attack' => 7 },
301
+ 'result' => 81
302
+ }
303
+ d.emit(message, get_time)
304
+ d.run
305
+
306
+ assert_equal :zadd, $command
307
+ assert_equal "george", $key
308
+ assert_equal 81, $score
309
+ assert_equal message, $message
310
+ end
311
+
312
+ def test_zset_with_no_score_path
313
+ config = %[
314
+ format_type plain
315
+ store_type zset
316
+ key_path user
317
+ ]
318
+
319
+ d = create_driver(config)
320
+ message = {
321
+ 'user' => 'george',
322
+ 'stat' => { 'attack' => 7 },
323
+ 'result' => 81
324
+ }
325
+ d.emit(message, get_time)
326
+ d.run
327
+
328
+ assert_equal :zadd, $command
329
+ assert_equal "george", $key
330
+ assert_equal get_time, $score
331
+ assert_equal message, $message
332
+ end
333
+
334
+ def test_publish
335
+ config = %[
336
+ format_type plain
337
+ store_type publish
338
+ key_path user
339
+ ]
340
+
341
+ d = create_driver(config)
342
+ message = {
343
+ 'user' => 'george'
344
+ }
345
+ d.emit(message, get_time)
346
+ d.run
347
+
348
+ assert_equal :publish, $command
349
+ assert_equal "george", $channel
350
+ assert_equal message, $message
351
+ end
352
+
353
+ def test_empty_key
354
+ config = %[
355
+ format_type plain
356
+ store_type string
357
+ key_path none
358
+ ]
359
+
360
+ d = create_driver(config)
361
+ message = { 'user' => 'george' }
362
+ d.emit(message, get_time)
363
+ assert_raise(Fluent::ConfigError) do
364
+ d.run
365
+ end
366
+ end
367
+ end
metadata ADDED
@@ -0,0 +1,110 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-redis-store-wejick
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ platform: ruby
6
+ authors:
7
+ - wejick
8
+ - Gian Giovani
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2016-08-02 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - ">="
19
+ - !ruby/object:Gem::Version
20
+ version: '0'
21
+ type: :development
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - ">="
26
+ - !ruby/object:Gem::Version
27
+ version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: test-unit
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :development
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
42
+ - !ruby/object:Gem::Dependency
43
+ name: fluentd
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ type: :runtime
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
56
+ - !ruby/object:Gem::Dependency
57
+ name: redis
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - ">="
61
+ - !ruby/object:Gem::Version
62
+ version: '0'
63
+ type: :runtime
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
71
+ email: wejick@gmail.com
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - AUTHORS
77
+ - Gemfile
78
+ - README.md
79
+ - Rakefile
80
+ - fluent-plugin-redis-store-wejick.gemspec
81
+ - lib/fluent/plugin/out_redis_store_wejick.rb
82
+ - test/helpers.rb
83
+ - test/plugin/test_out_redis_publish.rb
84
+ homepage: https://github.com/wejick/fluent-plugin-redis-store
85
+ licenses:
86
+ - Apache License Version 2.0
87
+ metadata: {}
88
+ post_install_message:
89
+ rdoc_options: []
90
+ require_paths:
91
+ - lib
92
+ required_ruby_version: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ required_rubygems_version: !ruby/object:Gem::Requirement
98
+ requirements:
99
+ - - ">="
100
+ - !ruby/object:Gem::Version
101
+ version: '0'
102
+ requirements: []
103
+ rubyforge_project:
104
+ rubygems_version: 2.5.1
105
+ signing_key:
106
+ specification_version: 4
107
+ summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
108
+ test_files:
109
+ - test/helpers.rb
110
+ - test/plugin/test_out_redis_publish.rb