fluent-plugin-redis-store-wejick 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 527ccf44f03adce04f4d258ebfd141682f458214
4
+ data.tar.gz: 2dffb41df7987498176f9c1f1d96fdfa28482c37
5
+ SHA512:
6
+ metadata.gz: 2996eac1fe39a32fdc986cf4ff5caada8841a272cfaa3df2ae90a84116678d4fdef3a77ca9c1afce46cb00c23530287ceed85ac38f6675fa1ab2776a3dcfdfb6
7
+ data.tar.gz: 498b42ca521cecda815e0e60e5a71821a601205bc70c67ecf101182fcdeece3746e94c6d4721dc22ede23473ad5aec47f17d8e143962d49087d6c6730d91b68b
data/AUTHORS ADDED
@@ -0,0 +1,2 @@
1
+ moaikids
2
+ HANAI Tohru aka pokehanai
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
data/README.md ADDED
@@ -0,0 +1,158 @@
1
+ Redis Output Plugin For fluentd
2
+ ===============================
3
+ [Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
4
+
5
+ [Fluentd]: http://fluentd.org/
6
+ [Redis]: http://redis.io/
7
+
8
+ Background
9
+ ----------
10
+
11
+ This is folked project from [fluent-plugin-redisstore][].
12
+
13
+ [fluent-plugin-redisstore]: https://github.com/moaikids/fluent-plugin-redisstore
14
+
15
+ Features
16
+ --------
17
+
18
+ #### Supported Redis commands
19
+
20
+ Currently the plugin supports following Redis commands:
21
+
22
+ - **set** by `string` type (of the plugin)
23
+ - **lpush**/**rpush** by `list` type
24
+ - **sadd** by `set` type
25
+ - **zadd** by `zset` type
26
+ - **publish** by `publish` type
27
+
28
+ #### Supported _value_ format
29
+
30
+ - plain(as is)
31
+ - JSON
32
+ - [MessagePack](http://msgpack.org/)
33
+
34
+ #### _key_ string for Redis storage
35
+
36
+ Redis commands require _key_ and _value_.
37
+ For _key_, the plugin supports either way;
38
+
39
+ 1. Specify a fixed key.
40
+ You can do this simply using `key` option in td-agent configuration file.
41
+
42
+ ```apache
43
+ type redis_store
44
+ key userdata
45
+ ```
46
+
47
+ 2. Lookup a key string in every event data by a lookup path.
48
+ If event data have structured data like
49
+
50
+ ```javascript
51
+ { "user": { "name": "Kei" } }
52
+ ```
53
+
54
+ and you want to use each name of user, you can use `key_path` option.
55
+
56
+ ```apache
57
+ type redis_store
58
+ key_path user.name
59
+ ```
60
+
61
+ With the above data, `Kei` will be a _key_.
62
+
63
+ In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
64
+
65
+ ```apache
66
+ type redis_store
67
+ key_path user.name
68
+ key_prefix ouruser.
69
+ key_suffix .accesslog
70
+ ```
71
+
72
+ With the previous data, _key_ will be `outuser.Kei.accesslog`.
73
+
74
+ #### _value_ data for Redis storage
75
+
76
+ To determine what _value_ in every event data to be srtored, you have two options;
77
+
78
+ 1. Store extracted data in event data, by a lookup path with `value_path` option.
79
+ It works like `key_path`.
80
+ 2. Store whole data.
81
+ This is default behavior. To do it, simply omit `value_path` option.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-redis-store
87
+
88
+ Configuration
89
+ -------------
90
+
91
+ ### Redis connection
92
+
93
+ | Key | Type | Required? | Default | Description |
94
+ | :---- | :----- | :---------- | :----------------------- | :------------ |
95
+ | `host` | string | Optional | 127.0.0.1 | host name of Redis server |
96
+ | `port` | int | Optional | 6379 | port number of Redis server |
97
+ | `password` | string | Optional | | password for Redis connection |
98
+ | `path` | string | Optional | | To connect via Unix socket, try '/tmp/redis.sock' |
99
+ | `db` | int | Optional | 0 | DB number of Redis |
100
+ | `timeout` | float | Optional | 5.0 | connection timeout in seconds |
101
+
102
+ ### common options for storages
103
+
104
+ | Key | Type | Default | Description |
105
+ | :---- | :----- | :----------------------- | :------------ |
106
+ | `key` | string | | Fixed _key_ used to store(publish) in Redis |
107
+ | `key_path` | string | | path to lookup for _key_ in the event data |
108
+ | `key_prefix` | string | | prefix of _key_ |
109
+ | `key_suffix` | string | | suffix of _key_ |
110
+ | `value_path` | string | (whole event data) | path to lookup for _value_ in the event data |
111
+ | `store_type` | string | zset | `string`/`list`/`set`/`zset`/`publish` |
112
+ | `format_type` | string | plain | format type for _value_ (`plain`/`json`/`msgpack`) |
113
+ | `key_expire` | int | -1 | If set, the key will be expired in specified seconds |
114
+ | `flush_interval` | time | 1 | Time interval which events will be flushed to Redis |
115
+ | `prevent_duplicate` | (0,1) | 0 | Prevent duplicated value in one key (list)
116
+ Note: either `key` or `key_path` is required.
117
+
118
+ ### `string` storage specific options
119
+
120
+ | Key | Type | Default | Description |
121
+ | :---- | :----- | :----------------------- | :------------ |
122
+ | `type` | string | | Fixed _key_ used to store(publish) in Redis |
123
+ No more options than common options.
124
+
125
+ ### `list` storage specific options
126
+
127
+ | Key | Type | Default | Description |
128
+ | :---- | :----- | :----------------------- | :------------ |
129
+ | `order` | string | asc | `asc`: **rpush**, `desc`: **lpush** |
130
+
131
+ ### `set` storage specific options
132
+
133
+ No more options than common options.
134
+
135
+ ### `zset` storage specific options
136
+
137
+ | Key | Type | Default | Description |
138
+ | :---- | :----- | :----------------------- | :------------ |
139
+ | `score_path` | string | (_time_ of log event) | path to lookup for _score_ in the event data |
140
+ | `value_expire` | int | | value expiration in seconds |
141
+
142
+ If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
143
+ based on *timestamp* and it deletes expired _members_ every after new event data arrives.
144
+
145
+ ### `publish` storage specific options
146
+
147
+ No more options than common options.
148
+
149
+
150
+ Copyright
151
+ ---------
152
+
153
+ Copyright (c) 2013 moaikids
154
+ Copyright (c) 2014 HANAI Tohru
155
+
156
+ License
157
+ -------
158
+ Apache License, Version 2.0
data/Rakefile ADDED
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+
4
+ require 'rake/testtask'
5
+ Rake::TestTask.new(:test) do |test|
6
+ test.libs << 'lib' << 'test'
7
+ test.pattern = 'test/**/test_*.rb'
8
+ test.verbose = true
9
+ end
10
+
11
+ task :default => :test
@@ -0,0 +1,21 @@
1
+ # -*- encoding: utf-8 -*-
2
+ Gem::Specification.new do |gem|
3
+ gem.name = "fluent-plugin-redis-store-wejick"
4
+ gem.email = "wejick@gmail.com"
5
+ gem.version = "0.0.1"
6
+ gem.authors = ["wejick", "Gian Giovani"]
7
+ gem.licenses = ["Apache License Version 2.0"]
8
+ gem.summary = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
9
+ gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
10
+ gem.homepage = "https://github.com/wejick/fluent-plugin-redis-store"
11
+
12
+ gem.files = `git ls-files`.split($\)
13
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
14
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
15
+ gem.require_paths = ["lib"]
16
+
17
+ gem.add_development_dependency "rake"
18
+ gem.add_development_dependency "test-unit"
19
+ gem.add_runtime_dependency "fluentd"
20
+ gem.add_runtime_dependency "redis"
21
+ end
@@ -0,0 +1,252 @@
1
+ module Fluent
2
+ class RedisStoreOutput < BufferedOutput
3
+ Fluent::Plugin.register_output('redis_store_wejick', self)
4
+
5
+ # redis connection
6
+ config_param :host, :string, :default => '127.0.0.1'
7
+ config_param :port, :integer, :default => 6379
8
+ config_param :path, :string, :default => nil
9
+ config_param :password, :string, :default => nil
10
+ config_param :db, :integer, :default => 0
11
+ config_param :timeout, :float, :default => 5.0
12
+
13
+ # redis command and parameters
14
+ config_param :format_type, :string, :default => 'json'
15
+ config_param :store_type, :string, :default => 'zset'
16
+ config_param :key_prefix, :string, :default => ''
17
+ config_param :key_suffix, :string, :default => ''
18
+ config_param :key, :string, :default => nil
19
+ config_param :key_path, :string, :default => nil
20
+ config_param :score_path, :string, :default => nil
21
+ config_param :value_path, :string, :default => ''
22
+ config_param :key_expire, :integer, :default => -1
23
+ config_param :value_expire, :integer, :default => -1
24
+ config_param :value_length, :integer, :default => -1
25
+ config_param :order, :string, :default => 'asc'
26
+ config_param :prevent_duplicate, :integer, :default => 0
27
+ config_param :only_alphabet,:integer, :default => 0
28
+ config_param :tidy_string, :integer, :default => 0
29
+
30
+ def initialize
31
+ super
32
+ require 'redis'
33
+ require 'msgpack'
34
+ require 'cgi'
35
+ end
36
+
37
+ def configure(conf)
38
+ super
39
+
40
+ if @key_path == nil and @key == nil
41
+ raise Fluent::ConfigError, "either key_path or key is required"
42
+ end
43
+ end
44
+
45
+ def start
46
+ super
47
+ if @path
48
+ @redis = Redis.new(:path => @path, :password => @password,
49
+ :timeout => @timeout, :thread_safe => true, :db => @db)
50
+ else
51
+ @redis = Redis.new(:host => @host, :port => @port, :password => @password,
52
+ :timeout => @timeout, :thread_safe => true, :db => @db)
53
+ end
54
+ end
55
+
56
+ def shutdown
57
+ @redis.quit
58
+ end
59
+
60
+ def format(tag, time, record)
61
+ [tag, time, record].to_msgpack
62
+ end
63
+
64
+ def write(chunk)
65
+ @redis.pipelined {
66
+ chunk.open { |io|
67
+ begin
68
+ MessagePack::Unpacker.new(io).each { |message|
69
+ begin
70
+ (tag, time, record) = message
71
+ case @store_type
72
+ when 'zset'
73
+ operation_for_zset(record, time)
74
+ when 'set'
75
+ operation_for_set(record)
76
+ when 'list'
77
+ operation_for_list(record)
78
+ when 'string'
79
+ operation_for_string(record)
80
+ when 'publish'
81
+ operation_for_publish(record)
82
+ end
83
+ rescue NoMethodError => e
84
+ puts e
85
+ end
86
+ }
87
+ rescue EOFError
88
+ # EOFError always occured when reached end of chunk.
89
+ end
90
+ }
91
+ }
92
+ end
93
+
94
+ def operation_for_zset(record, time)
95
+ key = get_key_from(record)
96
+ value = get_value_from(record)
97
+ score = get_score_from(record, time)
98
+ @redis.zadd key, score, value
99
+
100
+ set_key_expire key
101
+ if 0 < @value_expire
102
+ now = Time.now.to_i
103
+ @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
104
+ end
105
+ if 0 < @value_length
106
+ script = generate_zremrangebyrank_script(key, @value_length, @order)
107
+ @redis.eval script
108
+ end
109
+ end
110
+
111
+ def operation_for_set(record)
112
+ key = get_key_from(record)
113
+ value = get_value_from(record)
114
+ @redis.sadd key, value
115
+ set_key_expire key
116
+ end
117
+
118
+ def operation_for_list(record)
119
+ key = get_key_from(record)
120
+ value = get_value_from(record)
121
+
122
+ if 0 < @tidy_string
123
+ value = tidy_up_string(value)
124
+ end
125
+ if 0 < @only_alphabet
126
+ if ( /^[a-zA-Z0-9 ]*$/.match(value) ) != nil
127
+ else
128
+ return
129
+ end
130
+ end
131
+ if 0 < @prevent_duplicate
132
+ @redis.lrem key.to_s, 1, value.to_s
133
+ end
134
+ if @order == 'asc'
135
+ @redis.rpush key, value
136
+ else
137
+ @redis.lpush key, value
138
+ end
139
+ set_key_expire key
140
+ if 0 < @value_length
141
+ script = generate_ltrim_script(key, @value_length, @order)
142
+ @redis.eval script
143
+ end
144
+ end
145
+
146
+ def operation_for_string(record)
147
+ key = get_key_from(record)
148
+ value = get_value_from(record)
149
+ @redis.set key, value
150
+
151
+ set_key_expire key
152
+ end
153
+
154
+ def operation_for_publish(record)
155
+ key = get_key_from(record)
156
+ value = get_value_from(record)
157
+ @redis.publish key, value
158
+ end
159
+
160
+ # unescape and make it to downcase
161
+ def tidy_up_string(string)
162
+ string.downcase
163
+ string = CGI.unescape(string)
164
+ return string
165
+ end
166
+
167
+ def generate_zremrangebyrank_script(key, maxlen, order)
168
+ script = "local key = '" + key.to_s + "'\n"
169
+ script += "local maxlen = " + maxlen.to_s + "\n"
170
+ script += "local order ='" + order.to_s + "'\n"
171
+ script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
172
+ script += "if len > maxlen then\n"
173
+ script += " if order == 'asc' then\n"
174
+ script += " local l = len - maxlen\n"
175
+ script += " if l >= 0 then\n"
176
+ script += " return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
177
+ script += " end\n"
178
+ script += " else\n"
179
+ script += " return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
180
+ script += " end\n"
181
+ script += "end\n"
182
+ return script
183
+ end
184
+
185
+ def generate_ltrim_script(key, maxlen, order)
186
+ script = "local key = '" + key.to_s + "'\n"
187
+ script += "local maxlen = " + maxlen.to_s + "\n"
188
+ script += "local order ='" + order.to_s + "'\n"
189
+ script += "local len = tonumber(redis.call('LLEN', key))\n"
190
+ script += "if len > maxlen then\n"
191
+ script += " if order == 'asc' then\n"
192
+ script += " local l = len - maxlen\n"
193
+ script += " return redis.call('LTRIM', key, l, -1)\n"
194
+ script += " else\n"
195
+ script += " return redis.call('LTRIM', key, 0, maxlen - 1)\n"
196
+ script += " end\n"
197
+ script += "end\n"
198
+ return script
199
+ end
200
+
201
+ def traverse(data, key)
202
+ val = data
203
+ key.split('.').each{ |k|
204
+ if val.has_key?(k)
205
+ val = val[k]
206
+ else
207
+ return nil
208
+ end
209
+ }
210
+ return val
211
+ end
212
+
213
+ def get_key_from(record)
214
+ if @key
215
+ k = @key
216
+ else
217
+ k = traverse(record, @key_path).to_s
218
+ end
219
+ key = @key_prefix + k + @key_suffix
220
+
221
+ raise Fluent::ConfigError, "key is empty" if key == ''
222
+ key
223
+ end
224
+
225
+ def get_value_from(record)
226
+ value = traverse(record, @value_path)
227
+ case @format_type
228
+ when 'json'
229
+ value.to_json
230
+ when 'msgpack'
231
+ value.to_msgpack
232
+ else
233
+ value
234
+ end
235
+ end
236
+
237
+ def get_score_from(record, time)
238
+ if @score_path
239
+ traverse(record, @score_path)
240
+ else
241
+ time
242
+ end
243
+ end
244
+
245
+ def set_key_expire(key)
246
+ if 0 < @key_expire
247
+ @redis.expire key, @key_expire
248
+ end
249
+ end
250
+
251
+ end
252
+ end
data/test/helpers.rb ADDED
@@ -0,0 +1,28 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ begin
4
+ Bundler.setup(:default, :development)
5
+ rescue Bundler::BundlerError => e
6
+ $stderr.puts e.message
7
+ $stderr.puts "Run `bundle install` to install missing gems"
8
+ exit e.status_code
9
+ end
10
+ require 'test/unit'
11
+
12
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
14
+ require 'fluent/test'
15
+ unless ENV.has_key?('VERBOSE')
16
+ nulllogger = Object.new
17
+ nulllogger.instance_eval {|obj|
18
+ def method_missing(method, *args)
19
+ # pass
20
+ end
21
+ }
22
+ $log = nulllogger
23
+ end
24
+
25
+ require 'fluent/plugin/out_redis_store'
26
+
27
+ class Test::Unit::TestCase
28
+ end
@@ -0,0 +1,367 @@
1
+ require 'helpers'
2
+
3
+ require 'redis'
4
+
5
+ $channel = nil
6
+ $message = nil
7
+
8
+ class Redis
9
+ def initialize(options = {})
10
+ end
11
+
12
+ def pipelined
13
+ yield self
14
+ end
15
+
16
+ def set(key, message)
17
+ $command = :set
18
+ $key = key
19
+ $message = message
20
+ end
21
+
22
+ def rpush(key, message)
23
+ $command = :rpush
24
+ $key = key
25
+ $message = message
26
+ end
27
+
28
+ def lpush(key, message)
29
+ $command = :lpush
30
+ $key = key
31
+ $message = message
32
+ end
33
+
34
+ def sadd(key, message)
35
+ $command = :sadd
36
+ $key = key
37
+ $message = message
38
+ end
39
+
40
+ def zadd(key, score, message)
41
+ $command = :zadd
42
+ $key = key
43
+ $score = score
44
+ $message = message
45
+ end
46
+
47
+ def expire(key, ttl)
48
+ $expire_key = key
49
+ $ttl = ttl
50
+ end
51
+
52
+ def publish(channel, message)
53
+ $command = :publish
54
+ $channel = channel
55
+ $message = message
56
+ end
57
+
58
+ def quit
59
+ end
60
+ end
61
+
62
+ class RedisStoreOutputTest < Test::Unit::TestCase
63
+ def setup
64
+ Fluent::Test.setup
65
+ end
66
+
67
+ def create_driver(conf)
68
+ Fluent::Test::BufferedOutputTestDriver.new(Fluent::RedisStoreOutput).configure(conf)
69
+ end
70
+
71
+ def test_configure_defaults
72
+ config = %[
73
+ key_path a
74
+ score_path b
75
+ ]
76
+ d = create_driver(config)
77
+ assert_equal "127.0.0.1", d.instance.host
78
+ assert_equal 6379, d.instance.port
79
+ assert_equal nil, d.instance.path
80
+ assert_equal nil, d.instance.password
81
+ assert_equal 0, d.instance.db
82
+ assert_equal 5.0, d.instance.timeout
83
+ assert_equal 'json', d.instance.format_type
84
+ assert_equal '', d.instance.key_prefix
85
+ assert_equal '', d.instance.key_suffix
86
+ assert_equal 'zset', d.instance.store_type
87
+ assert_equal 'a', d.instance.key_path
88
+ assert_equal nil, d.instance.key
89
+ assert_equal 'b', d.instance.score_path
90
+ assert_equal '', d.instance.value_path
91
+ assert_equal -1, d.instance.key_expire
92
+ assert_equal -1, d.instance.value_expire
93
+ assert_equal -1, d.instance.value_length
94
+ assert_equal 'asc', d.instance.order
95
+ end
96
+
97
+ def test_configure_host_port_db
98
+ config = %[
99
+ host 192.168.2.3
100
+ port 9999
101
+ password abc
102
+ db 3
103
+ timeout 7
104
+ key a
105
+ score_path b
106
+ ]
107
+ d = create_driver(config)
108
+ assert_equal "192.168.2.3", d.instance.host
109
+ assert_equal 9999, d.instance.port
110
+ assert_equal nil, d.instance.path
111
+ assert_equal 'abc', d.instance.password
112
+ assert_equal 3, d.instance.db
113
+ assert_equal 7.0, d.instance.timeout
114
+ assert_equal nil, d.instance.key_path
115
+ assert_equal 'a', d.instance.key
116
+ end
117
+
118
+ def test_configure_path
119
+ config = %[
120
+ path /tmp/foo.sock
121
+ key a
122
+ score_path b
123
+ ]
124
+ d = create_driver(config)
125
+ assert_equal "/tmp/foo.sock", d.instance.path
126
+ end
127
+
128
+ def test_configure_exception
129
+ assert_raise(Fluent::ConfigError) do
130
+ create_driver(%[])
131
+ end
132
+ end
133
+
134
+ # def test_write
135
+ # d = create_driver(CONFIG1)
136
+ #
137
+ # time = Time.parse("2011-01-02 13:14:15 UTC").to_i
138
+ # d.emit({ "foo" => "bar" }, time)
139
+ # d.run
140
+ #
141
+ # assert_equal "test", $channel
142
+ # assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
143
+ # end
144
+
145
+ def get_time
146
+ Time.parse("2011-01-02 13:14:15 UTC").to_i
147
+ end
148
+
149
+ # it should return whole message
150
+ def test_omit_value_path
151
+ config = %[
152
+ format_type plain
153
+ store_type string
154
+ key_path user
155
+ ]
156
+ d = create_driver(config)
157
+ message = {
158
+ 'user' => 'george',
159
+ 'stat' => { 'attack' => 7 }
160
+ }
161
+ $ttl = nil
162
+ d.emit(message, get_time)
163
+ d.run
164
+
165
+ assert_equal "george", $key
166
+ assert_equal message, $message
167
+ assert_equal nil, $ttl
168
+ end
169
+
170
+ def test_key_value_paths
171
+ config = %[
172
+ format_type plain
173
+ store_type string
174
+ key_path user.name
175
+ value_path stat.attack
176
+ key_expire 3
177
+ ]
178
+ d = create_driver(config)
179
+ message = {
180
+ 'user' => { 'name' => 'george' },
181
+ 'stat' => { 'attack' => 7 }
182
+ }
183
+ $ttl = nil
184
+ d.emit(message, get_time)
185
+ d.run
186
+
187
+ assert_equal "george", $key
188
+ assert_equal 7, $message
189
+ assert_equal 3, $ttl
190
+ end
191
+
192
+ def test_json
193
+ config = %[
194
+ format_type json
195
+ store_type string
196
+ key_path user
197
+ ]
198
+ d = create_driver(config)
199
+ message = {
200
+ 'user' => 'george',
201
+ 'stat' => { 'attack' => 7 }
202
+ }
203
+ $ttl = nil
204
+ d.emit(message, get_time)
205
+ d.run
206
+
207
+ assert_equal "george", $key
208
+ assert_equal message.to_json, $message
209
+ end
210
+
211
+ def test_msgpack
212
+ config = %[
213
+ format_type msgpack
214
+ store_type string
215
+ key_path user
216
+ ]
217
+ d = create_driver(config)
218
+ message = {
219
+ 'user' => 'george',
220
+ 'stat' => { 'attack' => 7 }
221
+ }
222
+ $ttl = nil
223
+ d.emit(message, get_time)
224
+ d.run
225
+
226
+ assert_equal "george", $key
227
+ assert_equal message.to_msgpack, $message
228
+ end
229
+
230
+ def test_list_asc
231
+ config = %[
232
+ format_type plain
233
+ store_type list
234
+ key_path user
235
+ ]
236
+ d = create_driver(config)
237
+ message = {
238
+ 'user' => 'george',
239
+ 'stat' => { 'attack' => 7 }
240
+ }
241
+ d.emit(message, get_time)
242
+ d.run
243
+
244
+ assert_equal :rpush, $command
245
+ assert_equal "george", $key
246
+ assert_equal message, $message
247
+ end
248
+
249
+ def test_list_desc
250
+ config = %[
251
+ format_type plain
252
+ store_type list
253
+ key_path user
254
+ order desc
255
+ ]
256
+ d = create_driver(config)
257
+ message = {
258
+ 'user' => 'george',
259
+ 'stat' => { 'attack' => 7 }
260
+ }
261
+ d.emit(message, get_time)
262
+ d.run
263
+
264
+ assert_equal :lpush, $command
265
+ assert_equal "george", $key
266
+ assert_equal message, $message
267
+ end
268
+
269
+ def test_set
270
+ config = %[
271
+ format_type plain
272
+ store_type set
273
+ key_path user
274
+ order desc
275
+ ]
276
+ d = create_driver(config)
277
+ message = {
278
+ 'user' => 'george',
279
+ 'stat' => { 'attack' => 7 }
280
+ }
281
+ d.emit(message, get_time)
282
+ d.run
283
+
284
+ assert_equal :sadd, $command
285
+ assert_equal "george", $key
286
+ assert_equal message, $message
287
+ end
288
+
289
+ def test_zset
290
+ config = %[
291
+ format_type plain
292
+ store_type zset
293
+ key_path user
294
+ score_path result
295
+ ]
296
+
297
+ d = create_driver(config)
298
+ message = {
299
+ 'user' => 'george',
300
+ 'stat' => { 'attack' => 7 },
301
+ 'result' => 81
302
+ }
303
+ d.emit(message, get_time)
304
+ d.run
305
+
306
+ assert_equal :zadd, $command
307
+ assert_equal "george", $key
308
+ assert_equal 81, $score
309
+ assert_equal message, $message
310
+ end
311
+
312
+ def test_zset_with_no_score_path
313
+ config = %[
314
+ format_type plain
315
+ store_type zset
316
+ key_path user
317
+ ]
318
+
319
+ d = create_driver(config)
320
+ message = {
321
+ 'user' => 'george',
322
+ 'stat' => { 'attack' => 7 },
323
+ 'result' => 81
324
+ }
325
+ d.emit(message, get_time)
326
+ d.run
327
+
328
+ assert_equal :zadd, $command
329
+ assert_equal "george", $key
330
+ assert_equal get_time, $score
331
+ assert_equal message, $message
332
+ end
333
+
334
+ def test_publish
335
+ config = %[
336
+ format_type plain
337
+ store_type publish
338
+ key_path user
339
+ ]
340
+
341
+ d = create_driver(config)
342
+ message = {
343
+ 'user' => 'george'
344
+ }
345
+ d.emit(message, get_time)
346
+ d.run
347
+
348
+ assert_equal :publish, $command
349
+ assert_equal "george", $channel
350
+ assert_equal message, $message
351
+ end
352
+
353
+ def test_empty_key
354
+ config = %[
355
+ format_type plain
356
+ store_type string
357
+ key_path none
358
+ ]
359
+
360
+ d = create_driver(config)
361
+ message = { 'user' => 'george' }
362
+ d.emit(message, get_time)
363
+ assert_raise(Fluent::ConfigError) do
364
+ d.run
365
+ end
366
+ end
367
+ end
metadata ADDED
@@ -0,0 +1,110 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-redis-store-wejick
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ platform: ruby
6
+ authors:
7
+ - wejick
8
+ - Gian Giovani
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2016-08-02 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - ">="
19
+ - !ruby/object:Gem::Version
20
+ version: '0'
21
+ type: :development
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - ">="
26
+ - !ruby/object:Gem::Version
27
+ version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: test-unit
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :development
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
42
+ - !ruby/object:Gem::Dependency
43
+ name: fluentd
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ type: :runtime
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
56
+ - !ruby/object:Gem::Dependency
57
+ name: redis
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - ">="
61
+ - !ruby/object:Gem::Version
62
+ version: '0'
63
+ type: :runtime
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
71
+ email: wejick@gmail.com
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - AUTHORS
77
+ - Gemfile
78
+ - README.md
79
+ - Rakefile
80
+ - fluent-plugin-redis-store-wejick.gemspec
81
+ - lib/fluent/plugin/out_redis_store_wejick.rb
82
+ - test/helpers.rb
83
+ - test/plugin/test_out_redis_publish.rb
84
+ homepage: https://github.com/wejick/fluent-plugin-redis-store
85
+ licenses:
86
+ - Apache License Version 2.0
87
+ metadata: {}
88
+ post_install_message:
89
+ rdoc_options: []
90
+ require_paths:
91
+ - lib
92
+ required_ruby_version: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ required_rubygems_version: !ruby/object:Gem::Requirement
98
+ requirements:
99
+ - - ">="
100
+ - !ruby/object:Gem::Version
101
+ version: '0'
102
+ requirements: []
103
+ rubyforge_project:
104
+ rubygems_version: 2.5.1
105
+ signing_key:
106
+ specification_version: 4
107
+ summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
108
+ test_files:
109
+ - test/helpers.rb
110
+ - test/plugin/test_out_redis_publish.rb