fluent-plugin-redis-store-liu 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: ffde671b87e74392a925d30743f9737363b29dfa
4
+ data.tar.gz: 39e7f2da328e17a5838572a5bad7d3d70b405b79
5
+ SHA512:
6
+ metadata.gz: e3a3787c4fd269f30ffa47154d48e5bd1e2a526bcf0dca7d93a73cb1407daa75941401198f613d696e891b4ee719d964c1deb4decc49dc32e60901428dbfa225
7
+ data.tar.gz: 277b132259bff780e56ac3d2ac0903a2ce0928de1374a5e9cd991cadc0eb4c2e26251f632b523cbaf07bad2dfd8f92a8385448bb35c3e4aecaa5102331a83b3f
data/AUTHORS ADDED
@@ -0,0 +1,2 @@
1
+ moaikids
2
+ HANAI Tohru aka pokehanai
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
@@ -0,0 +1,157 @@
1
+ Redis Output Plugin For fluentd
2
+ ===============================
3
+ [Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
4
+
5
+ [Fluentd]: http://fluentd.org/
6
+ [Redis]: http://redis.io/
7
+
8
+ Background
9
+ ----------
10
+
11
+ This is folked project from [fluent-plugin-redisstore][].
12
+
13
+ [fluent-plugin-redisstore]: https://github.com/moaikids/fluent-plugin-redisstore
14
+
15
+ Features
16
+ --------
17
+
18
+ #### Supported Redis commands
19
+
20
+ Currently the plugin supports following Redis commands:
21
+
22
+ - **set** by `string` type (of the plugin)
23
+ - **lpush**/**rpush** by `list` type
24
+ - **sadd** by `set` type
25
+ - **zadd** by `zset` type
26
+ - **publish** by `publish` type
27
+
28
+ #### Supported _value_ format
29
+
30
+ - plain(as is)
31
+ - JSON
32
+ - [MessagePack](http://msgpack.org/)
33
+
34
+ #### _key_ string for Redis storage
35
+
36
+ Redis commands require _key_ and _value_.
37
+ For _key_, the plugin supports either way;
38
+
39
+ 1. Specify a fixed key.
40
+ You can do this simply using `key` option in td-agent configuration file.
41
+
42
+ ```apache
43
+ type redis_store
44
+ key userdata
45
+ ```
46
+
47
+ 2. Lookup a key string in every event data by a lookup path.
48
+ If event data have structured data like
49
+
50
+ ```javascript
51
+ { "user": { "name": "Kei" } }
52
+ ```
53
+
54
+ and you want to use each name of user, you can use `key_path` option.
55
+
56
+ ```apache
57
+ type redis_store
58
+ key_path user.name
59
+ ```
60
+
61
+ With the above data, `Kei` will be a _key_.
62
+
63
+ In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
64
+
65
+ ```apache
66
+ type redis_store
67
+ key_path user.name
68
+ key_prefix ouruser.
69
+ key_suffix .accesslog
70
+ ```
71
+
72
+ With the previous data, _key_ will be `outuser.Kei.accesslog`.
73
+
74
+ #### _value_ data for Redis storage
75
+
76
+ To determine what _value_ in every event data to be srtored, you have two options;
77
+
78
+ 1. Store extracted data in event data, by a lookup path with `value_path` option.
79
+ It works like `key_path`.
80
+ 2. Store whole data.
81
+ This is default behavior. To do it, simply omit `value_path` option.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-redis-store
87
+
88
+ Configuration
89
+ -------------
90
+
91
+ ### Redis connection
92
+
93
+ | Key | Type | Required? | Default | Description |
94
+ | :---- | :----- | :---------- | :----------------------- | :------------ |
95
+ | `url` | string | Optional | redis://127.0.0.1:6379/0 | URL to Redis server |
96
+ | `port` | int | Optional | 6379 | port number of Redis server |
97
+ | `password` | string | Optional | | password for Redis connection |
98
+ | `path` | string | Optional | | To connect via Unix socket, try '/tmp/redis.sock' |
99
+ | `db` | int | Optional | 0 | DB number of Redis |
100
+ | `timeout` | float | Optional | 5.0 | connection timeout in seconds |
101
+
102
+ ### common options for storages
103
+
104
+ | Key | Type | Default | Description |
105
+ | :---- | :----- | :----------------------- | :------------ |
106
+ | `key` | string | | Fixed _key_ used to store(publish) in Redis |
107
+ | `key_path` | string | | path to lookup for _key_ in the event data |
108
+ | `key_prefix` | string | | prefix of _key_ |
109
+ | `key_suffix` | string | | suffix of _key_ |
110
+ | `value_path` | string | (whole event data) | path to lookup for _value_ in the event data |
111
+ | `store_type` | string | zset | `string`/`list`/`set`/`zset`/`publish` |
112
+ | `format_type` | string | plain | format type for _value_ (`plain`/`json`/`msgpack`) |
113
+ | `key_expire` | int | -1 | If set, the key will be expired in specified seconds |
114
+
115
+ Note: either `key` or `key_path` is required.
116
+
117
+ ### `string` storage specific options
118
+
119
+ | Key | Type | Default | Description |
120
+ | :---- | :----- | :----------------------- | :------------ |
121
+ | `type` | string | | Fixed _key_ used to store(publish) in Redis |
122
+ No more options than common options.
123
+
124
+ ### `list` storage specific options
125
+
126
+ | Key | Type | Default | Description |
127
+ | :---- | :----- | :----------------------- | :------------ |
128
+ | `order` | string | asc | `asc`: **rpush**, `desc`: **lpush** |
129
+
130
+ ### `set` storage specific options
131
+
132
+ No more options than common options.
133
+
134
+ ### `zset` storage specific options
135
+
136
+ | Key | Type | Default | Description |
137
+ | :---- | :----- | :----------------------- | :------------ |
138
+ | `score_path` | string | (_time_ of log event) | path to lookup for _score_ in the event data |
139
+ | `value_expire` | int | | value expiration in seconds |
140
+
141
+ If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
142
+ based on *timestamp* and it deletes expired _members_ every after new event data arrives.
143
+
144
+ ### `publish` storage specific options
145
+
146
+ No more options than common options.
147
+
148
+
149
+ Copyright
150
+ ---------
151
+
152
+ Copyright (c) 2013 moaikids
153
+ Copyright (c) 2014 HANAI Tohru
154
+
155
+ License
156
+ -------
157
+ Apache License, Version 2.0
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+
4
+ require 'rake/testtask'
5
+ Rake::TestTask.new(:test) do |test|
6
+ test.libs << 'lib' << 'test'
7
+ test.pattern = 'test/**/test_*.rb'
8
+ test.verbose = true
9
+ end
10
+
11
+ task :default => :test
@@ -0,0 +1,21 @@
1
+ # -*- encoding: utf-8 -*-
2
+ Gem::Specification.new do |gem|
3
+ gem.name = "fluent-plugin-redis-store-liu"
4
+ gem.email = "liuhao7860286@126.com"
5
+ gem.version = "0.1.1"
6
+ gem.authors = ["moaikids", "HANAI Tohru aka pokehanai"]
7
+ gem.licenses = ["Apache License Version 2.0"]
8
+ gem.summary = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
9
+ gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
10
+ gem.homepage = "https://github.com/hao7860286/fluent-plugin-redis-store-liu"
11
+
12
+ gem.files = `git ls-files`.split($\)
13
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
14
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
15
+ gem.require_paths = ["lib"]
16
+
17
+ gem.add_development_dependency "rake"
18
+ gem.add_development_dependency "test-unit"
19
+ gem.add_runtime_dependency "fluentd"
20
+ gem.add_runtime_dependency "redis"
21
+ end
@@ -0,0 +1,229 @@
1
+ module Fluent
2
+ class RedisStoreOutput < BufferedOutput
3
+ Fluent::Plugin.register_output('redis_store', self)
4
+
5
+ # redis connection
6
+ config_param :host, :string, :default => '127.0.0.1'
7
+ config_param :port, :integer, :default => 6379
8
+ config_param :path, :string, :default => nil
9
+ config_param :password, :string, :default => nil
10
+ config_param :db, :integer, :default => 0
11
+ config_param :timeout, :float, :default => 5.0
12
+
13
+ # redis command and parameters
14
+ config_param :format_type, :string, :default => 'json'
15
+ config_param :store_type, :string, :default => 'zset'
16
+ config_param :key_prefix, :string, :default => ''
17
+ config_param :key_suffix, :string, :default => ''
18
+ config_param :key, :string, :default => nil
19
+ config_param :key_path, :string, :default => nil
20
+ config_param :score_path, :string, :default => nil
21
+ config_param :value_path, :string, :default => ''
22
+ config_param :key_expire, :integer, :default => -1
23
+ config_param :value_expire, :integer, :default => -1
24
+ config_param :value_length, :integer, :default => -1
25
+ config_param :order, :string, :default => 'asc'
26
+
27
+ def initialize
28
+ super
29
+ require 'redis'
30
+ require 'msgpack'
31
+ end
32
+
33
+ def configure(conf)
34
+ super
35
+
36
+ if @key_path == nil and @key == nil
37
+ raise Fluent::ConfigError, "either key_path or key is required"
38
+ end
39
+ end
40
+
41
+ def start
42
+ super
43
+ if @path
44
+ @redis = Redis.new(:path => @path, :password => @password,
45
+ :timeout => @timeout, :thread_safe => true, :db => @db)
46
+ else
47
+ @redis = Redis.new(:host => @host, :port => @port, :password => @password,
48
+ :timeout => @timeout, :thread_safe => true, :db => @db)
49
+ end
50
+ end
51
+
52
+ def shutdown
53
+ @redis.quit
54
+ end
55
+
56
+ def format(tag, time, record)
57
+ [tag, time, record].to_msgpack
58
+ end
59
+
60
+ def write(chunk)
61
+ @redis.pipelined {
62
+ chunk.open { |io|
63
+ begin
64
+ MessagePack::Unpacker.new(io).each { |message|
65
+ begin
66
+ (tag, time, record) = message
67
+ case @store_type
68
+ when 'zset'
69
+ operation_for_zset(record, time)
70
+ when 'set'
71
+ operation_for_set(record)
72
+ when 'list'
73
+ operation_for_list(record)
74
+ when 'string'
75
+ operation_for_string(record)
76
+ when 'publish'
77
+ operation_for_publish(record)
78
+ end
79
+ rescue NoMethodError => e
80
+ puts e
81
+ end
82
+ }
83
+ rescue EOFError
84
+ # EOFError always occured when reached end of chunk.
85
+ end
86
+ }
87
+ }
88
+ end
89
+
90
+ def operation_for_zset(record, time)
91
+ key = get_key_from(record)
92
+ value = get_value_from(record)
93
+ score = get_score_from(record, time)
94
+ @redis.zadd key, score, value
95
+
96
+ set_key_expire key
97
+ if 0 < @value_expire
98
+ now = Time.now.to_i
99
+ @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
100
+ end
101
+ if 0 < @value_length
102
+ script = generate_zremrangebyrank_script(key, @value_length, @order)
103
+ @redis.eval script
104
+ end
105
+ end
106
+
107
+ def operation_for_set(record)
108
+ key = get_key_from(record)
109
+ value = get_value_from(record)
110
+ @redis.sadd key, value
111
+ set_key_expire key
112
+ end
113
+
114
+ def operation_for_list(record)
115
+ key = get_key_from(record)
116
+ value = get_value_from(record)
117
+
118
+ if @order == 'asc'
119
+ @redis.rpush key, value
120
+ else
121
+ @redis.lpush key, value
122
+ end
123
+ set_key_expire key
124
+ if 0 < @value_length
125
+ script = generate_ltrim_script(key, @value_length, @order)
126
+ @redis.eval script
127
+ end
128
+ end
129
+
130
+ def operation_for_string(record)
131
+ key = get_key_from(record)
132
+ value = get_value_from(record)
133
+ @redis.set key, value
134
+
135
+ set_key_expire key
136
+ end
137
+
138
+ def operation_for_publish(record)
139
+ key = get_key_from(record)
140
+ value = get_value_from(record)
141
+ @redis.publish key, value
142
+ end
143
+
144
+ def generate_zremrangebyrank_script(key, maxlen, order)
145
+ script = "local key = '" + key.to_s + "'\n"
146
+ script += "local maxlen = " + maxlen.to_s + "\n"
147
+ script += "local order ='" + order.to_s + "'\n"
148
+ script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
149
+ script += "if len > maxlen then\n"
150
+ script += " if order == 'asc' then\n"
151
+ script += " local l = len - maxlen\n"
152
+ script += " if l >= 0 then\n"
153
+ script += " return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
154
+ script += " end\n"
155
+ script += " else\n"
156
+ script += " return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
157
+ script += " end\n"
158
+ script += "end\n"
159
+ return script
160
+ end
161
+
162
+ def generate_ltrim_script(key, maxlen, order)
163
+ script = "local key = '" + key.to_s + "'\n"
164
+ script += "local maxlen = " + maxlen.to_s + "\n"
165
+ script += "local order ='" + order.to_s + "'\n"
166
+ script += "local len = tonumber(redis.call('LLEN', key))\n"
167
+ script += "if len > maxlen then\n"
168
+ script += " if order == 'asc' then\n"
169
+ script += " local l = len - maxlen\n"
170
+ script += " return redis.call('LTRIM', key, l, -1)\n"
171
+ script += " else\n"
172
+ script += " return redis.call('LTRIM', key, 0, maxlen - 1)\n"
173
+ script += " end\n"
174
+ script += "end\n"
175
+ return script
176
+ end
177
+
178
+ def traverse(data, key)
179
+ val = data
180
+ key.split('.').each{ |k|
181
+ if val.has_key?(k)
182
+ val = val[k]
183
+ else
184
+ return nil
185
+ end
186
+ }
187
+ return val
188
+ end
189
+
190
+ def get_key_from(record)
191
+ if @key
192
+ k = @key
193
+ else
194
+ k = traverse(record, @key_path).to_s
195
+ end
196
+ key = @key_prefix + k + @key_suffix
197
+
198
+ raise Fluent::ConfigError, "key is empty" if key == ''
199
+ key
200
+ end
201
+
202
+ def get_value_from(record)
203
+ value = traverse(record, @value_path)
204
+ case @format_type
205
+ when 'json'
206
+ value.to_json
207
+ when 'msgpack'
208
+ value.to_msgpack
209
+ else
210
+ value
211
+ end
212
+ end
213
+
214
+ def get_score_from(record, time)
215
+ if @score_path
216
+ traverse(record, @score_path)
217
+ else
218
+ time
219
+ end
220
+ end
221
+
222
+ def set_key_expire(key)
223
+ if 0 < @key_expire
224
+ @redis.expire key, @key_expire
225
+ end
226
+ end
227
+
228
+ end
229
+ end
@@ -0,0 +1,28 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ begin
4
+ Bundler.setup(:default, :development)
5
+ rescue Bundler::BundlerError => e
6
+ $stderr.puts e.message
7
+ $stderr.puts "Run `bundle install` to install missing gems"
8
+ exit e.status_code
9
+ end
10
+ require 'test/unit'
11
+
12
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
14
+ require 'fluent/test'
15
+ unless ENV.has_key?('VERBOSE')
16
+ nulllogger = Object.new
17
+ nulllogger.instance_eval {|obj|
18
+ def method_missing(method, *args)
19
+ # pass
20
+ end
21
+ }
22
+ $log = nulllogger
23
+ end
24
+
25
+ require 'fluent/plugin/out_redis_store'
26
+
27
+ class Test::Unit::TestCase
28
+ end
@@ -0,0 +1,367 @@
1
+ require 'helpers'
2
+
3
+ require 'redis'
4
+
5
+ $channel = nil
6
+ $message = nil
7
+
8
+ class Redis
9
+ def initialize(options = {})
10
+ end
11
+
12
+ def pipelined
13
+ yield self
14
+ end
15
+
16
+ def set(key, message)
17
+ $command = :set
18
+ $key = key
19
+ $message = message
20
+ end
21
+
22
+ def rpush(key, message)
23
+ $command = :rpush
24
+ $key = key
25
+ $message = message
26
+ end
27
+
28
+ def lpush(key, message)
29
+ $command = :lpush
30
+ $key = key
31
+ $message = message
32
+ end
33
+
34
+ def sadd(key, message)
35
+ $command = :sadd
36
+ $key = key
37
+ $message = message
38
+ end
39
+
40
+ def zadd(key, score, message)
41
+ $command = :zadd
42
+ $key = key
43
+ $score = score
44
+ $message = message
45
+ end
46
+
47
+ def expire(key, ttl)
48
+ $expire_key = key
49
+ $ttl = ttl
50
+ end
51
+
52
+ def publish(channel, message)
53
+ $command = :publish
54
+ $channel = channel
55
+ $message = message
56
+ end
57
+
58
+ def quit
59
+ end
60
+ end
61
+
62
+ class RedisStoreOutputTest < Test::Unit::TestCase
63
+ def setup
64
+ Fluent::Test.setup
65
+ end
66
+
67
+ def create_driver(conf)
68
+ Fluent::Test::BufferedOutputTestDriver.new(Fluent::RedisStoreOutput).configure(conf)
69
+ end
70
+
71
+ def test_configure_defaults
72
+ config = %[
73
+ key_path a
74
+ score_path b
75
+ ]
76
+ d = create_driver(config)
77
+ assert_equal "127.0.0.1", d.instance.host
78
+ assert_equal 6379, d.instance.port
79
+ assert_equal nil, d.instance.path
80
+ assert_equal nil, d.instance.password
81
+ assert_equal 0, d.instance.db
82
+ assert_equal 5.0, d.instance.timeout
83
+ assert_equal 'json', d.instance.format_type
84
+ assert_equal '', d.instance.key_prefix
85
+ assert_equal '', d.instance.key_suffix
86
+ assert_equal 'zset', d.instance.store_type
87
+ assert_equal 'a', d.instance.key_path
88
+ assert_equal nil, d.instance.key
89
+ assert_equal 'b', d.instance.score_path
90
+ assert_equal '', d.instance.value_path
91
+ assert_equal -1, d.instance.key_expire
92
+ assert_equal -1, d.instance.value_expire
93
+ assert_equal -1, d.instance.value_length
94
+ assert_equal 'asc', d.instance.order
95
+ end
96
+
97
+ def test_configure_host_port_db
98
+ config = %[
99
+ host 192.168.2.3
100
+ port 9999
101
+ password abc
102
+ db 3
103
+ timeout 7
104
+ key a
105
+ score_path b
106
+ ]
107
+ d = create_driver(config)
108
+ assert_equal "192.168.2.3", d.instance.host
109
+ assert_equal 9999, d.instance.port
110
+ assert_equal nil, d.instance.path
111
+ assert_equal 'abc', d.instance.password
112
+ assert_equal 3, d.instance.db
113
+ assert_equal 7.0, d.instance.timeout
114
+ assert_equal nil, d.instance.key_path
115
+ assert_equal 'a', d.instance.key
116
+ end
117
+
118
+ def test_configure_path
119
+ config = %[
120
+ path /tmp/foo.sock
121
+ key a
122
+ score_path b
123
+ ]
124
+ d = create_driver(config)
125
+ assert_equal "/tmp/foo.sock", d.instance.path
126
+ end
127
+
128
+ def test_configure_exception
129
+ assert_raise(Fluent::ConfigError) do
130
+ create_driver(%[])
131
+ end
132
+ end
133
+
134
+ # def test_write
135
+ # d = create_driver(CONFIG1)
136
+ #
137
+ # time = Time.parse("2011-01-02 13:14:15 UTC").to_i
138
+ # d.emit({ "foo" => "bar" }, time)
139
+ # d.run
140
+ #
141
+ # assert_equal "test", $channel
142
+ # assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
143
+ # end
144
+
145
+ def get_time
146
+ Time.parse("2011-01-02 13:14:15 UTC").to_i
147
+ end
148
+
149
+ # it should return whole message
150
+ def test_omit_value_path
151
+ config = %[
152
+ format_type plain
153
+ store_type string
154
+ key_path user
155
+ ]
156
+ d = create_driver(config)
157
+ message = {
158
+ 'user' => 'george',
159
+ 'stat' => { 'attack' => 7 }
160
+ }
161
+ $ttl = nil
162
+ d.emit(message, get_time)
163
+ d.run
164
+
165
+ assert_equal "george", $key
166
+ assert_equal message, $message
167
+ assert_equal nil, $ttl
168
+ end
169
+
170
+ def test_key_value_paths
171
+ config = %[
172
+ format_type plain
173
+ store_type string
174
+ key_path user.name
175
+ value_path stat.attack
176
+ key_expire 3
177
+ ]
178
+ d = create_driver(config)
179
+ message = {
180
+ 'user' => { 'name' => 'george' },
181
+ 'stat' => { 'attack' => 7 }
182
+ }
183
+ $ttl = nil
184
+ d.emit(message, get_time)
185
+ d.run
186
+
187
+ assert_equal "george", $key
188
+ assert_equal 7, $message
189
+ assert_equal 3, $ttl
190
+ end
191
+
192
+ def test_json
193
+ config = %[
194
+ format_type json
195
+ store_type string
196
+ key_path user
197
+ ]
198
+ d = create_driver(config)
199
+ message = {
200
+ 'user' => 'george',
201
+ 'stat' => { 'attack' => 7 }
202
+ }
203
+ $ttl = nil
204
+ d.emit(message, get_time)
205
+ d.run
206
+
207
+ assert_equal "george", $key
208
+ assert_equal message.to_json, $message
209
+ end
210
+
211
+ def test_msgpack
212
+ config = %[
213
+ format_type msgpack
214
+ store_type string
215
+ key_path user
216
+ ]
217
+ d = create_driver(config)
218
+ message = {
219
+ 'user' => 'george',
220
+ 'stat' => { 'attack' => 7 }
221
+ }
222
+ $ttl = nil
223
+ d.emit(message, get_time)
224
+ d.run
225
+
226
+ assert_equal "george", $key
227
+ assert_equal message.to_msgpack, $message
228
+ end
229
+
230
+ def test_list_asc
231
+ config = %[
232
+ format_type plain
233
+ store_type list
234
+ key_path user
235
+ ]
236
+ d = create_driver(config)
237
+ message = {
238
+ 'user' => 'george',
239
+ 'stat' => { 'attack' => 7 }
240
+ }
241
+ d.emit(message, get_time)
242
+ d.run
243
+
244
+ assert_equal :rpush, $command
245
+ assert_equal "george", $key
246
+ assert_equal message, $message
247
+ end
248
+
249
+ def test_list_desc
250
+ config = %[
251
+ format_type plain
252
+ store_type list
253
+ key_path user
254
+ order desc
255
+ ]
256
+ d = create_driver(config)
257
+ message = {
258
+ 'user' => 'george',
259
+ 'stat' => { 'attack' => 7 }
260
+ }
261
+ d.emit(message, get_time)
262
+ d.run
263
+
264
+ assert_equal :lpush, $command
265
+ assert_equal "george", $key
266
+ assert_equal message, $message
267
+ end
268
+
269
+ def test_set
270
+ config = %[
271
+ format_type plain
272
+ store_type set
273
+ key_path user
274
+ order desc
275
+ ]
276
+ d = create_driver(config)
277
+ message = {
278
+ 'user' => 'george',
279
+ 'stat' => { 'attack' => 7 }
280
+ }
281
+ d.emit(message, get_time)
282
+ d.run
283
+
284
+ assert_equal :sadd, $command
285
+ assert_equal "george", $key
286
+ assert_equal message, $message
287
+ end
288
+
289
+ def test_zset
290
+ config = %[
291
+ format_type plain
292
+ store_type zset
293
+ key_path user
294
+ score_path result
295
+ ]
296
+
297
+ d = create_driver(config)
298
+ message = {
299
+ 'user' => 'george',
300
+ 'stat' => { 'attack' => 7 },
301
+ 'result' => 81
302
+ }
303
+ d.emit(message, get_time)
304
+ d.run
305
+
306
+ assert_equal :zadd, $command
307
+ assert_equal "george", $key
308
+ assert_equal 81, $score
309
+ assert_equal message, $message
310
+ end
311
+
312
+ def test_zset_with_no_score_path
313
+ config = %[
314
+ format_type plain
315
+ store_type zset
316
+ key_path user
317
+ ]
318
+
319
+ d = create_driver(config)
320
+ message = {
321
+ 'user' => 'george',
322
+ 'stat' => { 'attack' => 7 },
323
+ 'result' => 81
324
+ }
325
+ d.emit(message, get_time)
326
+ d.run
327
+
328
+ assert_equal :zadd, $command
329
+ assert_equal "george", $key
330
+ assert_equal get_time, $score
331
+ assert_equal message, $message
332
+ end
333
+
334
+ def test_publish
335
+ config = %[
336
+ format_type plain
337
+ store_type publish
338
+ key_path user
339
+ ]
340
+
341
+ d = create_driver(config)
342
+ message = {
343
+ 'user' => 'george'
344
+ }
345
+ d.emit(message, get_time)
346
+ d.run
347
+
348
+ assert_equal :publish, $command
349
+ assert_equal "george", $channel
350
+ assert_equal message, $message
351
+ end
352
+
353
+ def test_empty_key
354
+ config = %[
355
+ format_type plain
356
+ store_type string
357
+ key_path none
358
+ ]
359
+
360
+ d = create_driver(config)
361
+ message = { 'user' => 'george' }
362
+ d.emit(message, get_time)
363
+ assert_raise(Fluent::ConfigError) do
364
+ d.run
365
+ end
366
+ end
367
+ end
metadata ADDED
@@ -0,0 +1,110 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-redis-store-liu
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.1
5
+ platform: ruby
6
+ authors:
7
+ - moaikids
8
+ - HANAI Tohru aka pokehanai
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2017-08-17 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - ">="
19
+ - !ruby/object:Gem::Version
20
+ version: '0'
21
+ type: :development
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - ">="
26
+ - !ruby/object:Gem::Version
27
+ version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: test-unit
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :development
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
42
+ - !ruby/object:Gem::Dependency
43
+ name: fluentd
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ type: :runtime
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
56
+ - !ruby/object:Gem::Dependency
57
+ name: redis
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - ">="
61
+ - !ruby/object:Gem::Version
62
+ version: '0'
63
+ type: :runtime
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
71
+ email: liuhao7860286@126.com
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - AUTHORS
77
+ - Gemfile
78
+ - README.md
79
+ - Rakefile
80
+ - fluent-plugin-redis-store-liu.gemspec
81
+ - lib/fluent/plugin/out_redis_store.rb
82
+ - test/helpers.rb
83
+ - test/plugin/test_out_redis_publish.rb
84
+ homepage: https://github.com/hao7860286/fluent-plugin-redis-store-liu
85
+ licenses:
86
+ - Apache License Version 2.0
87
+ metadata: {}
88
+ post_install_message:
89
+ rdoc_options: []
90
+ require_paths:
91
+ - lib
92
+ required_ruby_version: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ required_rubygems_version: !ruby/object:Gem::Requirement
98
+ requirements:
99
+ - - ">="
100
+ - !ruby/object:Gem::Version
101
+ version: '0'
102
+ requirements: []
103
+ rubyforge_project:
104
+ rubygems_version: 2.5.1
105
+ signing_key:
106
+ specification_version: 4
107
+ summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
108
+ test_files:
109
+ - test/helpers.rb
110
+ - test/plugin/test_out_redis_publish.rb