fluent-plugin-redis-store-dtdream 1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: bfe76529a47096abf717cd31789af9ae71e16083
4
+ data.tar.gz: '0520972ef94f5223aa18059a0c3fa0eb58fb22a1'
5
+ SHA512:
6
+ metadata.gz: 81d8fd0e28f6b92ea6a98ed1934635bfd69432512d78c130521a7af358e2ba7cbc8610762489c359976695416b0925dff1eff06659cf840bd2b160848525b7af
7
+ data.tar.gz: 65f7c028af5d8a1f6789fa11f9121f4f713dd33d58b93cdb4a1a60210bb471744361c646e0fa5fc64c564db9111ac85c5f65a6dad0af40478970936af7eb775a
data/AUTHORS ADDED
@@ -0,0 +1,2 @@
1
+ moaikids
2
+ HANAI Tohru aka pokehanai
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
@@ -0,0 +1,158 @@
1
+ Redis Output Plugin For fluentd
2
+ ===============================
3
+ [Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
4
+
5
+ [Fluentd]: http://fluentd.org/
6
+ [Redis]: http://redis.io/
7
+
8
+ Background
9
+ ----------
10
+
11
+ This is folked project from [fluent-plugin-redisstore][].
12
+
13
+ [fluent-plugin-redisstore]: https://github.com/moaikids/fluent-plugin-redisstore
14
+
15
+ Features
16
+ --------
17
+
18
+ #### Supported Redis commands
19
+
20
+ Currently the plugin supports following Redis commands:
21
+
22
+ - **set** by `string` type (of the plugin)
23
+ - **lpush**/**rpush** by `list` type
24
+ - **sadd** by `set` type
25
+ - **zadd** by `zset` type
26
+ - **publish** by `publish` type
27
+
28
+ #### Supported _value_ format
29
+
30
+ - plain(as is)
31
+ - JSON
32
+ - [MessagePack](http://msgpack.org/)
33
+
34
+ #### _key_ string for Redis storage
35
+
36
+ Redis commands require _key_ and _value_.
37
+ For _key_, the plugin supports either way;
38
+
39
+ 1. Specify a fixed key.
40
+ You can do this simply using `key` option in td-agent configuration file.
41
+
42
+ ```apache
43
+ type redis_store
44
+ key userdata
45
+ ```
46
+
47
+ 2. Lookup a key string in every event data by a lookup path.
48
+ If event data have structured data like
49
+
50
+ ```javascript
51
+ { "user": { "name": "Kei" } }
52
+ ```
53
+
54
+ and you want to use each name of user, you can use `key_path` option.
55
+
56
+ ```apache
57
+ type redis_store
58
+ key_path user.name
59
+ ```
60
+
61
+ With the above data, `Kei` will be a _key_.
62
+
63
+ In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
64
+
65
+ ```apache
66
+ type redis_store
67
+ key_path user.name
68
+ key_prefix ouruser.
69
+ key_suffix .accesslog
70
+ ```
71
+
72
+ With the previous data, _key_ will be `outuser.Kei.accesslog`.
73
+
74
+ #### _value_ data for Redis storage
75
+
76
+ To determine what _value_ in every event data to be srtored, you have two options;
77
+
78
+ 1. Store extracted data in event data, by a lookup path with `value_path` option.
79
+ It works like `key_path`.
80
+ 2. Store whole data.
81
+ This is default behavior. To do it, simply omit `value_path` option.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-redis-store
87
+
88
+ Configuration
89
+ -------------
90
+
91
+ ### Redis connection
92
+
93
+ | Key | Type | Required? | Default | Description |
94
+ | :---- | :----- | :---------- | :----------------------- | :------------ |
95
+ | `host` | string | Optional | 127.0.0.1 | host name of Redis server |
96
+ | `port` | int | Optional | 6379 | port number of Redis server |
97
+ | `password` | string | Optional | | password for Redis connection |
98
+ | `path` | string | Optional | | To connect via Unix socket, try '/tmp/redis.sock' |
99
+ | `db` | int | Optional | 0 | DB number of Redis |
100
+ | `timeout` | float | Optional | 5.0 | connection timeout in seconds |
101
+
102
+ ### common options for storages
103
+
104
+ | Key | Type | Default | Description |
105
+ | :---- | :----- | :----------------------- | :------------ |
106
+ | `key` | string | | Fixed _key_ used to store(publish) in Redis |
107
+ | `key_path` | string | | path to lookup for _key_ in the event data |
108
+ | `key_prefix` | string | | prefix of _key_ |
109
+ | `key_suffix` | string | | suffix of _key_ |
110
+ | `value_path` | string | (whole event data) | path to lookup for _value_ in the event data |
111
+ | `store_type` | string | zset | `string`/`list`/`set`/`zset`/`publish` |
112
+ | `format_type` | string | plain | format type for _value_ (`plain`/`json`/`msgpack`) |
113
+ | `key_expire` | int | -1 | If set, the key will be expired in specified seconds |
114
+ | `flush_interval` | time | 1 | Time interval which events will be flushed to Redis |
115
+
116
+ Note: either `key` or `key_path` is required.
117
+
118
+ ### `string` storage specific options
119
+
120
+ | Key | Type | Default | Description |
121
+ | :---- | :----- | :----------------------- | :------------ |
122
+ | `type` | string | | Fixed _key_ used to store(publish) in Redis |
123
+ No more options than common options.
124
+
125
+ ### `list` storage specific options
126
+
127
+ | Key | Type | Default | Description |
128
+ | :---- | :----- | :----------------------- | :------------ |
129
+ | `order` | string | asc | `asc`: **rpush**, `desc`: **lpush** |
130
+
131
+ ### `set` storage specific options
132
+
133
+ No more options than common options.
134
+
135
+ ### `zset` storage specific options
136
+
137
+ | Key | Type | Default | Description |
138
+ | :---- | :----- | :----------------------- | :------------ |
139
+ | `score_path` | string | (_time_ of log event) | path to lookup for _score_ in the event data |
140
+ | `value_expire` | int | | value expiration in seconds |
141
+
142
+ If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
143
+ based on *timestamp* and it deletes expired _members_ every after new event data arrives.
144
+
145
+ ### `publish` storage specific options
146
+
147
+ No more options than common options.
148
+
149
+
150
+ Copyright
151
+ ---------
152
+
153
+ Copyright (c) 2013 moaikids
154
+ Copyright (c) 2014 HANAI Tohru
155
+
156
+ License
157
+ -------
158
+ Apache License, Version 2.0
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+
4
+ require 'rake/testtask'
5
+ Rake::TestTask.new(:test) do |test|
6
+ test.libs << 'lib' << 'test'
7
+ test.pattern = 'test/**/test_*.rb'
8
+ test.verbose = true
9
+ end
10
+
11
+ task :default => :test
@@ -0,0 +1,20 @@
1
+ # -*- encoding: utf-8 -*-
2
+ Gem::Specification.new do |gem|
3
+ gem.name = "fluent-plugin-redis-store-dtdream"
4
+ gem.version = "1.0"
5
+ gem.authors = ["moaikids", "HANAI Tohru aka pokehanai"]
6
+ gem.licenses = ["Apache License Version 2.0"]
7
+ gem.summary = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
8
+ gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
9
+ gem.homepage = "https://10.96.6.7/container/fluent-plugin-redis-store"
10
+
11
+ gem.files = `git ls-files`.split($\)
12
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
13
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
14
+ gem.require_paths = ["lib"]
15
+
16
+ gem.add_development_dependency "rake"
17
+ gem.add_development_dependency "test-unit"
18
+ gem.add_runtime_dependency "fluentd"
19
+ gem.add_runtime_dependency "redis"
20
+ end
@@ -0,0 +1,231 @@
1
+ module Fluent
2
+ class RedisStoreOutput < BufferedOutput
3
+ Fluent::Plugin.register_output('redis_store', self)
4
+
5
+ # redis connection
6
+ config_param :host, :string, :default => '127.0.0.1'
7
+ config_param :port, :integer, :default => 6379
8
+ config_param :path, :string, :default => nil
9
+ config_param :password, :string, :default => nil
10
+ config_param :db, :integer, :default => 0
11
+ config_param :timeout, :float, :default => 5.0
12
+
13
+ # redis command and parameters
14
+ config_param :format_type, :string, :default => 'json'
15
+ config_param :store_type, :string, :default => 'zset'
16
+ config_param :key_prefix, :string, :default => ''
17
+ config_param :key_suffix, :string, :default => ''
18
+ config_param :key, :string, :default => nil
19
+ config_param :key_path, :string, :default => nil
20
+ config_param :score_path, :string, :default => nil
21
+ config_param :value_path, :string, :default => ''
22
+ config_param :key_expire, :integer, :default => -1
23
+ config_param :value_expire, :integer, :default => -1
24
+ config_param :value_length, :integer, :default => -1
25
+ config_param :order, :string, :default => 'asc'
26
+
27
+ def initialize
28
+ super
29
+ require 'oj'
30
+ require 'redis'
31
+ require 'msgpack'
32
+ end
33
+
34
+ def configure(conf)
35
+ super
36
+
37
+ if @key_path == nil and @key == nil
38
+ raise Fluent::ConfigError, "either key_path or key is required"
39
+ end
40
+ end
41
+
42
+ def start
43
+ super
44
+ if @path
45
+ @redis = Redis.new(:path => @path, :password => @password,
46
+ :timeout => @timeout, :thread_safe => true, :db => @db)
47
+ else
48
+ @redis = Redis.new(:host => @host, :port => @port, :password => @password,
49
+ :timeout => @timeout, :thread_safe => true, :db => @db)
50
+ end
51
+ end
52
+
53
+ def shutdown
54
+ @redis.quit
55
+ end
56
+
57
+ def format(tag, time, record)
58
+ [tag, time, record].to_msgpack
59
+ end
60
+
61
+ def write(chunk)
62
+ @redis.pipelined {
63
+ chunk.open { |io|
64
+ begin
65
+ MessagePack::Unpacker.new(io).each { |message|
66
+ begin
67
+ (tag, time, record) = message
68
+ case @store_type
69
+ when 'zset'
70
+ operation_for_zset(record, time)
71
+ when 'set'
72
+ operation_for_set(record)
73
+ when 'list'
74
+ operation_for_list(record)
75
+ when 'string'
76
+ operation_for_string(record)
77
+ when 'publish'
78
+ operation_for_publish(record)
79
+ end
80
+ rescue NoMethodError => e
81
+ puts e
82
+ end
83
+ }
84
+ rescue EOFError
85
+ # EOFError always occured when reached end of chunk.
86
+ end
87
+ }
88
+ }
89
+ end
90
+
91
+ def operation_for_zset(record, time)
92
+ key = get_key_from(record)
93
+ value = get_value_from(record)
94
+ score = get_score_from(record, time)
95
+ @redis.zadd key, score, value
96
+
97
+ set_key_expire key
98
+ if 0 < @value_expire
99
+ now = Time.now.to_i
100
+ @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
101
+ end
102
+ if 0 < @value_length
103
+ script = generate_zremrangebyrank_script(key, @value_length, @order)
104
+ @redis.eval script
105
+ end
106
+ end
107
+
108
+ def operation_for_set(record)
109
+ key = get_key_from(record)
110
+ value = get_value_from(record)
111
+ @redis.sadd key, value
112
+ set_key_expire key
113
+ end
114
+
115
+ def operation_for_list(record)
116
+ key = get_key_from(record)
117
+ value = get_value_from(record)
118
+
119
+ if @order == 'asc'
120
+ @redis.rpush key, value
121
+ else
122
+ @redis.lpush key, value
123
+ end
124
+ set_key_expire key
125
+ if 0 < @value_length
126
+ script = generate_ltrim_script(key, @value_length, @order)
127
+ @redis.eval script
128
+ end
129
+ end
130
+
131
+ def operation_for_string(record)
132
+ key = get_key_from(record)
133
+ value = get_value_from(record)
134
+ @redis.set key, value
135
+
136
+ set_key_expire key
137
+ end
138
+
139
+ def operation_for_publish(record)
140
+ key = get_key_from(record)
141
+ value = get_value_from(record)
142
+ @redis.publish key, value
143
+ end
144
+
145
+ def generate_zremrangebyrank_script(key, maxlen, order)
146
+ script = "local key = '" + key.to_s + "'\n"
147
+ script += "local maxlen = " + maxlen.to_s + "\n"
148
+ script += "local order ='" + order.to_s + "'\n"
149
+ script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
150
+ script += "if len > maxlen then\n"
151
+ script += " if order == 'asc' then\n"
152
+ script += " local l = len - maxlen\n"
153
+ script += " if l >= 0 then\n"
154
+ script += " return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
155
+ script += " end\n"
156
+ script += " else\n"
157
+ script += " return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
158
+ script += " end\n"
159
+ script += "end\n"
160
+ return script
161
+ end
162
+
163
+ def generate_ltrim_script(key, maxlen, order)
164
+ script = "local key = '" + key.to_s + "'\n"
165
+ script += "local maxlen = " + maxlen.to_s + "\n"
166
+ script += "local order ='" + order.to_s + "'\n"
167
+ script += "local len = tonumber(redis.call('LLEN', key))\n"
168
+ script += "if len > maxlen then\n"
169
+ script += " if order == 'asc' then\n"
170
+ script += " local l = len - maxlen\n"
171
+ script += " return redis.call('LTRIM', key, l, -1)\n"
172
+ script += " else\n"
173
+ script += " return redis.call('LTRIM', key, 0, maxlen - 1)\n"
174
+ script += " end\n"
175
+ script += "end\n"
176
+ return script
177
+ end
178
+
179
+ def traverse(data, key)
180
+ val = data
181
+ key.split('.').each{ |k|
182
+ if val.has_key?(k)
183
+ val = val[k]
184
+ else
185
+ return nil
186
+ end
187
+ }
188
+ return val
189
+ end
190
+
191
+ def get_key_from(record)
192
+ if @key
193
+ k = @key
194
+ else
195
+ k = traverse(record, @key_path).to_s
196
+ end
197
+ key = @key_prefix + k + @key_suffix
198
+
199
+ raise Fluent::ConfigError, "key is empty" if key == ''
200
+ key
201
+ end
202
+
203
+ def get_value_from(record)
204
+ value = traverse(record, @value_path)
205
+ case @format_type
206
+ when 'json'
207
+ #value.to_json
208
+ value = Oj.dump(value)
209
+ when 'msgpack'
210
+ value.to_msgpack
211
+ else
212
+ value
213
+ end
214
+ end
215
+
216
+ def get_score_from(record, time)
217
+ if @score_path
218
+ traverse(record, @score_path)
219
+ else
220
+ time
221
+ end
222
+ end
223
+
224
+ def set_key_expire(key)
225
+ if 0 < @key_expire
226
+ @redis.expire key, @key_expire
227
+ end
228
+ end
229
+
230
+ end
231
+ end
@@ -0,0 +1,28 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ begin
4
+ Bundler.setup(:default, :development)
5
+ rescue Bundler::BundlerError => e
6
+ $stderr.puts e.message
7
+ $stderr.puts "Run `bundle install` to install missing gems"
8
+ exit e.status_code
9
+ end
10
+ require 'test/unit'
11
+
12
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
14
+ require 'fluent/test'
15
+ unless ENV.has_key?('VERBOSE')
16
+ nulllogger = Object.new
17
+ nulllogger.instance_eval {|obj|
18
+ def method_missing(method, *args)
19
+ # pass
20
+ end
21
+ }
22
+ $log = nulllogger
23
+ end
24
+
25
+ require 'fluent/plugin/out_redis_store'
26
+
27
+ class Test::Unit::TestCase
28
+ end
@@ -0,0 +1,367 @@
1
+ require 'helpers'
2
+
3
+ require 'redis'
4
+
5
+ $channel = nil
6
+ $message = nil
7
+
8
+ class Redis
9
+ def initialize(options = {})
10
+ end
11
+
12
+ def pipelined
13
+ yield self
14
+ end
15
+
16
+ def set(key, message)
17
+ $command = :set
18
+ $key = key
19
+ $message = message
20
+ end
21
+
22
+ def rpush(key, message)
23
+ $command = :rpush
24
+ $key = key
25
+ $message = message
26
+ end
27
+
28
+ def lpush(key, message)
29
+ $command = :lpush
30
+ $key = key
31
+ $message = message
32
+ end
33
+
34
+ def sadd(key, message)
35
+ $command = :sadd
36
+ $key = key
37
+ $message = message
38
+ end
39
+
40
+ def zadd(key, score, message)
41
+ $command = :zadd
42
+ $key = key
43
+ $score = score
44
+ $message = message
45
+ end
46
+
47
+ def expire(key, ttl)
48
+ $expire_key = key
49
+ $ttl = ttl
50
+ end
51
+
52
+ def publish(channel, message)
53
+ $command = :publish
54
+ $channel = channel
55
+ $message = message
56
+ end
57
+
58
+ def quit
59
+ end
60
+ end
61
+
62
+ class RedisStoreOutputTest < Test::Unit::TestCase
63
+ def setup
64
+ Fluent::Test.setup
65
+ end
66
+
67
+ def create_driver(conf)
68
+ Fluent::Test::BufferedOutputTestDriver.new(Fluent::RedisStoreOutput).configure(conf)
69
+ end
70
+
71
+ def test_configure_defaults
72
+ config = %[
73
+ key_path a
74
+ score_path b
75
+ ]
76
+ d = create_driver(config)
77
+ assert_equal "127.0.0.1", d.instance.host
78
+ assert_equal 6379, d.instance.port
79
+ assert_equal nil, d.instance.path
80
+ assert_equal nil, d.instance.password
81
+ assert_equal 0, d.instance.db
82
+ assert_equal 5.0, d.instance.timeout
83
+ assert_equal 'json', d.instance.format_type
84
+ assert_equal '', d.instance.key_prefix
85
+ assert_equal '', d.instance.key_suffix
86
+ assert_equal 'zset', d.instance.store_type
87
+ assert_equal 'a', d.instance.key_path
88
+ assert_equal nil, d.instance.key
89
+ assert_equal 'b', d.instance.score_path
90
+ assert_equal '', d.instance.value_path
91
+ assert_equal -1, d.instance.key_expire
92
+ assert_equal -1, d.instance.value_expire
93
+ assert_equal -1, d.instance.value_length
94
+ assert_equal 'asc', d.instance.order
95
+ end
96
+
97
+ def test_configure_host_port_db
98
+ config = %[
99
+ host 192.168.2.3
100
+ port 9999
101
+ password abc
102
+ db 3
103
+ timeout 7
104
+ key a
105
+ score_path b
106
+ ]
107
+ d = create_driver(config)
108
+ assert_equal "192.168.2.3", d.instance.host
109
+ assert_equal 9999, d.instance.port
110
+ assert_equal nil, d.instance.path
111
+ assert_equal 'abc', d.instance.password
112
+ assert_equal 3, d.instance.db
113
+ assert_equal 7.0, d.instance.timeout
114
+ assert_equal nil, d.instance.key_path
115
+ assert_equal 'a', d.instance.key
116
+ end
117
+
118
+ def test_configure_path
119
+ config = %[
120
+ path /tmp/foo.sock
121
+ key a
122
+ score_path b
123
+ ]
124
+ d = create_driver(config)
125
+ assert_equal "/tmp/foo.sock", d.instance.path
126
+ end
127
+
128
+ def test_configure_exception
129
+ assert_raise(Fluent::ConfigError) do
130
+ create_driver(%[])
131
+ end
132
+ end
133
+
134
+ # def test_write
135
+ # d = create_driver(CONFIG1)
136
+ #
137
+ # time = Time.parse("2011-01-02 13:14:15 UTC").to_i
138
+ # d.emit({ "foo" => "bar" }, time)
139
+ # d.run
140
+ #
141
+ # assert_equal "test", $channel
142
+ # assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
143
+ # end
144
+
145
+ def get_time
146
+ Time.parse("2011-01-02 13:14:15 UTC").to_i
147
+ end
148
+
149
+ # it should return whole message
150
+ def test_omit_value_path
151
+ config = %[
152
+ format_type plain
153
+ store_type string
154
+ key_path user
155
+ ]
156
+ d = create_driver(config)
157
+ message = {
158
+ 'user' => 'george',
159
+ 'stat' => { 'attack' => 7 }
160
+ }
161
+ $ttl = nil
162
+ d.emit(message, get_time)
163
+ d.run
164
+
165
+ assert_equal "george", $key
166
+ assert_equal message, $message
167
+ assert_equal nil, $ttl
168
+ end
169
+
170
+ def test_key_value_paths
171
+ config = %[
172
+ format_type plain
173
+ store_type string
174
+ key_path user.name
175
+ value_path stat.attack
176
+ key_expire 3
177
+ ]
178
+ d = create_driver(config)
179
+ message = {
180
+ 'user' => { 'name' => 'george' },
181
+ 'stat' => { 'attack' => 7 }
182
+ }
183
+ $ttl = nil
184
+ d.emit(message, get_time)
185
+ d.run
186
+
187
+ assert_equal "george", $key
188
+ assert_equal 7, $message
189
+ assert_equal 3, $ttl
190
+ end
191
+
192
+ def test_json
193
+ config = %[
194
+ format_type json
195
+ store_type string
196
+ key_path user
197
+ ]
198
+ d = create_driver(config)
199
+ message = {
200
+ 'user' => 'george',
201
+ 'stat' => { 'attack' => 7 }
202
+ }
203
+ $ttl = nil
204
+ d.emit(message, get_time)
205
+ d.run
206
+
207
+ assert_equal "george", $key
208
+ assert_equal message.to_json, $message
209
+ end
210
+
211
+ def test_msgpack
212
+ config = %[
213
+ format_type msgpack
214
+ store_type string
215
+ key_path user
216
+ ]
217
+ d = create_driver(config)
218
+ message = {
219
+ 'user' => 'george',
220
+ 'stat' => { 'attack' => 7 }
221
+ }
222
+ $ttl = nil
223
+ d.emit(message, get_time)
224
+ d.run
225
+
226
+ assert_equal "george", $key
227
+ assert_equal message.to_msgpack, $message
228
+ end
229
+
230
+ def test_list_asc
231
+ config = %[
232
+ format_type plain
233
+ store_type list
234
+ key_path user
235
+ ]
236
+ d = create_driver(config)
237
+ message = {
238
+ 'user' => 'george',
239
+ 'stat' => { 'attack' => 7 }
240
+ }
241
+ d.emit(message, get_time)
242
+ d.run
243
+
244
+ assert_equal :rpush, $command
245
+ assert_equal "george", $key
246
+ assert_equal message, $message
247
+ end
248
+
249
+ def test_list_desc
250
+ config = %[
251
+ format_type plain
252
+ store_type list
253
+ key_path user
254
+ order desc
255
+ ]
256
+ d = create_driver(config)
257
+ message = {
258
+ 'user' => 'george',
259
+ 'stat' => { 'attack' => 7 }
260
+ }
261
+ d.emit(message, get_time)
262
+ d.run
263
+
264
+ assert_equal :lpush, $command
265
+ assert_equal "george", $key
266
+ assert_equal message, $message
267
+ end
268
+
269
+ def test_set
270
+ config = %[
271
+ format_type plain
272
+ store_type set
273
+ key_path user
274
+ order desc
275
+ ]
276
+ d = create_driver(config)
277
+ message = {
278
+ 'user' => 'george',
279
+ 'stat' => { 'attack' => 7 }
280
+ }
281
+ d.emit(message, get_time)
282
+ d.run
283
+
284
+ assert_equal :sadd, $command
285
+ assert_equal "george", $key
286
+ assert_equal message, $message
287
+ end
288
+
289
+ def test_zset
290
+ config = %[
291
+ format_type plain
292
+ store_type zset
293
+ key_path user
294
+ score_path result
295
+ ]
296
+
297
+ d = create_driver(config)
298
+ message = {
299
+ 'user' => 'george',
300
+ 'stat' => { 'attack' => 7 },
301
+ 'result' => 81
302
+ }
303
+ d.emit(message, get_time)
304
+ d.run
305
+
306
+ assert_equal :zadd, $command
307
+ assert_equal "george", $key
308
+ assert_equal 81, $score
309
+ assert_equal message, $message
310
+ end
311
+
312
+ def test_zset_with_no_score_path
313
+ config = %[
314
+ format_type plain
315
+ store_type zset
316
+ key_path user
317
+ ]
318
+
319
+ d = create_driver(config)
320
+ message = {
321
+ 'user' => 'george',
322
+ 'stat' => { 'attack' => 7 },
323
+ 'result' => 81
324
+ }
325
+ d.emit(message, get_time)
326
+ d.run
327
+
328
+ assert_equal :zadd, $command
329
+ assert_equal "george", $key
330
+ assert_equal get_time, $score
331
+ assert_equal message, $message
332
+ end
333
+
334
+ def test_publish
335
+ config = %[
336
+ format_type plain
337
+ store_type publish
338
+ key_path user
339
+ ]
340
+
341
+ d = create_driver(config)
342
+ message = {
343
+ 'user' => 'george'
344
+ }
345
+ d.emit(message, get_time)
346
+ d.run
347
+
348
+ assert_equal :publish, $command
349
+ assert_equal "george", $channel
350
+ assert_equal message, $message
351
+ end
352
+
353
+ def test_empty_key
354
+ config = %[
355
+ format_type plain
356
+ store_type string
357
+ key_path none
358
+ ]
359
+
360
+ d = create_driver(config)
361
+ message = { 'user' => 'george' }
362
+ d.emit(message, get_time)
363
+ assert_raise(Fluent::ConfigError) do
364
+ d.run
365
+ end
366
+ end
367
+ end
metadata ADDED
@@ -0,0 +1,110 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-redis-store-dtdream
3
+ version: !ruby/object:Gem::Version
4
+ version: '1.0'
5
+ platform: ruby
6
+ authors:
7
+ - moaikids
8
+ - HANAI Tohru aka pokehanai
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2017-08-24 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - ">="
19
+ - !ruby/object:Gem::Version
20
+ version: '0'
21
+ type: :development
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - ">="
26
+ - !ruby/object:Gem::Version
27
+ version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: test-unit
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :development
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
42
+ - !ruby/object:Gem::Dependency
43
+ name: fluentd
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ type: :runtime
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
56
+ - !ruby/object:Gem::Dependency
57
+ name: redis
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - ">="
61
+ - !ruby/object:Gem::Version
62
+ version: '0'
63
+ type: :runtime
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
71
+ email:
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - AUTHORS
77
+ - Gemfile
78
+ - README.md
79
+ - Rakefile
80
+ - fluent-plugin-redis-store.gemspec
81
+ - lib/fluent/plugin/out_redis_store.rb
82
+ - test/helpers.rb
83
+ - test/plugin/test_out_redis_publish.rb
84
+ homepage: https://10.96.6.7/container/fluent-plugin-redis-store
85
+ licenses:
86
+ - Apache License Version 2.0
87
+ metadata: {}
88
+ post_install_message:
89
+ rdoc_options: []
90
+ require_paths:
91
+ - lib
92
+ required_ruby_version: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ required_rubygems_version: !ruby/object:Gem::Requirement
98
+ requirements:
99
+ - - ">="
100
+ - !ruby/object:Gem::Version
101
+ version: '0'
102
+ requirements: []
103
+ rubyforge_project:
104
+ rubygems_version: 2.5.1
105
+ signing_key:
106
+ specification_version: 4
107
+ summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
108
+ test_files:
109
+ - test/helpers.rb
110
+ - test/plugin/test_out_redis_publish.rb