fluent-plugin-redis-store-gabfl 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 5bd41dcfe8cbac06b60d874cca8f8695c98fbecb
4
+ data.tar.gz: 91d2bb8c1dfa542d636cf9252d539a073e6a1b9a
5
+ SHA512:
6
+ metadata.gz: ee80c06d1d2115ca6f910894ed3a389372adc48a12572f9a8058549e5b5b77b2fa528f6a016819d3c765f93f7731933dcd2c8773ce525a99279f1e2b4bf3e9b8
7
+ data.tar.gz: 1824b3f7e04759fd9c94511e63097503939798523fa52631ee76152fd2aa9c283609012266aeb93399c8c3ded48dbc7ff4f97759efde5a99864aad76270cbf31
data/.gitignore ADDED
@@ -0,0 +1 @@
1
+ *.gem
data/AUTHORS ADDED
@@ -0,0 +1,2 @@
1
+ moaikids
2
+ HANAI Tohru aka pokehanai
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
data/README.md ADDED
@@ -0,0 +1,164 @@
1
+ Redis Output Plugin For fluentd
2
+ ===============================
3
+ [Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
4
+
5
+ [Fluentd]: http://fluentd.org/
6
+ [Redis]: http://redis.io/
7
+
8
+ Background
9
+ ----------
10
+
11
+ This is forked project from [fluent-plugin-redis-store][].
12
+
13
+ [fluent-plugin-redis-store]: https://github.com/pokehanai/fluent-plugin-redis-store
14
+
15
+ Features
16
+ --------
17
+
18
+ #### Supported Redis commands
19
+
20
+ Currently the plugin supports following Redis commands:
21
+
22
+ - **set** by `string` type (of the plugin)
23
+ - **lpush**/**rpush** by `list` type
24
+ - **sadd** by `set` type
25
+ - **zadd** by `zset` type
26
+ - **publish** by `publish` type
27
+
28
+ #### Supported _value_ format
29
+
30
+ - plain(as is)
31
+ - JSON
32
+ - [MessagePack](http://msgpack.org/)
33
+
34
+ #### _key_ string for Redis storage
35
+
36
+ Redis commands require _key_ and _value_.
37
+ For _key_, the plugin supports either way;
38
+
39
+ 1. Specify a fixed key.
40
+ You can do this simply using `key` option in td-agent configuration file.
41
+
42
+ ```apache
43
+ type redis_store
44
+ key userdata
45
+ ```
46
+
47
+ 2. Lookup a key string in every event data by a lookup path.
48
+ If event data have structured data like
49
+
50
+ ```javascript
51
+ { "user": { "name": "Kei" } }
52
+ ```
53
+
54
+ and you want to use each name of user, you can use `key_path` option.
55
+
56
+ ```apache
57
+ type redis_store
58
+ key_path user.name
59
+ ```
60
+
61
+ With the above data, `Kei` will be a _key_.
62
+
63
+ In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
64
+
65
+ ```apache
66
+ type redis_store
67
+ key_path user.name
68
+ key_prefix ouruser.
69
+ key_suffix .accesslog
70
+ ```
71
+
72
+ With the previous data, _key_ will be `outuser.Kei.accesslog`.
73
+
74
+ #### _value_ data for Redis storage
75
+
76
+ To determine what _value_ in every event data to be srtored, you have two options;
77
+
78
+ 1. Store extracted data in event data, by a lookup path with `value_path` option.
79
+ It works like `key_path`.
80
+ 2. Store whole data.
81
+ This is default behavior. To do it, simply omit `value_path` option.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ ```bash
87
+ fluent-gem install fluent-plugin-redis-store-gabfl
88
+
89
+ # or if you are using td-agent:
90
+ td-agent-gem install fluent-plugin-redis-store-gabfl
91
+ ```
92
+
93
+ Configuration
94
+ -------------
95
+
96
+ ### Redis connection
97
+
98
+ | Key | Type | Required? | Default | Description |
99
+ | :---- | :----- | :---------- | :----------------------- | :------------ |
100
+ | `host` | string | Optional | 127.0.0.1 | host name of Redis server |
101
+ | `port` | int | Optional | 6379 | port number of Redis server |
102
+ | `password` | string | Optional | | password for Redis connection |
103
+ | `path` | string | Optional | | To connect via Unix socket, try '/tmp/redis.sock' |
104
+ | `db` | int | Optional | 0 | DB number of Redis |
105
+ | `timeout` | float | Optional | 5.0 | connection timeout in seconds |
106
+
107
+ ### common options for storages
108
+
109
+ | Key | Type | Default | Description |
110
+ | :---- | :----- | :----------------------- | :------------ |
111
+ | `key` | string | | Fixed _key_ used to store(publish) in Redis |
112
+ | `key_path` | string | | path to lookup for _key_ in the event data |
113
+ | `key_prefix` | string | | prefix of _key_ |
114
+ | `key_suffix` | string | | suffix of _key_ |
115
+ | `value_path` | string | (whole event data) | path to lookup for _value_ in the event data |
116
+ | `store_type` | string | zset | `string`/`list`/`set`/`zset`/`publish` |
117
+ | `format_type` | string | plain | format type for _value_ (`plain`/`json`/`msgpack`) |
118
+ | `key_expire` | int | -1 | If set, the key will be expired in specified seconds |
119
+ | `flush_interval` | time | 1 | Time interval which events will be flushed to Redis |
120
+
121
+ Note: either `key` or `key_path` is required.
122
+
123
+ ### `string` storage specific options
124
+
125
+ | Key | Type | Default | Description |
126
+ | :---- | :----- | :----------------------- | :------------ |
127
+ | `type` | string | | Fixed _key_ used to store(publish) in Redis |
128
+ No more options than common options.
129
+
130
+ ### `list` storage specific options
131
+
132
+ | Key | Type | Default | Description |
133
+ | :---- | :----- | :----------------------- | :------------ |
134
+ | `order` | string | asc | `asc`: **rpush**, `desc`: **lpush** |
135
+
136
+ ### `set` storage specific options
137
+
138
+ No more options than common options.
139
+
140
+ ### `zset` storage specific options
141
+
142
+ | Key | Type | Default | Description |
143
+ | :---- | :----- | :----------------------- | :------------ |
144
+ | `score_path` | string | (_time_ of log event) | path to lookup for _score_ in the event data |
145
+ | `collision_policy` | string | (nil) | Only update elements that already exist (XX) or add a new element (NX) |
146
+ | `value_expire` | int | | value expiration in seconds |
147
+
148
+ If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
149
+ based on *timestamp* and it deletes expired _members_ every after new event data arrives.
150
+
151
+ ### `publish` storage specific options
152
+
153
+ No more options than common options.
154
+
155
+
156
+ Copyright
157
+ ---------
158
+
159
+ Copyright (c) 2013 moaikids
160
+ Copyright (c) 2014 HANAI Tohru
161
+
162
+ License
163
+ -------
164
+ Apache License, Version 2.0
data/Rakefile ADDED
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+
4
+ require 'rake/testtask'
5
+ Rake::TestTask.new(:test) do |test|
6
+ test.libs << 'lib' << 'test'
7
+ test.pattern = 'test/**/test_*.rb'
8
+ test.verbose = true
9
+ end
10
+
11
+ task :default => :test
@@ -0,0 +1,21 @@
1
+ # -*- encoding: utf-8 -*-
2
+ Gem::Specification.new do |gem|
3
+ gem.name = "fluent-plugin-redis-store-gabfl"
4
+ gem.email = "pypi@gab.lc"
5
+ gem.version = "0.1.2"
6
+ gem.authors = ["moaikids", "HANAI Tohru aka pokehanai", "Gabriel Bordeaux"]
7
+ gem.licenses = ["Apache License Version 2.0"]
8
+ gem.summary = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
9
+ gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
10
+ gem.homepage = "https://github.com/gabfl/fluent-plugin-redis-store"
11
+
12
+ gem.files = `git ls-files`.split($\)
13
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
14
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
15
+ gem.require_paths = ["lib"]
16
+
17
+ gem.add_development_dependency "rake"
18
+ gem.add_development_dependency "test-unit"
19
+ gem.add_runtime_dependency "fluentd"
20
+ gem.add_runtime_dependency "redis"
21
+ end
@@ -0,0 +1,239 @@
1
+ module Fluent
2
+ class RedisStoreOutput < BufferedOutput
3
+ Fluent::Plugin.register_output('redis_store', self)
4
+
5
+ # redis connection
6
+ config_param :host, :string, :default => '127.0.0.1'
7
+ config_param :port, :integer, :default => 6379
8
+ config_param :path, :string, :default => nil
9
+ config_param :password, :string, :default => nil
10
+ config_param :db, :integer, :default => 0
11
+ config_param :timeout, :float, :default => 5.0
12
+
13
+ # redis command and parameters
14
+ config_param :format_type, :string, :default => 'json'
15
+ config_param :store_type, :string, :default => 'zset'
16
+ config_param :key_prefix, :string, :default => ''
17
+ config_param :key_suffix, :string, :default => ''
18
+ config_param :key, :string, :default => nil
19
+ config_param :key_path, :string, :default => nil
20
+ config_param :score_path, :string, :default => nil
21
+ config_param :value_path, :string, :default => ''
22
+ config_param :key_expire, :integer, :default => -1
23
+ config_param :value_expire, :integer, :default => -1
24
+ config_param :value_length, :integer, :default => -1
25
+ config_param :order, :string, :default => 'asc'
26
+ config_param :collision_policy, :string, :default => nil
27
+ config_set_default :flush_interval, 1
28
+
29
+ def initialize
30
+ super
31
+ require 'redis'
32
+ require 'msgpack'
33
+ end
34
+
35
+ def configure(conf)
36
+ super
37
+
38
+ if @key_path == nil and @key == nil
39
+ raise Fluent::ConfigError, "either key_path or key is required"
40
+ end
41
+ end
42
+
43
+ def start
44
+ super
45
+ if @path
46
+ @redis = Redis.new(:path => @path, :password => @password,
47
+ :timeout => @timeout, :thread_safe => true, :db => @db)
48
+ else
49
+ @redis = Redis.new(:host => @host, :port => @port, :password => @password,
50
+ :timeout => @timeout, :thread_safe => true, :db => @db)
51
+ end
52
+ end
53
+
54
+ def shutdown
55
+ @redis.quit
56
+ end
57
+
58
+ def format(tag, time, record)
59
+ [tag, time, record].to_msgpack
60
+ end
61
+
62
+ def write(chunk)
63
+ @redis.pipelined {
64
+ chunk.open { |io|
65
+ begin
66
+ MessagePack::Unpacker.new(io).each { |message|
67
+ begin
68
+ (tag, time, record) = message
69
+ case @store_type
70
+ when 'zset'
71
+ operation_for_zset(record, time)
72
+ when 'set'
73
+ operation_for_set(record)
74
+ when 'list'
75
+ operation_for_list(record)
76
+ when 'string'
77
+ operation_for_string(record)
78
+ when 'publish'
79
+ operation_for_publish(record)
80
+ end
81
+ rescue NoMethodError => e
82
+ puts e
83
+ end
84
+ }
85
+ rescue EOFError
86
+ # EOFError always occured when reached end of chunk.
87
+ end
88
+ }
89
+ }
90
+ end
91
+
92
+ def operation_for_zset(record, time)
93
+ key = get_key_from(record)
94
+ value = get_value_from(record)
95
+ score = get_score_from(record, time)
96
+ if @collision_policy
97
+ if @collision_policy == 'NX'
98
+ @redis.zadd(key, score, value, :nx => true)
99
+ elsif @collision_policy == 'XX'
100
+ @redis.zadd(key, score, value, :xx => true)
101
+ end
102
+ else
103
+ @redis.zadd(key, score, value)
104
+ end
105
+
106
+ set_key_expire key
107
+ if 0 < @value_expire
108
+ now = Time.now.to_i
109
+ @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
110
+ end
111
+ if 0 < @value_length
112
+ script = generate_zremrangebyrank_script(key, @value_length, @order)
113
+ @redis.eval script
114
+ end
115
+ end
116
+
117
+ def operation_for_set(record)
118
+ key = get_key_from(record)
119
+ value = get_value_from(record)
120
+ @redis.sadd key, value
121
+ set_key_expire key
122
+ end
123
+
124
+ def operation_for_list(record)
125
+ key = get_key_from(record)
126
+ value = get_value_from(record)
127
+
128
+ if @order == 'asc'
129
+ @redis.rpush key, value
130
+ else
131
+ @redis.lpush key, value
132
+ end
133
+ set_key_expire key
134
+ if 0 < @value_length
135
+ script = generate_ltrim_script(key, @value_length, @order)
136
+ @redis.eval script
137
+ end
138
+ end
139
+
140
+ def operation_for_string(record)
141
+ key = get_key_from(record)
142
+ value = get_value_from(record)
143
+ @redis.set key, value
144
+
145
+ set_key_expire key
146
+ end
147
+
148
+ def operation_for_publish(record)
149
+ key = get_key_from(record)
150
+ value = get_value_from(record)
151
+ @redis.publish key, value
152
+ end
153
+
154
+ def generate_zremrangebyrank_script(key, maxlen, order)
155
+ script = "local key = '" + key.to_s + "'\n"
156
+ script += "local maxlen = " + maxlen.to_s + "\n"
157
+ script += "local order ='" + order.to_s + "'\n"
158
+ script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
159
+ script += "if len > maxlen then\n"
160
+ script += " if order == 'asc' then\n"
161
+ script += " local l = len - maxlen\n"
162
+ script += " if l >= 0 then\n"
163
+ script += " return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
164
+ script += " end\n"
165
+ script += " else\n"
166
+ script += " return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
167
+ script += " end\n"
168
+ script += "end\n"
169
+ return script
170
+ end
171
+
172
+ def generate_ltrim_script(key, maxlen, order)
173
+ script = "local key = '" + key.to_s + "'\n"
174
+ script += "local maxlen = " + maxlen.to_s + "\n"
175
+ script += "local order ='" + order.to_s + "'\n"
176
+ script += "local len = tonumber(redis.call('LLEN', key))\n"
177
+ script += "if len > maxlen then\n"
178
+ script += " if order == 'asc' then\n"
179
+ script += " local l = len - maxlen\n"
180
+ script += " return redis.call('LTRIM', key, l, -1)\n"
181
+ script += " else\n"
182
+ script += " return redis.call('LTRIM', key, 0, maxlen - 1)\n"
183
+ script += " end\n"
184
+ script += "end\n"
185
+ return script
186
+ end
187
+
188
+ def traverse(data, key)
189
+ val = data
190
+ key.split('.').each{ |k|
191
+ if val.has_key?(k)
192
+ val = val[k]
193
+ else
194
+ return nil
195
+ end
196
+ }
197
+ return val
198
+ end
199
+
200
+ def get_key_from(record)
201
+ if @key
202
+ k = @key
203
+ else
204
+ k = traverse(record, @key_path).to_s
205
+ end
206
+ key = @key_prefix + k + @key_suffix
207
+
208
+ raise Fluent::ConfigError, "key is empty" if key == ''
209
+ key
210
+ end
211
+
212
+ def get_value_from(record)
213
+ value = traverse(record, @value_path)
214
+ case @format_type
215
+ when 'json'
216
+ value.to_json
217
+ when 'msgpack'
218
+ value.to_msgpack
219
+ else
220
+ value
221
+ end
222
+ end
223
+
224
+ def get_score_from(record, time)
225
+ if @score_path
226
+ traverse(record, @score_path)
227
+ else
228
+ time
229
+ end
230
+ end
231
+
232
+ def set_key_expire(key)
233
+ if 0 < @key_expire
234
+ @redis.expire key, @key_expire
235
+ end
236
+ end
237
+
238
+ end
239
+ end
data/test/helpers.rb ADDED
@@ -0,0 +1,28 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ begin
4
+ Bundler.setup(:default, :development)
5
+ rescue Bundler::BundlerError => e
6
+ $stderr.puts e.message
7
+ $stderr.puts "Run `bundle install` to install missing gems"
8
+ exit e.status_code
9
+ end
10
+ require 'test/unit'
11
+
12
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
14
+ require 'fluent/test'
15
+ unless ENV.has_key?('VERBOSE')
16
+ nulllogger = Object.new
17
+ nulllogger.instance_eval {|obj|
18
+ def method_missing(method, *args)
19
+ # pass
20
+ end
21
+ }
22
+ $log = nulllogger
23
+ end
24
+
25
+ require 'fluent/plugin/out_redis_store'
26
+
27
+ class Test::Unit::TestCase
28
+ end
@@ -0,0 +1,368 @@
1
+ require 'helpers'
2
+
3
+ require 'redis'
4
+
5
+ $channel = nil
6
+ $message = nil
7
+
8
+ class Redis
9
+ def initialize(options = {})
10
+ end
11
+
12
+ def pipelined
13
+ yield self
14
+ end
15
+
16
+ def set(key, message)
17
+ $command = :set
18
+ $key = key
19
+ $message = message
20
+ end
21
+
22
+ def rpush(key, message)
23
+ $command = :rpush
24
+ $key = key
25
+ $message = message
26
+ end
27
+
28
+ def lpush(key, message)
29
+ $command = :lpush
30
+ $key = key
31
+ $message = message
32
+ end
33
+
34
+ def sadd(key, message)
35
+ $command = :sadd
36
+ $key = key
37
+ $message = message
38
+ end
39
+
40
+ def zadd(key, score, message)
41
+ $command = :zadd
42
+ $key = key
43
+ $score = score
44
+ $message = message
45
+ end
46
+
47
+ def expire(key, ttl)
48
+ $expire_key = key
49
+ $ttl = ttl
50
+ end
51
+
52
+ def publish(channel, message)
53
+ $command = :publish
54
+ $channel = channel
55
+ $message = message
56
+ end
57
+
58
+ def quit
59
+ end
60
+ end
61
+
62
+ class RedisStoreOutputTest < Test::Unit::TestCase
63
+ def setup
64
+ Fluent::Test.setup
65
+ end
66
+
67
+ def create_driver(conf)
68
+ Fluent::Test::BufferedOutputTestDriver.new(Fluent::RedisStoreOutput).configure(conf)
69
+ end
70
+
71
+ def test_configure_defaults
72
+ config = %[
73
+ key_path a
74
+ score_path b
75
+ ]
76
+ d = create_driver(config)
77
+ assert_equal "127.0.0.1", d.instance.host
78
+ assert_equal 6379, d.instance.port
79
+ assert_equal nil, d.instance.path
80
+ assert_equal nil, d.instance.password
81
+ assert_equal 0, d.instance.db
82
+ assert_equal 5.0, d.instance.timeout
83
+ assert_equal 'json', d.instance.format_type
84
+ assert_equal '', d.instance.key_prefix
85
+ assert_equal '', d.instance.key_suffix
86
+ assert_equal 'zset', d.instance.store_type
87
+ assert_equal 'a', d.instance.key_path
88
+ assert_equal nil, d.instance.key
89
+ assert_equal 'b', d.instance.score_path
90
+ assert_equal '', d.instance.value_path
91
+ assert_equal -1, d.instance.key_expire
92
+ assert_equal -1, d.instance.value_expire
93
+ assert_equal -1, d.instance.value_length
94
+ assert_equal 'asc', d.instance.order
95
+ assert_equal nil, d.instance.collision_policy
96
+ end
97
+
98
+ def test_configure_host_port_db
99
+ config = %[
100
+ host 192.168.2.3
101
+ port 9999
102
+ password abc
103
+ db 3
104
+ timeout 7
105
+ key a
106
+ score_path b
107
+ ]
108
+ d = create_driver(config)
109
+ assert_equal "192.168.2.3", d.instance.host
110
+ assert_equal 9999, d.instance.port
111
+ assert_equal nil, d.instance.path
112
+ assert_equal 'abc', d.instance.password
113
+ assert_equal 3, d.instance.db
114
+ assert_equal 7.0, d.instance.timeout
115
+ assert_equal nil, d.instance.key_path
116
+ assert_equal 'a', d.instance.key
117
+ end
118
+
119
+ def test_configure_path
120
+ config = %[
121
+ path /tmp/foo.sock
122
+ key a
123
+ score_path b
124
+ ]
125
+ d = create_driver(config)
126
+ assert_equal "/tmp/foo.sock", d.instance.path
127
+ end
128
+
129
+ def test_configure_exception
130
+ assert_raise(Fluent::ConfigError) do
131
+ create_driver(%[])
132
+ end
133
+ end
134
+
135
+ # def test_write
136
+ # d = create_driver(CONFIG1)
137
+ #
138
+ # time = Time.parse("2011-01-02 13:14:15 UTC").to_i
139
+ # d.emit({ "foo" => "bar" }, time)
140
+ # d.run
141
+ #
142
+ # assert_equal "test", $channel
143
+ # assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
144
+ # end
145
+
146
+ def get_time
147
+ Time.parse("2011-01-02 13:14:15 UTC").to_i
148
+ end
149
+
150
+ # it should return whole message
151
+ def test_omit_value_path
152
+ config = %[
153
+ format_type plain
154
+ store_type string
155
+ key_path user
156
+ ]
157
+ d = create_driver(config)
158
+ message = {
159
+ 'user' => 'george',
160
+ 'stat' => { 'attack' => 7 }
161
+ }
162
+ $ttl = nil
163
+ d.emit(message, get_time)
164
+ d.run
165
+
166
+ assert_equal "george", $key
167
+ assert_equal message, $message
168
+ assert_equal nil, $ttl
169
+ end
170
+
171
+ def test_key_value_paths
172
+ config = %[
173
+ format_type plain
174
+ store_type string
175
+ key_path user.name
176
+ value_path stat.attack
177
+ key_expire 3
178
+ ]
179
+ d = create_driver(config)
180
+ message = {
181
+ 'user' => { 'name' => 'george' },
182
+ 'stat' => { 'attack' => 7 }
183
+ }
184
+ $ttl = nil
185
+ d.emit(message, get_time)
186
+ d.run
187
+
188
+ assert_equal "george", $key
189
+ assert_equal 7, $message
190
+ assert_equal 3, $ttl
191
+ end
192
+
193
+ def test_json
194
+ config = %[
195
+ format_type json
196
+ store_type string
197
+ key_path user
198
+ ]
199
+ d = create_driver(config)
200
+ message = {
201
+ 'user' => 'george',
202
+ 'stat' => { 'attack' => 7 }
203
+ }
204
+ $ttl = nil
205
+ d.emit(message, get_time)
206
+ d.run
207
+
208
+ assert_equal "george", $key
209
+ assert_equal message.to_json, $message
210
+ end
211
+
212
+ def test_msgpack
213
+ config = %[
214
+ format_type msgpack
215
+ store_type string
216
+ key_path user
217
+ ]
218
+ d = create_driver(config)
219
+ message = {
220
+ 'user' => 'george',
221
+ 'stat' => { 'attack' => 7 }
222
+ }
223
+ $ttl = nil
224
+ d.emit(message, get_time)
225
+ d.run
226
+
227
+ assert_equal "george", $key
228
+ assert_equal message.to_msgpack, $message
229
+ end
230
+
231
+ def test_list_asc
232
+ config = %[
233
+ format_type plain
234
+ store_type list
235
+ key_path user
236
+ ]
237
+ d = create_driver(config)
238
+ message = {
239
+ 'user' => 'george',
240
+ 'stat' => { 'attack' => 7 }
241
+ }
242
+ d.emit(message, get_time)
243
+ d.run
244
+
245
+ assert_equal :rpush, $command
246
+ assert_equal "george", $key
247
+ assert_equal message, $message
248
+ end
249
+
250
+ def test_list_desc
251
+ config = %[
252
+ format_type plain
253
+ store_type list
254
+ key_path user
255
+ order desc
256
+ ]
257
+ d = create_driver(config)
258
+ message = {
259
+ 'user' => 'george',
260
+ 'stat' => { 'attack' => 7 }
261
+ }
262
+ d.emit(message, get_time)
263
+ d.run
264
+
265
+ assert_equal :lpush, $command
266
+ assert_equal "george", $key
267
+ assert_equal message, $message
268
+ end
269
+
270
+ def test_set
271
+ config = %[
272
+ format_type plain
273
+ store_type set
274
+ key_path user
275
+ order desc
276
+ ]
277
+ d = create_driver(config)
278
+ message = {
279
+ 'user' => 'george',
280
+ 'stat' => { 'attack' => 7 }
281
+ }
282
+ d.emit(message, get_time)
283
+ d.run
284
+
285
+ assert_equal :sadd, $command
286
+ assert_equal "george", $key
287
+ assert_equal message, $message
288
+ end
289
+
290
+ def test_zset
291
+ config = %[
292
+ format_type plain
293
+ store_type zset
294
+ key_path user
295
+ score_path result
296
+ ]
297
+
298
+ d = create_driver(config)
299
+ message = {
300
+ 'user' => 'george',
301
+ 'stat' => { 'attack' => 7 },
302
+ 'result' => 81
303
+ }
304
+ d.emit(message, get_time)
305
+ d.run
306
+
307
+ assert_equal :zadd, $command
308
+ assert_equal "george", $key
309
+ assert_equal 81, $score
310
+ assert_equal message, $message
311
+ end
312
+
313
+ def test_zset_with_no_score_path
314
+ config = %[
315
+ format_type plain
316
+ store_type zset
317
+ key_path user
318
+ ]
319
+
320
+ d = create_driver(config)
321
+ message = {
322
+ 'user' => 'george',
323
+ 'stat' => { 'attack' => 7 },
324
+ 'result' => 81
325
+ }
326
+ d.emit(message, get_time)
327
+ d.run
328
+
329
+ assert_equal :zadd, $command
330
+ assert_equal "george", $key
331
+ assert_equal get_time, $score
332
+ assert_equal message, $message
333
+ end
334
+
335
+ def test_publish
336
+ config = %[
337
+ format_type plain
338
+ store_type publish
339
+ key_path user
340
+ ]
341
+
342
+ d = create_driver(config)
343
+ message = {
344
+ 'user' => 'george'
345
+ }
346
+ d.emit(message, get_time)
347
+ d.run
348
+
349
+ assert_equal :publish, $command
350
+ assert_equal "george", $channel
351
+ assert_equal message, $message
352
+ end
353
+
354
+ def test_empty_key
355
+ config = %[
356
+ format_type plain
357
+ store_type string
358
+ key_path none
359
+ ]
360
+
361
+ d = create_driver(config)
362
+ message = { 'user' => 'george' }
363
+ d.emit(message, get_time)
364
+ assert_raise(Fluent::ConfigError) do
365
+ d.run
366
+ end
367
+ end
368
+ end
metadata ADDED
@@ -0,0 +1,112 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-redis-store-gabfl
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.2
5
+ platform: ruby
6
+ authors:
7
+ - moaikids
8
+ - HANAI Tohru aka pokehanai
9
+ - Gabriel Bordeaux
10
+ autorequire:
11
+ bindir: bin
12
+ cert_chain: []
13
+ date: 2017-07-10 00:00:00.000000000 Z
14
+ dependencies:
15
+ - !ruby/object:Gem::Dependency
16
+ name: rake
17
+ requirement: !ruby/object:Gem::Requirement
18
+ requirements:
19
+ - - '>='
20
+ - !ruby/object:Gem::Version
21
+ version: '0'
22
+ type: :development
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ requirements:
26
+ - - '>='
27
+ - !ruby/object:Gem::Version
28
+ version: '0'
29
+ - !ruby/object:Gem::Dependency
30
+ name: test-unit
31
+ requirement: !ruby/object:Gem::Requirement
32
+ requirements:
33
+ - - '>='
34
+ - !ruby/object:Gem::Version
35
+ version: '0'
36
+ type: :development
37
+ prerelease: false
38
+ version_requirements: !ruby/object:Gem::Requirement
39
+ requirements:
40
+ - - '>='
41
+ - !ruby/object:Gem::Version
42
+ version: '0'
43
+ - !ruby/object:Gem::Dependency
44
+ name: fluentd
45
+ requirement: !ruby/object:Gem::Requirement
46
+ requirements:
47
+ - - '>='
48
+ - !ruby/object:Gem::Version
49
+ version: '0'
50
+ type: :runtime
51
+ prerelease: false
52
+ version_requirements: !ruby/object:Gem::Requirement
53
+ requirements:
54
+ - - '>='
55
+ - !ruby/object:Gem::Version
56
+ version: '0'
57
+ - !ruby/object:Gem::Dependency
58
+ name: redis
59
+ requirement: !ruby/object:Gem::Requirement
60
+ requirements:
61
+ - - '>='
62
+ - !ruby/object:Gem::Version
63
+ version: '0'
64
+ type: :runtime
65
+ prerelease: false
66
+ version_requirements: !ruby/object:Gem::Requirement
67
+ requirements:
68
+ - - '>='
69
+ - !ruby/object:Gem::Version
70
+ version: '0'
71
+ description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
72
+ email: pypi@gab.lc
73
+ executables: []
74
+ extensions: []
75
+ extra_rdoc_files: []
76
+ files:
77
+ - .gitignore
78
+ - AUTHORS
79
+ - Gemfile
80
+ - README.md
81
+ - Rakefile
82
+ - fluent-plugin-redis-store-gabfl.gemspec
83
+ - lib/fluent/plugin/out_redis_store.rb
84
+ - test/helpers.rb
85
+ - test/plugin/test_out_redis_publish.rb
86
+ homepage: https://github.com/gabfl/fluent-plugin-redis-store
87
+ licenses:
88
+ - Apache License Version 2.0
89
+ metadata: {}
90
+ post_install_message:
91
+ rdoc_options: []
92
+ require_paths:
93
+ - lib
94
+ required_ruby_version: !ruby/object:Gem::Requirement
95
+ requirements:
96
+ - - '>='
97
+ - !ruby/object:Gem::Version
98
+ version: '0'
99
+ required_rubygems_version: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - '>='
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ requirements: []
105
+ rubyforge_project:
106
+ rubygems_version: 2.0.14.1
107
+ signing_key:
108
+ specification_version: 4
109
+ summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
110
+ test_files:
111
+ - test/helpers.rb
112
+ - test/plugin/test_out_redis_publish.rb