fluent-plugin-ddl-redis-store 0.2.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 1f204dfec7c332be1ea6915acb4a008cb957e10bf1686116403fe717fbd535d7
+  data.tar.gz: 281b6c3ce41822f415bb1774b3e9e3729cd96604ac15c2838207adbbad7854f6
+SHA512:
+  metadata.gz: c7fb2cfb2a447f2a1d90e9b1d60467c5c24f5794c57b115b6cd522d2126ee66c911945eaea79533f023f2f4e67c1d68dd766b7d7bf6588434dc0d0f623575d64
+  data.tar.gz: d6fcfc4cb3e89b2602f327a43fb73aca598edcad5e6d9071748dac6455a71b7b2ecf3fbf15bc3c7b52daa0a5578d58668e452920ce8e17d9d955b754ed18d4eb

data/.gitignore

@@ -0,0 +1 @@
+*.gem

data/AUTHORS

@@ -0,0 +1,2 @@
+moaikids
+HANAI Tohru aka pokehanai

data/Gemfile

@@ -0,0 +1,3 @@
+source "http://rubygems.org"
+
+gemspec

data/README.md

@@ -0,0 +1,183 @@
+Redis Output Plugin For fluentd
+===============================
+[Fluentd][] output plugin to upload/publish event data to [Redis][] storage.
+
+[Fluentd]: http://fluentd.org/
+[Redis]: http://redis.io/
+
+Requirements
+------------
+
+| fluent-plugin-redis-store | fluentd | ruby |
+|------------------------|---------|------|
+| >= 0.2.0 | >= v0.14.15 | >= 2.1 |
+|  < 0.2.0 | >= v0.12.0 | >= 1.9 |
+
+Background
+----------
+
+This is folked project from [fluent-plugin-redisstore][].
+
+[fluent-plugin-redisstore]: https://github.com/moaikids/fluent-plugin-redisstore
+
+Features
+--------
+
+#### Supported Redis commands
+
+Currently the plugin supports following Redis commands:
+
+- **set** by `string` type (of the plugin)
+- **lpush**/**rpush** by `list` type
+- **sadd** by `set` type
+- **zadd** by `zset` type
+- **publish** by `publish` type
+
+#### Supported _value_ format
+
+- plain(as is)
+- JSON
+- [MessagePack](http://msgpack.org/)
+
+#### _key_ string for Redis storage
+
+Redis commands require _key_ and _value_.  
+For _key_, the plugin supports either way;
+
+1. Specify a fixed key.  
+   You can do this simply using `key` option in td-agent configuration file.
+
+   ```apache
+   type redis_store
+   key userdata
+   ```
+
+2. Lookup a key string in every event data by a lookup path.  
+   If event data have structured data like
+
+   ```javascript
+   { "user": { "name": "Kei" } }
+   ```
+
+   and you want to use each name of user, you can use `key_path` option.
+
+   ```apache
+   type redis_store
+   key_path user.name
+   ```
+
+   With the above data, `Kei` will be a _key_.
+
+In addition, `key_prefix` and `key_suffix` are useful in some cases. Both are available either `key` and `key_path`
+
+   ```apache
+   type redis_store
+   key_path user.name
+   key_prefix ouruser.
+   key_suffix .accesslog
+   ```
+
+With the previous data, _key_ will be `outuser.Kei.accesslog`.
+
+#### _value_ data for Redis storage
+
+To determine what _value_ in every event data to be srtored, you have two options;
+
+1. Store extracted data in event data, by a lookup path with `value_path` option.  
+   It works like `key_path`.
+2. Store whole data.  
+   This is default behavior. To do it, simply omit `value_path` option.
+
+Installation
+------------
+
+```bash
+fluent-gem install fluent-plugin-redis-store
+
+# or if you are using td-agent:
+td-agent-gem install fluent-plugin-redis-store
+```
+
+Configuration
+-------------
+
+### Redis connection
+
+| Key        | Type   | Required?   |                  Default | Description                                       |
+| :----      | :----- | :---------- | :----------------------- | :------------                                     |
+| `host`     | string | Optional    |                127.0.0.1 | host name of Redis server                         |
+| `port`     | int    | Optional    |                     6379 | port number of Redis server                       |
+| `password` | string | Optional    |                          | password for Redis connection                     |
+| `path`     | string | Optional    |                          | To connect via Unix socket, try '/tmp/redis.sock' |
+| `db`       | int    | Optional    |                        0 | DB number of Redis                                |
+| `timeout`  | float  | Optional    |                      5.0 | connection timeout in seconds                     |
+
+### common options for storages
+
+| Key              | Type   | Default                  | Description                                          |
+| :----            | :----- | :----------------------- | :------------                                        |
+| `key`            | string |                          | Fixed _key_ used to store(publish) in Redis          |
+| `key_path`       | string |                          | path to lookup for _key_ in the event data           |
+| `key_prefix`     | string |                          | prefix of _key_                                      |
+| `key_suffix`     | string |                          | suffix of _key_                                      |
+| `value_path`     | string | (whole event data)       | path to lookup for _value_ in the event data         |
+| `store_type`     | string | zset                     | `string`/`list`/`set`/`zset`/`publish`               |
+| `format_type`    | string | plain                    | format type for _value_ (`plain`/`json`/`msgpack`)   |
+| `key_expire`     | int    | -1                       | If set, the key will be expired in specified seconds |
+| `flush_interval` | time   | 1                        | Time interval which events will be flushed to Redis  |
+
+Note: either `key` or `key_path` is required.
+
+### `string` storage specific options
+
+| Key    | Type   | Default                  | Description                                 |
+| :----  | :----- | :----------------------- | :------------                               |
+| `type` | string |                          | Fixed _key_ used to store(publish) in Redis |
+No more options than common options.
+
+### `list` storage specific options
+
+| Key     | Type   | Default                  | Description                         |
+| :----   | :----- | :----------------------- | :------------                       |
+| `order` | string | asc                      | `asc`: **rpush**, `desc`: **lpush** |
+
+### `set` storage specific options
+
+No more options than common options.
+
+### `zset` storage specific options
+
+| Key                 | Type   | Default                  | Description                                                             |
+| :----               | :----- | :----------------------- | :------------                                                           |
+| `score_path`        | string | (_time_ of log event)    | path to lookup for _score_ in the event data                            |
+| `collision_policy`  | string | (nil)                    | Only update elements that already exist (XX) or add a new element (NX)  |
+| `value_expire`      | int    |                          | value expiration in seconds                                             |
+
+If `value_expire` is set, the plugin assumes that the _score_ in the **SortedSet** is
+based on *timestamp* and it deletes expired _members_ every after new event data arrives.
+
+### `publish` storage specific options
+
+No more options than common options.
+
+
+Contributors
+------------
+
+moaikids  
+HANAI tohru  
+Mohit Khanna  
+yamada-shinji  
+Heitor de Souza  
+Hiroshi Hatake  
+Gabriel Bordeaux  
+
+Copyright
+---------
+
+Copyright (c) 2013 moaikids  
+Copyright (c) 2014 HANAI Tohru  
+
+License
+-------
+Apache License, Version 2.0

data/Rakefile

@@ -0,0 +1,11 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+    test.libs << 'lib' << 'test'
+    test.pattern = 'test/**/test_*.rb'
+    test.verbose = true
+end
+
+task :default => :test

data/fluent-plugin-ddl-redis-store.gemspec

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+Gem::Specification.new do |gem|
+    gem.name        = "fluent-plugin-ddl-redis-store"
+    gem.email       = "hanai@pokelabo.co.jp"
+    gem.version     = "0.2.1"
+    gem.authors     = ["moaikids", "HANAI Tohru aka pokehanai"]
+    gem.licenses    = ["Apache License Version 2.0"]
+    gem.summary     = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd}
+    gem.description = %q{Redis(zset/set/list/string/publish) output plugin for Fluentd...}
+    gem.homepage    = "https://github.com/pokehanai/fluent-plugin-redis-store"
+
+    gem.files         = `git ls-files`.split($\)
+    gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+    gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+    gem.require_paths = ["lib"]
+
+    gem.add_development_dependency "rake"
+    gem.add_development_dependency "test-unit"
+    gem.add_runtime_dependency "fluentd", [">= 0.14.15", "< 2"]
+    gem.add_runtime_dependency "redis"
+end

data/lib/fluent/plugin/out_redis_store.rb

@@ -0,0 +1,263 @@
+require 'fluent/plugin/output'
+
+module Fluent::Plugin
+  class RedisStoreOutput < Output
+    Fluent::Plugin.register_output('redis_store', self)
+
+    helpers :compat_parameters
+
+    DEFAULT_BUFFER_TYPE = "memory"
+
+    # redis connection
+    config_param :host,      :string,  :default => '127.0.0.1'
+    config_param :port,      :integer, :default => 6379
+    config_param :path,      :string,  :default => nil
+    config_param :password,  :string,  :default => nil
+    config_param :db,        :integer, :default => 0
+    config_param :timeout,   :float,   :default => 5.0
+
+    # redis command and parameters
+    config_param :format_type,       :string,  :default => 'json'
+    config_param :store_type,        :string,  :default => 'zset'
+    config_param :key_prefix,        :string,  :default => ''
+    config_param :key_suffix,        :string,  :default => ''
+    config_param :key,               :string,  :default => nil
+    config_param :key_path,          :string,  :default => nil
+    config_param :score_path,        :string,  :default => nil
+    config_param :value_path,        :string,  :default => ''
+    config_param :key_expire,        :integer, :default => -1
+    config_param :value_expire,      :integer, :default => -1
+    config_param :value_length,      :integer, :default => -1
+    config_param :order,             :string,  :default => 'asc'
+    config_param :collision_policy,  :string,  :default => nil
+    config_set_default :flush_interval, 1
+
+    config_section :buffer do
+      config_set_default :@type, DEFAULT_BUFFER_TYPE
+    end
+
+    def initialize
+      super
+      require 'redis' unless defined?(Redis) == 'constant'
+      require 'msgpack'
+    end
+
+    def configure(conf)
+      compat_parameters_convert(conf, :buffer)
+      super
+
+      if @key_path == nil and @key == nil
+        raise Fluent::ConfigError, "either key_path or key is required"
+      end
+    end
+
+    def start
+      super
+      if @path
+        @redis = Redis.new(:path => @path, :password => @password,
+                           :timeout => @timeout, :thread_safe => true, :db => @db)
+      else
+        @redis = Redis.new(:host => @host, :port => @port, :password => @password,
+                           :timeout => @timeout, :thread_safe => true, :db => @db)
+      end
+    end
+
+    def shutdown
+      @redis.quit
+      super
+    end
+
+    def format(tag, time, record)
+      [tag, time.to_f, record].to_msgpack
+    end
+
+    def formatted_to_msgpack_binary?
+      true
+    end
+
+    def multi_workers_ready?
+      true
+    end
+
+    def write(chunk)
+      @redis.pipelined {
+        chunk.open { |io|
+          begin
+            MessagePack::Unpacker.new(io).each { |message|
+              begin
+                (_, time, record) = message
+                case @store_type
+                when 'zset'
+                  operation_for_zset(record, time)
+                when 'set'
+                  operation_for_set(record)
+                when 'list'
+                  operation_for_list(record)
+                when 'string'
+                  operation_for_string(record)
+                when 'publish'
+                  operation_for_publish(record)
+                end
+              rescue NoMethodError => e
+                puts e
+              rescue Encoding::UndefinedConversionError => e
+                log.error "Plugin error: " + e.to_s
+                log.error "Original record: " + record.to_s
+                puts e
+              end
+            }
+          rescue EOFError
+            # EOFError always occured when reached end of chunk.
+          end
+        }
+      }
+    end
+
+    def operation_for_zset(record, time)
+      key = get_key_from(record)
+      value = get_value_from(record)
+      score = get_score_from(record, time)
+      if @collision_policy
+        if @collision_policy == 'NX'
+          @redis.zadd(key, score, value, :nx => true)
+        elsif @collision_policy == 'XX'
+          @redis.zadd(key, score, value, :xx => true)
+        end
+      else
+        @redis.zadd(key, score, value)
+      end
+
+      set_key_expire key
+      if 0 < @value_expire
+        now = Time.now.to_i
+        @redis.zremrangebyscore key , '-inf' , (now - @value_expire)
+      end
+      if 0 < @value_length
+        script = generate_zremrangebyrank_script(key, @value_length, @order)
+        @redis.eval script
+      end
+    end
+
+    def operation_for_set(record)
+      key = get_key_from(record)
+      value = get_value_from(record)
+      @redis.sadd key, value
+      set_key_expire key
+    end
+
+    def operation_for_list(record)
+      key = get_key_from(record)
+      value = get_value_from(record)
+
+      if @order == 'asc'
+        @redis.rpush key, value
+      else
+        @redis.lpush key, value
+      end
+      set_key_expire key
+      if 0 < @value_length
+        script = generate_ltrim_script(key, @value_length, @order)
+        @redis.eval script
+      end
+    end
+
+    def operation_for_string(record)
+      key = get_key_from(record)
+      value = get_value_from(record)
+      @redis.set key, value
+
+      set_key_expire key
+    end
+
+    def operation_for_publish(record)
+      key = get_key_from(record)
+      value = get_value_from(record)
+      @redis.publish key, value
+    end
+
+    def generate_zremrangebyrank_script(key, maxlen, order)
+      script  = "local key = '" + key.to_s + "'\n"
+      script += "local maxlen = " + maxlen.to_s + "\n"
+      script += "local order ='" + order.to_s + "'\n"
+      script += "local len = tonumber(redis.call('ZCOUNT', key, '-inf', '+inf'))\n"
+      script += "if len > maxlen then\n"
+      script += "    if order == 'asc' then\n"
+      script += "       local l = len - maxlen\n"
+      script += "       if l >= 0 then\n"
+      script += "           return redis.call('ZREMRANGEBYRANK', key, 0, l)\n"
+      script += "       end\n"
+      script += "    else\n"
+      script += "       return redis.call('ZREMRANGEBYRANK', key, maxlen, -1)\n"
+      script += "    end\n"
+      script += "end\n"
+      return script
+    end
+
+    def generate_ltrim_script(key, maxlen, order)
+      script  = "local key = '" + key.to_s + "'\n"
+      script += "local maxlen = " + maxlen.to_s + "\n"
+      script += "local order ='" + order.to_s + "'\n"
+      script += "local len = tonumber(redis.call('LLEN', key))\n"
+      script += "if len > maxlen then\n"
+      script += "    if order == 'asc' then\n"
+      script += "        local l = len - maxlen\n"
+      script += "        return redis.call('LTRIM', key, l, -1)\n"
+      script += "    else\n"
+      script += "        return redis.call('LTRIM', key, 0, maxlen - 1)\n"
+      script += "    end\n"
+      script += "end\n"
+      return script
+    end
+
+    def traverse(data, key)
+      val = data
+      key.split('.').each{ |k|
+        if val.has_key?(k)
+          val = val[k]
+        else
+          return nil
+        end
+      }
+      return val
+    end
+
+    def get_key_from(record)
+      if @key
+        k = @key
+      else
+        k = traverse(record, @key_path).to_s
+      end
+      key = @key_prefix + k + @key_suffix
+
+      raise Fluent::ConfigError, "key is empty" if key == ''
+      key
+    end
+
+    def get_value_from(record)
+      value = traverse(record, @value_path)
+      case @format_type
+      when 'json'
+        Yajl.dump(value)
+      when 'msgpack'
+        value.to_msgpack
+      else
+        value
+      end
+    end
+
+    def get_score_from(record, time)
+      if @score_path
+        traverse(record, @score_path)
+      else
+        time
+      end
+    end
+
+    def set_key_expire(key)
+      if 0 < @key_expire
+        @redis.expire key, @key_expire
+      end
+    end
+
+  end
+end

data/test/helpers.rb

@@ -0,0 +1,31 @@
+require 'rubygems'
+require 'bundler'
+begin
+  Bundler.setup(:default, :development)
+rescue Bundler::BundlerError => e
+  $stderr.puts e.message
+  $stderr.puts "Run `bundle install` to install missing gems"
+  exit e.status_code
+end
+require 'test/unit'
+
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+$LOAD_PATH.unshift(File.dirname(__FILE__))
+require 'fluent/test'
+require 'fluent/test/driver/output'
+require 'fluent/test/helpers'
+unless ENV.has_key?('VERBOSE')
+  nulllogger = Object.new
+  nulllogger.instance_eval {|obj|
+    def method_missing(method, *args)
+      # pass
+    end
+  }
+  $log = nulllogger
+end
+
+require 'fluent/plugin/out_redis_store'
+
+class Test::Unit::TestCase
+  include Fluent::Test::Helpers
+end

data/test/plugin/test_out_redis_publish.rb

@@ -0,0 +1,418 @@
+require 'helpers'
+
+$channel = nil
+$message = nil
+
+class Redis
+  def initialize(options = {})
+  end
+
+  def pipelined
+    yield self
+  end
+
+  def set(key, message)
+    $command = :set
+    $key = key
+    $message = message
+  end
+
+  def rpush(key, message)
+    $command = :rpush
+    $key = key
+    $message = message
+  end
+
+  def lpush(key, message)
+    $command = :lpush
+    $key = key
+    $message = message
+  end
+
+  def sadd(key, message)
+    $command = :sadd
+    $key = key
+    $message = message
+  end
+
+  def zadd(key, score, message)
+    $command = :zadd
+    $key = key
+    $score = score
+    $message = message
+  end
+
+  def expire(key, ttl)
+    $expire_key = key
+    $ttl = ttl
+  end
+
+  def publish(channel, message)
+    $command = :publish
+    $channel = channel
+    $message = message
+  end
+
+  def quit
+  end
+end
+
+class RedisStoreOutputTest < Test::Unit::TestCase
+  def setup
+    Fluent::Test.setup
+  end
+
+  def create_driver(conf)
+    Fluent::Test::Driver::Output.new(Fluent::Plugin::RedisStoreOutput).configure(conf)
+  end
+
+  def test_configure_defaults
+    config = %[
+      key_path a
+      score_path b
+    ]
+    d = create_driver(config)
+    assert_equal("127.0.0.1", d.instance.host)
+    assert_equal(6379, d.instance.port)
+    assert_equal(nil, d.instance.path)
+    assert_equal(nil, d.instance.password)
+    assert_equal(0, d.instance.db)
+    assert_equal(5.0, d.instance.timeout)
+    assert_equal('json', d.instance.format_type)
+    assert_equal('', d.instance.key_prefix)
+    assert_equal('', d.instance.key_suffix)
+    assert_equal('zset', d.instance.store_type)
+    assert_equal('a', d.instance.key_path)
+    assert_equal(nil, d.instance.key)
+    assert_equal('b', d.instance.score_path)
+    assert_equal('', d.instance.value_path)
+    assert_equal(-1, d.instance.key_expire)
+    assert_equal(-1, d.instance.value_expire)
+    assert_equal(-1, d.instance.value_length)
+    assert_equal('asc', d.instance.order)
+    assert_equal(nil, d.instance.collision_policy)
+  end
+
+  def test_configure_host_port_db
+    config = %[
+      host 192.168.2.3
+      port 9999
+      password abc
+      db 3
+      timeout 7
+      key a
+      score_path b
+    ]
+    d = create_driver(config)
+    assert_equal "192.168.2.3", d.instance.host
+    assert_equal 9999, d.instance.port
+    assert_equal nil, d.instance.path
+    assert_equal 'abc', d.instance.password
+    assert_equal 3, d.instance.db
+    assert_equal 7.0, d.instance.timeout
+    assert_equal nil, d.instance.key_path
+    assert_equal 'a', d.instance.key
+  end
+
+  def test_configure_path
+    config = %[
+      path /tmp/foo.sock
+      key a
+      score_path b
+    ]
+    d = create_driver(config)
+    assert_equal "/tmp/foo.sock", d.instance.path
+  end
+
+  def test_configure_exception
+    assert_raise(Fluent::ConfigError) do
+      create_driver(%[])
+    end
+  end
+
+#  def test_write
+#    d = create_driver(CONFIG1)
+#
+#    time = event_time("2011-01-02 13:14:15 UTC")
+#    d.run(default_tag 'test') do
+#      d.feed({ "foo" => "bar" }, time)
+#    end
+#
+#    assert_equal "test", $channel
+#    assert_equal(%Q[{"foo":"bar","time":#{time}}], $message)
+#  end
+
+  def get_time
+    event_time("2011-01-02 13:14:15 UTC")
+  end
+
+  # it should return whole message
+  def test_omit_value_path
+    config = %[
+      format_type plain
+      store_type string
+      key_path   user
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 }
+    }
+    $ttl = nil
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal "george", $key
+    assert_equal message, $message
+    assert_equal nil, $ttl
+  end
+
+  def test_key_value_paths
+    config = %[
+      format_type plain
+      store_type string
+      key_path   user.name
+      value_path stat.attack
+      key_expire 3
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => { 'name' => 'george' },
+      'stat' => { 'attack' => 7 }
+    }
+    $ttl = nil
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal "george", $key
+    assert_equal 7, $message
+    assert_equal 3, $ttl
+  end
+
+  def test_json
+    config = %[
+      format_type json
+      store_type string
+      key_path   user
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 }
+    }
+    $ttl = nil
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal "george", $key
+    assert_equal message.to_json, $message
+  end
+
+  def test_json_with_malformed_utf8
+    config = %[
+      format_type json
+      store_type string
+      key_path   user
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'data' => ([0x80, 0x80].map {|x| x.chr}.join).force_encoding('UTF-8')
+
+    }
+    $ttl = nil
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal message, Yajl.load($message, :check_utf8 => false)
+  end
+  
+  def test_msgpack
+    config = %[
+      format_type msgpack
+      store_type string
+      key_path   user
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 }
+    }
+    $ttl = nil
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal "george", $key
+    assert_equal message.to_msgpack, $message
+  end
+
+  def test_list_asc
+    config = %[
+      format_type plain
+      store_type list
+      key_path   user
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 }
+    }
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal :rpush, $command
+    assert_equal "george", $key
+    assert_equal message, $message
+  end
+
+  def test_list_desc
+    config = %[
+      format_type plain
+      store_type list
+      key_path   user
+      order      desc
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 }
+    }
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal :lpush, $command
+    assert_equal "george", $key
+    assert_equal message, $message
+  end
+
+  def test_set
+    config = %[
+      format_type plain
+      store_type set
+      key_path   user
+      order      desc
+    ]
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 }
+    }
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal :sadd, $command
+    assert_equal "george", $key
+    assert_equal message, $message
+  end
+
+  def test_zset
+    config = %[
+      format_type plain
+      store_type zset
+      key_path   user
+      score_path result
+    ]
+
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 },
+      'result' => 81
+    }
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal :zadd, $command
+    assert_equal "george", $key
+    assert_equal 81, $score
+    assert_equal message, $message
+  end
+
+  def test_zset_with_no_score_path
+    config = %[
+      format_type plain
+      store_type zset
+      key_path   user
+    ]
+
+    d = create_driver(config)
+    message = {
+      'user' => 'george',
+      'stat' => { 'attack' => 7 },
+      'result' => 81
+    }
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal :zadd, $command
+    assert_equal "george", $key
+    assert_equal get_time, $score
+    assert_equal message, $message
+  end
+
+  def test_publish
+    config = %[
+      format_type plain
+      store_type publish
+      key_path   user
+    ]
+
+    d = create_driver(config)
+    message = {
+      'user' => 'george'
+    }
+    d.run(default_tag: 'test') do
+      d.feed(get_time, message)
+    end
+
+    assert_equal :publish, $command
+    assert_equal "george", $channel
+    assert_equal message, $message
+  end
+
+  def test_empty_key
+    config = %[
+      format_type plain
+      store_type string
+      key_path   none
+    ]
+
+    d = create_driver(config)
+    message = { 'user' => 'george' }
+    suppress_output do
+      assert_raise(Fluent::ConfigError) do
+        d.run(default_tag: 'test') do
+          d.feed(get_time, message)
+        end
+      end
+    end
+  end
+
+  def suppress_output
+    begin
+      original_stderr = $stderr.clone
+      original_stdout = $stdout.clone
+      $stderr.reopen(File.new('/dev/null', 'w'))
+      $stdout.reopen(File.new('/dev/null', 'w'))
+      retval = yield
+    rescue Exception => e
+      $stdout.reopen(original_stdout)
+      $stderr.reopen(original_stderr)
+      raise e
+    ensure
+      $stdout.reopen(original_stdout)
+      $stderr.reopen(original_stderr)
+    end
+    retval
+  end
+end

metadata

@@ -0,0 +1,116 @@
+--- !ruby/object:Gem::Specification
+name: fluent-plugin-ddl-redis-store
+version: !ruby/object:Gem::Version
+  version: 0.2.1
+platform: ruby
+authors:
+- moaikids
+- HANAI Tohru aka pokehanai
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2024-11-20 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: test-unit
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: fluentd
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.14.15
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.14.15
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2'
+- !ruby/object:Gem::Dependency
+  name: redis
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Redis(zset/set/list/string/publish) output plugin for Fluentd...
+email: hanai@pokelabo.co.jp
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- AUTHORS
+- Gemfile
+- README.md
+- Rakefile
+- fluent-plugin-ddl-redis-store.gemspec
+- lib/fluent/plugin/out_redis_store.rb
+- test/helpers.rb
+- test/plugin/test_out_redis_publish.rb
+homepage: https://github.com/pokehanai/fluent-plugin-redis-store
+licenses:
+- Apache License Version 2.0
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.3.1
+signing_key: 
+specification_version: 4
+summary: Redis(zset/set/list/string/publish) output plugin for Fluentd
+test_files:
+- test/helpers.rb
+- test/plugin/test_out_redis_publish.rb