fluent-plugin-growthforecast 0.1.4 → 0.1.5

data/Gemfile

@@ -1,16 +1,4 @@
-source "http://rubygems.org"
-# Add dependencies required to use your gem here.
-# Example:
-#   gem "activesupport", ">= 2.3.5"
+source 'https://rubygems.org'
 
-# Add dependencies to develop your gem here.
-# Include everything needed to run rake, tests, features, etc.
-group :development do
-  gem "shoulda", ">= 0"
-  gem "bundler", "~> 1.0.0"
-  gem "jeweler", "~> 1.6.4"
-  gem "simplecov", ">= 0"
-end
-
-gem "fluentd"
-gem "rdoc"
+# Specify your gem's dependencies in fluent-plugin-growthforecast.gemspec
+gemspec

data/README.md

@@ -0,0 +1,82 @@
+# fluent-plugin-growthforecast
+
+## GrowthForecastOutput
+
+Plugin to output numbers(metrics) to 'GrowthForecast', metrics drawing tool over HTTP.
+
+About GrowthForecast, see:
+* Github: https://github.com/kazeburo/growthforecast
+* Product site (japanese): http://kazeburo.github.com/GrowthForecast/
+* Japanese blog post by @kazeburo: http://blog.nomadscafe.jp/2011/12/growthforecast.html
+
+GrowthForecast is very simple and powerful tool to draw graphs what we want, with GrowthForecastOutput and Fluentd.
+
+### Configuration
+
+For messages such as:
+    tag:metrics {"field1":300, "field2":20, "field3diff":-30}
+    
+Configuration example for graphs in growthforecast with POST api url 'http://growthforecast.local/api/service1/metrics1/metrics_FIELDNAME'.
+
+    <match metrics>
+      type growthforecast
+      gfapi_url http://growthforecast.local/api/
+      service   service1
+      section   metrics1
+      name_keys field1,field2,field3diff
+    </match>
+
+With this configuration, out_growthforecast posts urls below.
+
+    http://growthforecast.local/api/service1/metrics1/metrics_field1
+    http://growthforecast.local/api/service1/metrics1/metrics_field2
+    http://growthforecast.local/api/service1/metrics1/metrics_field3diff
+
+If you want to use tags for `section` or `service`  in GrowthForecast, use `tag_for` options and `remove_prefix` (and not to set the `section` or `service` that the value of 'tag_for' used to.).
+
+    <match metrics.**>
+      type growthforecast
+      gfapi_url http://growthforecast.local/api/
+      service   service1
+      name_keys field1,field2,field3diff
+      tag_for   section    # or 'name_prefix'(default) or 'ignore' or 'service'
+      remove_prefix metrics
+    </match>
+
+`mode` option available with `gauge`(default), `count`, `modified`, just same as `mode` of GrowthForecast POST parameter.
+
+`name_key_pattern REGEXP` available instead of `name_keys` like this:
+
+    <match metrics.**>
+      type growthforecast
+      gfapi_url http://growthforecast.local/api/
+      service   service1
+      tag_for   section    # or 'name_prefix'(default) or 'ignore' or 'service'
+      remove_prefix metrics
+      name_key_pattern ^(field|key)\d+$
+    </match>
+
+This configuration matches only with metrics.field1, metrics.key20, .... and doesn't match with metrics.field or metrics.foo.
+
+If your GrowthForecast protected with basic authentication, specify `authentication` option:
+
+    <match metrics.**>
+      type growthforecast
+      gfapi_url http://growthforecast.protected.anywhere.example.com/api/
+      service   yourservice
+      tag_for   section
+      name_keys fieldname
+      authentication basic
+      username yourusername
+      password secret!
+    </match>
+
+## TODO
+
+* patches welcome!
+
+## Copyright
+
+* Copyright (c) 2012- TAGOMORI Satoshi (tagomoris)
+* License
+  * Apache License, Version 2.0

data/Rakefile

@@ -1,36 +1,5 @@
-# encoding: utf-8
-
-require 'rubygems'
-require 'bundler'
-begin
-  Bundler.setup(:default, :development)
-rescue Bundler::BundlerError => e
-  $stderr.puts e.message
-  $stderr.puts "Run `bundle install` to install missing gems"
-  exit e.status_code
-end
-require 'rake'
-
-require 'jeweler'
-Jeweler::Tasks.new do |gem|
-  # gem is a Gem::Specification... see http://docs.rubygems.org/read/chapter/20 for more options
-  gem.name = "fluent-plugin-growthforecast"
-  gem.description = "Plugin to post numbers to GrowthForecast (by kazeburo)"
-  gem.homepage = "http://github.com/tagomoris/fluent-plugin-growthforecast"
-  gem.summary = gem.description
-  gem.email = "tagomoris@gmail.com"
-  gem.authors = ["TAGOMORI Satoshi"]
-  gem.has_rdoc = "false"
-  # dependencies defined in Gemfile
-  gem.files       = `git ls-files`.split("\n")
-  gem.test_files  = `git ls-files -- {test,spec,features}/*`.split("\n")
-  gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
-  gem.require_paths = ['lib']
-  gem.add_dependency "fluentd", "~> 0.10.8"
-  gem.add_development_dependency "rake", ">= 0.9.2"
-  gem.add_development_dependency "simplecov", ">= 0.5.4"
-end
-Jeweler::RubygemsDotOrgTasks.new
+#!/usr/bin/env rake
+require "bundler/gem_tasks"
 
 require 'rake/testtask'
 Rake::TestTask.new(:test) do |test|
@@ -40,13 +9,3 @@ Rake::TestTask.new(:test) do |test|
 end
 
 task :default => :test
-
-require 'rdoc/task'
-Rake::RDocTask.new do |rdoc|
-  version = File.exist?('VERSION') ? File.read('VERSION') : ""
-
-  rdoc.rdoc_dir = 'rdoc'
-  rdoc.title = "fluent-plugin-growthforecast #{version}"
-  rdoc.rdoc_files.include('README*')
-  rdoc.rdoc_files.include('lib/**/*.rb')
-end

data/fluent-plugin-growthforecast.gemspec

@@ -1,75 +1,22 @@
-# Generated by jeweler
-# DO NOT EDIT THIS FILE DIRECTLY
-# Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec'
 # -*- encoding: utf-8 -*-
 
-Gem::Specification.new do |s|
-  s.name = "fluent-plugin-growthforecast"
-  s.version = "0.1.4"
+Gem::Specification.new do |gem|
+  gem.name          = "fluent-plugin-growthforecast"
+  gem.version       = "0.1.5"
+  gem.authors       = ["TAGOMORI Satoshi"]
+  gem.email         = ["tagomoris@gmail.com"]
+  gem.summary       = %q{Fluentd output plugin to post numbers to GrowthForecast (by kazeburo)}
+  gem.description   = %q{For GrowthForecast, see http://kazeburo.github.com/GrowthForecast/}
+  gem.homepage      = "https://github.com/tagomoris/fluent-plugin-growthforecast"
 
-  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
-  s.authors = ["TAGOMORI Satoshi"]
-  s.date = "2012-12-05"
-  s.description = "Plugin to post numbers to GrowthForecast (by kazeburo)"
-  s.email = "tagomoris@gmail.com"
-  s.extra_rdoc_files = [
-    "LICENSE.txt",
-    "README.rdoc"
-  ]
-  s.files = [
-    ".document",
-    ".gitignore",
-    "AUTHORS",
-    "Gemfile",
-    "LICENSE.txt",
-    "README.rdoc",
-    "Rakefile",
-    "VERSION",
-    "fluent-plugin-growthforecast.gemspec",
-    "lib/fluent/plugin/out_growthforecast.rb",
-    "test/helper.rb",
-    "test/plugin/test_out_growthforecast.rb"
-  ]
-  s.homepage = "http://github.com/tagomoris/fluent-plugin-growthforecast"
-  s.require_paths = ["lib"]
-  s.rubygems_version = "1.8.21"
-  s.summary = "Plugin to post numbers to GrowthForecast (by kazeburo)"
-  s.test_files = ["test/helper.rb", "test/plugin/test_out_growthforecast.rb"]
+  gem.files         = `git ls-files`.split($\)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
 
-  if s.respond_to? :specification_version then
-    s.specification_version = 3
-
-    if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
-      s.add_runtime_dependency(%q<fluentd>, [">= 0"])
-      s.add_runtime_dependency(%q<rdoc>, [">= 0"])
-      s.add_development_dependency(%q<shoulda>, [">= 0"])
-      s.add_development_dependency(%q<bundler>, ["~> 1.0.0"])
-      s.add_development_dependency(%q<jeweler>, ["~> 1.6.4"])
-      s.add_development_dependency(%q<simplecov>, [">= 0"])
-      s.add_runtime_dependency(%q<fluentd>, ["~> 0.10.8"])
-      s.add_development_dependency(%q<rake>, [">= 0.9.2"])
-      s.add_development_dependency(%q<simplecov>, [">= 0.5.4"])
-    else
-      s.add_dependency(%q<fluentd>, [">= 0"])
-      s.add_dependency(%q<rdoc>, [">= 0"])
-      s.add_dependency(%q<shoulda>, [">= 0"])
-      s.add_dependency(%q<bundler>, ["~> 1.0.0"])
-      s.add_dependency(%q<jeweler>, ["~> 1.6.4"])
-      s.add_dependency(%q<simplecov>, [">= 0"])
-      s.add_dependency(%q<fluentd>, ["~> 0.10.8"])
-      s.add_dependency(%q<rake>, [">= 0.9.2"])
-      s.add_dependency(%q<simplecov>, [">= 0.5.4"])
-    end
-  else
-    s.add_dependency(%q<fluentd>, [">= 0"])
-    s.add_dependency(%q<rdoc>, [">= 0"])
-    s.add_dependency(%q<shoulda>, [">= 0"])
-    s.add_dependency(%q<bundler>, ["~> 1.0.0"])
-    s.add_dependency(%q<jeweler>, ["~> 1.6.4"])
-    s.add_dependency(%q<simplecov>, [">= 0"])
-    s.add_dependency(%q<fluentd>, ["~> 0.10.8"])
-    s.add_dependency(%q<rake>, [">= 0.9.2"])
-    s.add_dependency(%q<simplecov>, [">= 0.5.4"])
-  end
+  gem.add_development_dependency "bundler"
+  gem.add_development_dependency "fluentd"
+  gem.add_development_dependency "fluent-mixin-config-placeholders"
+  gem.add_runtime_dependency "fluentd"
+  gem.add_runtime_dependency "fluent-mixin-config-placeholders"
 end
-

data/lib/fluent/plugin/out_growthforecast.rb

@@ -8,24 +8,30 @@ class Fluent::GrowthForecastOutput < Fluent::Output
   end
 
   config_param :gfapi_url, :string # growth.forecast.local/api/
-  config_param :service, :string
+  config_param :service, :string, :default => nil
   config_param :section, :string, :default => nil
 
+  config_param :ssl, :bool, :default => false
+  config_param :verify_ssl, :bool, :default => false
+
   config_param :name_keys, :string, :default => nil
   config_param :name_key_pattern, :string, :default => nil
 
   config_param :mode, :string, :default => 'gauge' # or count/modified
 
   config_param :remove_prefix, :string, :default => nil
-  config_param :tag_for, :string, :default => 'name_prefix' # or 'ignore' or 'section'
+  config_param :tag_for, :string, :default => 'name_prefix' # or 'ignore' or 'section' or 'service'
   
+  config_param :authentication, :string, :default => nil # nil or 'none' or 'basic'
+  config_param :username, :string, :default => ''
+  config_param :password, :string, :default => ''
+
   def configure(conf)
     super
 
     if @gfapi_url !~ /\/api\/\Z/
       raise Fluent::ConfigError, "gfapi_url must end with /api/"
     end
-    @gfurl = @gfapi_url + @service + '/'
 
     if @name_keys.nil? and @name_key_pattern.nil?
       raise Fluent::ConfigError, "missing both of name_keys and name_key_pattern"
@@ -49,17 +55,27 @@ class Fluent::GrowthForecastOutput < Fluent::Output
     @tag_for = case @tag_for
                when 'ignore' then :ignore
                when 'section' then :section
+               when 'service' then :service
                else
                  :name_prefix
                end
     if @tag_for != :section and @section.nil?
       raise Fluent::ConfigError, "section parameter is needed when tag_for is not 'section'"
     end
+    if @tag_for != :service and @service.nil?
+      raise Fluent::ConfigError, "service parameter is needed when tag_for is not 'service'"
+    end
 
     if @remove_prefix
       @removed_prefix_string = @remove_prefix + '.'
       @removed_length = @removed_prefix_string.length
     end
+
+    @auth = case @authentication
+            when 'basic' then :basic
+            else
+              :none
+            end
   end
 
   def start
@@ -71,25 +87,44 @@ class Fluent::GrowthForecastOutput < Fluent::Output
   end
 
   def format_url(tag, name)
-    name_esc = URI.escape(name)
+    if @remove_prefix and
+        ( (tag.start_with?(@removed_prefix_string) and tag.length > @removed_length) or tag == @remove_prefix)
+      tag = tag[@removed_length..-1]
+    end
+
     case @tag_for
     when :ignore
-      @gfurl + @section + '/' + name_esc
+      @gfapi_url + URI.escape(@service + '/' + @section + '/' + name)
     when :section
-      @gfurl + tag + '/' + name_esc
+      @gfapi_url + URI.escape(@service + '/' + tag + '/' + name)
+    when :service
+      @gfapi_url + URI.escape(tag + '/' + @section + '/' + name)
     when :name_prefix
-      @gfurl + @section + '/' + tag + '_' + name_esc
+      @gfapi_url + URI.escape(@service + '/' + @section + '/' + tag + '_' + name)
     end
   end
 
   def post(tag, name, value)
     url = format_url(tag,name)
+    res = nil
     begin
-      res = Net::HTTP.post_form(URI.parse(url), {'number' => value.to_i, 'mode' => @mode.to_s})
+      url = URI.parse(url)
+      req = Net::HTTP::Post.new(url.path)
+      if @auth and @auth == :basic
+        req.basic_auth(@username, @password)
+      end
+      req.set_form_data({'number' => value.to_i, 'mode' => @mode.to_s})
+      http = Net::HTTP.new(url.host, url.port)
+      if @ssl
+        http.use_ssl = true
+        unless @verify_ssl
+          http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+        end
+      end
+      res = http.start {|http| http.request(req) }
     rescue IOError, EOFError, SystemCallError
       # server didn't respond
       $log.warn "Net::HTTP.post_form raises exception: #{$!.class}, '#{$!.message}'"
-      res = nil
     end
     unless res and res.is_a?(Net::HTTPSuccess)
       $log.warn "failed to post to growthforecast: #{url}, number: #{value}, code: #{res && res.code}"
@@ -97,10 +132,6 @@ class Fluent::GrowthForecastOutput < Fluent::Output
   end
 
   def emit(tag, es, chain)
-    if @remove_prefix and
-        ( (tag.start_with?(@removed_prefix_string) and tag.length > @removed_length) or tag == @remove_prefix)
-      tag = tag[@removed_length..-1]
-    end
     if @name_keys
       es.each {|time,record|
         @name_keys.each {|name|

data/test/helper.rb

@@ -8,12 +8,45 @@ rescue Bundler::BundlerError => e
   exit e.status_code
 end
 require 'test/unit'
-require 'shoulda'
 
 $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
 $LOAD_PATH.unshift(File.dirname(__FILE__))
 require 'fluent/test'
+unless ENV.has_key?('VERBOSE')
+  nulllogger = Object.new
+  nulllogger.instance_eval {|obj|
+    def method_missing(method, *args)
+      # pass
+    end
+  }
+  $log = nulllogger
+end
+
 require 'fluent/plugin/out_growthforecast'
 
 class Test::Unit::TestCase
 end
+
+require 'webrick'
+
+# to handle POST/PUT/DELETE ...
+module WEBrick::HTTPServlet
+  class ProcHandler < AbstractServlet
+    alias do_POST   do_GET
+    alias do_PUT    do_GET
+    alias do_DELETE do_GET
+  end
+end
+
+def get_code(server, port, path, headers={})
+  require 'net/http' 
+  Net::HTTP.start(server, port){|http|
+    http.get(path, headers).code
+  } 
+end 
+def get_content(server, port, path, headers={})
+  require 'net/http'
+  Net::HTTP.start(server, port){|http|
+    http.get(path, headers).body
+  } 
+end

data/test/plugin/test_out_growthforecast.rb

@@ -1,7 +1,421 @@
 require 'helper'
 
-class TestFluentPluginGrowthforecast < Test::Unit::TestCase
-  should "probably rename this file and start testing for real" do
-    flunk "hey buddy, you should probably rename this file and start testing for real"
+class GrowthForecastOutputTest < Test::Unit::TestCase
+  # setup/teardown and tests of dummy growthforecast server defined at the end of this class...
+  GF_TEST_LISTEN_PORT = 5125
+
+  CONFIG1 = %[
+      gfapi_url http://127.0.0.1:5125/api/
+      service   service
+      section   metrics
+      name_keys field1,field2,otherfield
+      tag_for   name_prefix
+  ]
+
+  CONFIG2 = %[
+      gfapi_url http://127.0.0.1:5125/api/
+      service   service
+      section   metrics
+      tag_for   ignore
+      name_keys field1,field2,otherfield
+      mode count
+  ]
+
+  CONFIG3 = %[
+      gfapi_url http://127.0.0.1:5125/api/
+      service   service
+      tag_for   section
+      remove_prefix test
+      name_key_pattern ^(field|key)\\d+$
+      mode modified
+  ]
+
+  CONFIG4 = %[
+      gfapi_url http://127.0.0.1:5125/api/
+      section   metrics
+      name_keys field1,field2,otherfield
+      tag_for   service
+      remove_prefix test
+  ]
+  
+  CONFIG_SPACE = %[
+      gfapi_url http://127.0.0.1:5125/api/
+      service   service x
+      section   metrics y
+      name_keys field z
+      tag_for   ignore
+  ]
+
+  def create_driver(conf=CONFIG1, tag='test.metrics')
+    Fluent::Test::OutputTestDriver.new(Fluent::GrowthForecastOutput, tag).configure(conf)
+  end
+
+  def test_configure_and_format_url
+    d = create_driver
+    assert_equal 'http://127.0.0.1:5125/api/', d.instance.gfapi_url
+    assert_equal 'service', d.instance.service
+    assert_equal 'metrics', d.instance.section
+    assert_equal ['field1', 'field2', 'otherfield'], d.instance.name_keys
+    assert_nil d.instance.remove_prefix
+    assert_equal :name_prefix, d.instance.tag_for
+    assert_equal :gauge, d.instance.mode
+
+    assert_equal 'http://127.0.0.1:5125/api/service/metrics/test.data1_field1', d.instance.format_url('test.data1', 'field1')
+
+    d = create_driver(CONFIG2)
+    assert_equal 'http://127.0.0.1:5125/api/', d.instance.gfapi_url
+    assert_equal 'service', d.instance.service
+    assert_equal 'metrics', d.instance.section
+    assert_equal ['field1', 'field2', 'otherfield'], d.instance.name_keys
+    assert_nil d.instance.remove_prefix
+    assert_equal :ignore, d.instance.tag_for
+    assert_equal :count, d.instance.mode
+
+    assert_equal 'http://127.0.0.1:5125/api/service/metrics/field1', d.instance.format_url('test.data1', 'field1')
+
+    d = create_driver(CONFIG3)
+    assert_equal 'http://127.0.0.1:5125/api/', d.instance.gfapi_url
+    assert_equal 'service', d.instance.service
+    assert_nil d.instance.section
+    assert_equal Regexp.new('^(field|key)\d+$'), d.instance.name_key_pattern
+    assert_equal 'test', d.instance.remove_prefix
+    assert_equal :section, d.instance.tag_for
+    assert_equal 'test.', d.instance.instance_eval{ @removed_prefix_string }
+    assert_equal :modified, d.instance.mode
+
+    assert_equal 'http://127.0.0.1:5125/api/service/data1/field1', d.instance.format_url('test.data1', 'field1')
+  end
+
+  # CONFIG1 = %[
+  #     gfapi_url http://127.0.0.1:5125/api/
+  #     service   service
+  #     section   metrics
+  #     name_keys field1,field2,otherfield
+  #     tag_for   name_prefix
+  # ]
+  def test_emit_1
+    d = create_driver(CONFIG1, 'test.metrics')
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run
+
+    assert_equal 3, @posted.size
+    v1st = @posted[0]
+    v2nd = @posted[1]
+    v3rd = @posted[2]
+
+    assert_equal 50, v1st[:data][:number]
+    assert_equal 'gauge', v1st[:data][:mode]
+    assert_nil v1st[:auth]
+    assert_equal 'service', v1st[:service]
+    assert_equal 'metrics', v1st[:section]
+    assert_equal 'test.metrics_field1', v1st[:name]
+
+    assert_equal 20, v2nd[:data][:number]
+    assert_equal 'test.metrics_field2', v2nd[:name]
+
+    assert_equal 1, v3rd[:data][:number]
+    assert_equal 'test.metrics_otherfield', v3rd[:name]
+  end
+
+  # CONFIG2 = %[
+  #     gfapi_url http://127.0.0.1:5125/api/
+  #     service   service
+  #     section   metrics
+  #     tag_for   ignore
+  #     name_keys field1,field2,otherfield
+  #     mode count
+  # ]
+  def test_emit_2
+    d = create_driver(CONFIG2, 'test.metrics')
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run
+
+    assert_equal 3, @posted.size
+    v1st = @posted[0]
+    v2nd = @posted[1]
+    v3rd = @posted[2]
+
+    assert_equal 50, v1st[:data][:number]
+    assert_equal 'count', v1st[:data][:mode]
+    assert_nil v1st[:auth]
+    assert_equal 'service', v1st[:service]
+    assert_equal 'metrics', v1st[:section]
+    assert_equal 'field1', v1st[:name]
+
+    assert_equal 20, v2nd[:data][:number]
+    assert_equal 'field2', v2nd[:name]
+
+    assert_equal 1, v3rd[:data][:number]
+    assert_equal 'otherfield', v3rd[:name]
+  end
+
+  # CONFIG3 = %[
+  #     gfapi_url http://127.0.0.1:5125/api/
+  #     service   service
+  #     tag_for   section
+  #     remove_prefix test
+  #     name_key_pattern ^(field|key)\\d+$
+  #     mode modified
+  # ]
+  def test_emit_3
+    d = create_driver(CONFIG3, 'test.metrics')
+    # recent ruby's Hash saves elements order....
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run
+
+    assert_equal 3, @posted.size
+    v1st = @posted[0]
+    v2nd = @posted[1]
+    v3rd = @posted[2]
+
+    assert_equal 50, v1st[:data][:number]
+    assert_equal 'modified', v1st[:data][:mode]
+    assert_nil v1st[:auth]
+    assert_equal 'service', v1st[:service]
+    assert_equal 'metrics', v1st[:section]
+    assert_equal 'field1', v1st[:name]
+
+    assert_equal 20, v2nd[:data][:number]
+    assert_equal 'field2', v2nd[:name]
+
+    assert_equal 10, v3rd[:data][:number]
+    assert_equal 'field3', v3rd[:name]
+  end
+
+  # CONFIG1 = %[
+  #     gfapi_url http://127.0.0.1:5125/api/
+  #     service   service
+  #     section   metrics
+  #     name_keys field1,field2,otherfield
+  #     tag_for   name_prefix
+  # ]
+  def test_emit_4_auth
+    @auth = true # enable authentication of dummy server
+
+    d = create_driver(CONFIG1, 'test.metrics')
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run # failed in background, and output warn log
+
+    assert_equal 0, @posted.size
+    assert_equal 3, @prohibited
+
+    d = create_driver(CONFIG1 + %[
+      authentication basic
+      username alice
+      password wrong_password
+    ], 'test.metrics')
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run # failed in background, and output warn log
+
+    assert_equal 0, @posted.size
+    assert_equal 6, @prohibited
+
+    d = create_driver(CONFIG1 + %[
+      authentication basic
+      username alice
+      password secret!
+    ], 'test.metrics')
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run # failed in background, and output warn log
+
+    assert_equal 6, @prohibited
+    assert_equal 3, @posted.size
   end
+
+  # CONFIG4 = %[
+  #     gfapi_url http://127.0.0.1:5125/api/
+  #     section   metrics
+  #     name_keys field1,field2,otherfield
+  #     tag_for   service
+  #     remove_prefix test
+  # ]
+
+  def test_emit_5
+    d = create_driver(CONFIG4, 'test.service')
+    # recent ruby's Hash saves elements order....
+    d.emit({ 'field1' => 50, 'field2' => 20, 'field3' => 10, 'otherfield' => 1 })
+    d.run
+
+    assert_equal 3, @posted.size
+    v1st = @posted[0]
+    v2nd = @posted[1]
+    v3rd = @posted[2]
+
+    assert_equal 50, v1st[:data][:number]
+    assert_equal 'gauge', v1st[:data][:mode]
+    assert_nil v1st[:auth]
+    assert_equal 'service', v1st[:service]
+    assert_equal 'metrics', v1st[:section]
+    assert_equal 'field1', v1st[:name]
+
+    assert_equal 20, v2nd[:data][:number]
+    assert_equal 'field2', v2nd[:name]
+
+    assert_equal 1, v3rd[:data][:number]
+    assert_equal 'otherfield', v3rd[:name]
+  end
+
+  # CONFIG_SPACE = %[
+  #     gfapi_url http://127.0.0.1:5125/api/
+  #     service   service x
+  #     section   metrics y
+  #     name_keys field z
+  #     tag_for   ignore
+  # ]
+  def test_with_space_1
+    d = create_driver(CONFIG_SPACE, 'test.foo')
+    d.emit({ 'field z' => 3 })
+    d.run
+
+    assert_equal 1, @posted.size
+    v = @posted[0]
+
+    assert_equal 3, v[:data][:number]
+    assert_equal 'gauge', v[:data][:mode]
+    assert_nil v[:auth]
+    assert_equal 'service x', v[:service]
+    assert_equal 'metrics y', v[:section]
+    assert_equal 'field z', v[:name]
+  end
+
+  # setup / teardown for servers
+  def setup
+    Fluent::Test.setup
+    @posted = []
+    @prohibited = 0
+    @auth = false
+    @dummy_server_thread = Thread.new do
+      srv = if ENV['VERBOSE']
+              WEBrick::HTTPServer.new({:BindAddress => '127.0.0.1', :Port => GF_TEST_LISTEN_PORT})
+            else
+              logger = WEBrick::Log.new('/dev/null', WEBrick::BasicLog::DEBUG)
+              WEBrick::HTTPServer.new({:BindAddress => '127.0.0.1', :Port => GF_TEST_LISTEN_PORT, :Logger => logger, :AccessLog => []})
+            end
+      begin
+        srv.mount_proc('/api') { |req,res| # /api/:service/:section/:name
+          unless req.request_method == 'POST'
+            res.status = 405
+            res.body = 'request method mismatch'
+            next
+          end
+          if @auth and req.header['authorization'][0] == 'Basic YWxpY2U6c2VjcmV0IQ==' # pattern of user='alice' passwd='secret!'
+            # ok, authorized
+          elsif @auth
+            res.status = 403
+            @prohibited += 1
+            next
+          else
+            # ok, authorization not required
+          end
+
+          req.path =~ /^\/api\/([^\/]*)\/([^\/]*)\/(.*)$/
+          service = $1
+          section = $2
+          graph_name = $3
+          post_param = Hash[*(req.body.split('&').map{|kv|kv.split('=')}.flatten)]
+
+          @posted.push({
+              :service => service,
+              :section => section,
+              :name => graph_name,
+              :auth => nil,
+              :data => { :number => post_param['number'].to_i, :mode => post_param['mode'] },
+            })
+
+          res.status = 200
+        }
+        srv.mount_proc('/') { |req,res|
+          res.status = 200
+          res.body = 'running'
+        }
+        srv.start
+      ensure
+        srv.shutdown
+      end
+    end
+
+    # to wait completion of dummy server.start()
+    require 'thread'
+    cv = ConditionVariable.new
+    watcher = Thread.new {
+      connected = false
+      while not connected
+        begin
+          get_content('localhost', GF_TEST_LISTEN_PORT, '/')
+          connected = true
+        rescue Errno::ECONNREFUSED
+          sleep 0.1
+        rescue StandardError => e
+          p e
+          sleep 0.1
+        end
+      end
+      cv.signal
+    } 
+    mutex = Mutex.new
+    mutex.synchronize {
+      cv.wait(mutex)
+    } 
+  end
+
+  def test_dummy_server
+    d = create_driver
+    d.instance.gfapi_url =~ /^http:\/\/([.:a-z0-9]+)\//
+    server = $1
+    host = server.split(':')[0]
+    port = server.split(':')[1].to_i
+    client = Net::HTTP.start(host, port)
+
+    assert_equal '200', client.request_get('/').code
+    assert_equal '200', client.request_post('/api/service/metrics/hoge', 'number=1&mode=gauge').code
+
+    assert_equal 1, @posted.size
+
+    assert_equal 1, @posted[0][:data][:number]
+    assert_equal 'gauge', @posted[0][:data][:mode]
+    assert_nil @posted[0][:auth]
+    assert_equal 'service', @posted[0][:service]
+    assert_equal 'metrics', @posted[0][:section]
+    assert_equal 'hoge', @posted[0][:name]
+
+    assert_equal '200', client.request_post(URI.escape('/api/service x/metrics/hoge'), 'number=1&mode=gauge').code
+
+    assert_equal 2, @posted.size
+
+    assert_equal 1, @posted[1][:data][:number]
+    assert_equal 'gauge', @posted[1][:data][:mode]
+    assert_nil @posted[1][:auth]
+    assert_equal 'service x', @posted[1][:service]
+    assert_equal 'metrics', @posted[1][:section]
+    assert_equal 'hoge', @posted[1][:name]
+
+    @auth = true
+
+    assert_equal '403', client.request_post('/api/service/metrics/pos', 'number=30&mode=gauge').code
+
+    req_with_auth = lambda do |number, mode, user, pass|
+      url = URI.parse("http://#{host}:#{port}/api/service/metrics/pos")
+      req = Net::HTTP::Post.new(url.path)
+      req.basic_auth user, pass
+      req.set_form_data({'number'=>number, 'mode'=>mode})
+      req
+    end
+
+    assert_equal '403', client.request(req_with_auth.call(500, 'count', 'alice', 'wrong password!')).code
+
+    assert_equal '403', client.request(req_with_auth.call(500, 'count', 'alice', 'wrong password!')).code
+
+    assert_equal 2, @posted.size
+
+    assert_equal '200', client.request(req_with_auth.call(500, 'count', 'alice', 'secret!')).code
+
+    assert_equal 3, @posted.size
+
+  end
+
+  def teardown
+    @dummy_server_thread.kill
+    @dummy_server_thread.join
+  end
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-growthforecast
 version: !ruby/object:Gem::Version
-  version: 0.1.4
+  version: 0.1.5
   prerelease: 
 platform: ruby
 authors:
@@ -9,33 +9,17 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-12-05 00:00:00.000000000 Z
+date: 2013-03-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: fluentd
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: rdoc
+  name: bundler
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
         version: '0'
-  type: :runtime
+  type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     none: false
@@ -44,7 +28,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: shoulda
+  name: fluentd
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
@@ -60,39 +44,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 1.0.0
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 1.0.0
-- !ruby/object:Gem::Dependency
-  name: jeweler
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 1.6.4
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 1.6.4
-- !ruby/object:Gem::Dependency
-  name: simplecov
+  name: fluent-mixin-config-placeholders
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
@@ -109,73 +61,56 @@ dependencies:
         version: '0'
 - !ruby/object:Gem::Dependency
   name: fluentd
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 0.10.8
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 0.10.8
-- !ruby/object:Gem::Dependency
-  name: rake
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
-        version: 0.9.2
-  type: :development
+        version: '0'
+  type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
-        version: 0.9.2
+        version: '0'
 - !ruby/object:Gem::Dependency
-  name: simplecov
+  name: fluent-mixin-config-placeholders
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
-        version: 0.5.4
-  type: :development
+        version: '0'
+  type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
       - !ruby/object:Gem::Version
-        version: 0.5.4
-description: Plugin to post numbers to GrowthForecast (by kazeburo)
-email: tagomoris@gmail.com
+        version: '0'
+description: For GrowthForecast, see http://kazeburo.github.com/GrowthForecast/
+email:
+- tagomoris@gmail.com
 executables: []
 extensions: []
-extra_rdoc_files:
-- LICENSE.txt
-- README.rdoc
+extra_rdoc_files: []
 files:
 - .document
 - .gitignore
 - AUTHORS
 - Gemfile
 - LICENSE.txt
-- README.rdoc
+- README.md
 - Rakefile
 - VERSION
 - fluent-plugin-growthforecast.gemspec
 - lib/fluent/plugin/out_growthforecast.rb
 - test/helper.rb
 - test/plugin/test_out_growthforecast.rb
-homepage: http://github.com/tagomoris/fluent-plugin-growthforecast
+homepage: https://github.com/tagomoris/fluent-plugin-growthforecast
 licenses: []
 post_install_message: 
 rdoc_options: []
@@ -187,9 +122,6 @@ required_ruby_version: !ruby/object:Gem::Requirement
   - - ! '>='
     - !ruby/object:Gem::Version
       version: '0'
-      segments:
-      - 0
-      hash: -1202506327029138940
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -201,7 +133,7 @@ rubyforge_project:
 rubygems_version: 1.8.21
 signing_key: 
 specification_version: 3
-summary: Plugin to post numbers to GrowthForecast (by kazeburo)
+summary: Fluentd output plugin to post numbers to GrowthForecast (by kazeburo)
 test_files:
 - test/helper.rb
 - test/plugin/test_out_growthforecast.rb

data/README.rdoc

@@ -1,59 +0,0 @@
-= fluent-plugin-growthforecast
-
-== Component
-
-=== GrowthForecastOutput
-
-Plugin to output numbers(metrics) to 'GrowthForecast', metrics drawing tool over HTTP.
-
-About GrowthForecast, see:
-- Github: https://github.com/kazeburo/growthforecast
-- Japanese blog post by @kazeburo: http://blog.nomadscafe.jp/2011/12/growthforecast.html
-
-GrowthForecast is very simple but useful tool to draw graphs what we want, with GrowthForecastOutput and Fluentd.
-
-== Configuration
-
-=== GrowthForecastOutput
-
-For messages such as:
-    tag:metrics {"field1":300, "field2":20, "field3diff":-30}
-    
-Configuration example for graphs in growthforecast with POST api url 'http://growthforecast.local/api/service1/metrics1/metrics_FIELDNAME'.
-
-    <match metrics>
-      type growthforecast
-      gfapi_url http://growthforecast.local/api/
-      service   service1
-      section   metrics1
-      name_keys field1,field2,field3diff
-    </match>
-
-With this configuration, out_growthforecast posts urls below.
-- http://growthforecast.local/api/service1/metrics1/metrics_field1
-- http://growthforecast.local/api/service1/metrics1/metrics_field2
-- http://growthforecast.local/api/service1/metrics1/metrics_field3diff
-
-
-If you want to use tags for 'section' in GrowthForecast, use 'tag_for' options and remove_prefix (and not to set 'section').
-
-    <match metrics.**>
-      type growthforecast
-      gfapi_url http://growthforecast.local/api/
-      service   service1
-      name_keys field1,field2,field3diff
-      tag_for   section    # or 'name_prefix'(default) or 'ignore'
-      remove_prefix metrics
-    </match>
-
-'mode' option available with 'gauge'(default), 'count', 'modified', just same as 'mode' of GrowthForecast POST parameter.
-
-== TODO
-
-- consider what to do next
-- patches welcome!
-
-== Copyright
-
-Copyright:: Copyright (c) 2012- TAGOMORI Satoshi (tagomoris)
-License::   Apache License, Version 2.0