fluentd 1.4.0 → 1.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: ca1f6da9527b76b4421c1507ec0e2702687e196d
-  data.tar.gz: 8ae3c1fbac074dbc699bb36cb0975c78e94e3769
+  metadata.gz: b5edf62d55c4f1fbdadda5a367479cdf5b875f3b
+  data.tar.gz: e931a66ae995f73c3309ab9fcf29d2a8e36ef0c8
 SHA512:
-  metadata.gz: 4a2298e9e76277c4ce41c7b43e239643a0b3ad8a953ab47bc242a9e22645e0d517853f73f8e37699ad5bbc57dc36e9d5cfb0504da155fb575c36eebf1893cfb7
-  data.tar.gz: 783fb04e180adc4711a6ebf83b91d6fa2d318300ca31ba0c49c8d247c2eceed4992b22be98765f7b30e316cd7815cf9aac7e81873b221c480f4df80f51a3fccb
+  metadata.gz: 6c1e49998c72cb8a4d491e355095804167a86dd9a26b391d1d6188c51976ae30c77d28b43c0433e57b83e8a1c4503bc248fce0cbe4b98d079ba795414f6c33ca
+  data.tar.gz: ec39e3c4c5dafab999a61799a003ab170bf08df076dc1a98ad5c32910e3dc6e4a6aef5d83182b44e7f3cc2621b06a6e3e8333179a62ae466aa5b940ff8ded2d0

data/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,13 @@
+<!--
+Thank you for contributing to Fluentd!
+Please provide the following information to help us make the most of your pull request:
+-->
+
+**Which issue(s) this PR fixes**: 
+Fixes #
+
+**What this PR does / why we need it**: 
+
+**Docs Changes**:
+
+**Release Note**: 

data/.gitignore

@@ -25,4 +25,5 @@ Gemfile.local
 coverage/*
 .vagrant/
 cov-int/
-cov-fluentd.tar.gz
+cov-fluentd.tar.gz
+.vscode

data/CHANGELOG.md

@@ -1,4 +1,20 @@
-# v1.3
+# v1.4
+
+## Release v1.4.1 - 2019/03/18
+
+### Enhancements
+
+* system: Add worker_id to process_name when workers is larger than 1
+  https://github.com/fluent/fluentd/pull/2321
+* in_syslog: Check message length when read from buffer in octet counting
+  https://github.com/fluent/fluentd/pull/2323
+* parser_regexp: Check named captures. When no named captures, configuration error is raised
+  https://github.com/fluent/fluentd/pull/2331
+
+### Bug fixes
+
+* out_forward: Make tls_client_private_key_passphrase secret
+  https://github.com/fluent/fluentd/pull/2324
 
 ## Release v1.4.0 - 2019/02/24
 
@@ -15,10 +31,10 @@
 
 * output: Add MessagePack unpacker error to unrecoverable error list
   https://github.com/fluent/fluentd/pull/2301
-* output: Reduce flush deley when large timekey and small timekey_wait are specified
+* output: Reduce flush delay when large timekey and small timekey_wait are specified
   https://github.com/fluent/fluentd/pull/2291
 * config: Support embedded ruby code in section argument.
-  https://github.com/fluent/fluentd/pull/2291
+  https://github.com/fluent/fluentd/pull/2295
 * in_tail: Improve encoding parameter handling
   https://github.com/fluent/fluentd/pull/2305
 * in_tcp/in_udp: Add <parse> section check
@@ -31,6 +47,8 @@
 * server: Ignore EPIPE in TLS accept to avoid fluentd restart
   https://github.com/fluent/fluentd/pull/2253
 
+# v1.3
+
 ## Release v1.3.3 - 2019/01/06
 
 ### Enhancements
@@ -673,12 +691,12 @@ See [CNCF announcment](https://www.cncf.io/blog/2017/12/06/fluentd-v1-0/) :)
 
 * output: Secondary calculation should consider 'retry_max_times'
   https://github.com/fluent/fluentd/pull/1452
-* Fix regression of deprecatd 'process' module
+* Fix regression of deprecated 'process' module
   https://github.com/fluent/fluentd/pull/1443
 * Fix missing parser_regex require
   https://github.com/fluent/fluentd/issues/1458
   https://github.com/fluent/fluentd/pull/1453
-* Keep 'Fluent::BufferQueueLimitError' for exsting plugins
+* Keep 'Fluent::BufferQueueLimitError' for existing plugins
   https://github.com/fluent/fluentd/pull/1456
 * in_tail: Untracked files should be removed from watching list to avoid memory bloat
   https://github.com/fluent/fluentd/pull/1467
@@ -930,7 +948,7 @@ See [CNCF announcment](https://www.cncf.io/blog/2017/12/06/fluentd-v1-0/) :)
 
 ### Bug fixes
 
-* Raise configuration errors to clearify what's wrong when "@type" is missing
+* Raise configuration errors to clarify what's wrong when "@type" is missing
   https://github.com/fluent/fluentd/pull/1202
 * Fix the bug not to launch Fluentd when v0.12 MultiOutput plugin is configured
   https://github.com/fluent/fluentd/pull/1206

data/GithubWorkflow.md

@@ -0,0 +1,78 @@
+# Github workflow for contributing to fluentd
+
+Table of Contents
+
+* [Fork a repository](#fork-a-repository)
+* [Clone fork repository to local](#clone-fork-repository-to-local)
+* [Create a branch to add a new feature or fix issues](#create-a-branch-to-add-a-new-feature-or-fix-issues)
+* [Commit and Push](#commit-and-push)
+* [Create a Pull Request](#create-a-pull-request)
+
+
+The [fluentd](https://github.com/fluent/fluentd.git) code is hosted on Github (https://github.com/fluent/fluentd). The repository is called `upstream`. Contributors will develop and commit their changes in a clone of upstream repository. Then contributors push their change to their forked repository (`origin`) and create a Pull Request (PR), the PR will be merged to `upstream` repository if it meets the all the necessary requirements.		
+
+## Fork a repository
+
+ Go to https://github.com/fluent/fluentd then hit the `Fork` button to fork your own copy of repository **fluentd** to your github account.
+
+## Clone the forked repository to local
+
+Clone the forked repo in [above step](#fork-a-repository) to your local working directory:
+```sh
+$ git clone https://github.com/$your_github_account/fluentd.git   
+```
+
+Keep your fork in sync with the main repo, add an `upstream` remote:
+```sh
+$ cd fluentd
+$ git remote add upstream https://github.com/fluentd/fluentd.git
+$ git remote -v
+
+origin  https://github.com/$your_github_account/fluentd.git (fetch)
+origin  https://github.com/$your_github_account/fluentd.git (push)
+upstream        https://github.com/fluentd/fluentd.git (fetch)
+upstream        https://github.com/fluentd/fluentd.git (push)
+```
+
+Sync your local `master` branch:
+```sh
+$ git checkout master
+$ git pull origin master
+$ git fetch upstream
+$ git rebase upstream/master
+```
+
+## Create a branch to add a new feature or fix issues
+
+Before making any change, create a new branch:
+```sh
+$ git checkout master
+$ git pull upstream master
+$ git checkout -b new-feature
+```
+
+## Commit and Push
+
+Make any change on the branch `new-feature`  then build and test your codes.  
+Include in what will be committed:
+```sh
+$ git add <file>
+```
+
+Commit your changes with `sign-offs`
+```sh
+$ git commit -s
+```
+
+Enter your commit message to describe the changes. See the tips for a good commit message at [here](https://chris.beams.io/posts/git-commit/).  
+Likely you go back and edit/build/test some more then `git commit --amend`  
+
+Push your branch `new-feature` to your forked repository:
+```sh
+$ git push -u origin new-feature
+```
+
+## Create a Pull Request
+
+* Go to your fork at https://github.com/$your_github_account/fluentd
+* Create a Pull Request from the branch you recently pushed by hitting the button `Compare & pull request` next to branch name.

data/README.md

@@ -71,7 +71,7 @@ Many enterprises run Fluentd in production to handle all of their logging needs.
 - Project repository: https://github.com/fluent
 - Discussion: https://groups.google.com/group/fluentd
 - Slack / Community: https://slack.fluentd.org
-- Newsletters: https://www.fluentd.org/newsletter_signup
+- Newsletters: https://www.fluentd.org/newsletter
 - Author: [Sadayuki Furuhashi](https://github.com/frsyuki)
 - Copyright: 2011-2018 Fluentd Authors
 - License: Apache License, Version 2.0

data/lib/fluent/command/plugin_config_formatter.rb

@@ -32,7 +32,8 @@ class FluentPluginConfigFormatter
     "buffer", "parser", "formatter", "storage"
   ]
 
-  DOCS_BASE_URL = "https://docs.fluentd.org/v1.0/articles/"
+  DOCS_BASE_URL = "https://docs.fluentd.org/v1.0/articles/quickstart"
+  DOCS_ARTICLE_BASE_URL = "https://docs.fluentd.org/v1.0/articles/"
 
   def initialize(argv = ARGV)
     @argv = argv
@@ -185,7 +186,7 @@ class FluentPluginConfigFormatter
   end
 
   def plugin_helper_url(plugin_helper)
-    "#{DOCS_BASE_URL}api-plugin-helper-#{plugin_helper}"
+    "#{DOCS_ARTICLE_BASE_URL}api-plugin-helper-#{plugin_helper}"
   end
 
   def plugin_helper_markdown_link(plugin_helper)
@@ -194,7 +195,7 @@ class FluentPluginConfigFormatter
 
   def plugin_overview_url(class_name)
     plugin_type = class_name.slice(/::(\w+)\z/, 1).downcase
-    "#{DOCS_BASE_URL}#{plugin_type}-plugin-overview"
+    "#{DOCS_ARTICLE_BASE_URL}#{plugin_type}-plugin-overview"
   end
 
   def plugin_overview_markdown_link(class_name)

data/lib/fluent/counter/client.rb

@@ -288,7 +288,7 @@ module Fluent
       def join
         until @set
           @mutex.synchronize do
-            @loop.run_once(0.0001) # retun a lock as soon as possible
+            @loop.run_once(0.0001) # return a lock as soon as possible
           end
         end
       end

data/lib/fluent/plugin/compressable.rb

@@ -44,7 +44,7 @@ module Fluent
           # check compressed_data(String) is 0 length
           compressed_data
         when output_io
-          # exeucte after checking compressed_data is empty or not
+          # execute after checking compressed_data is empty or not
           io = StringIO.new(compressed_data)
           io_decompress(io, output_io)
         else

data/lib/fluent/plugin/in_syslog.rb

@@ -169,6 +169,10 @@ module Fluent::Plugin
               num = Integer(buffer[pos..idx])
               pos = idx + num
               msg = buffer[idx + 1...pos]
+              if msg.size < num - 1
+                pos = pos - num - num.to_s.size
+                break
+              end
               message_handler(msg, conn)
             end
           else

data/lib/fluent/plugin/out_forward.rb

@@ -102,7 +102,7 @@ module Fluent::Plugin
     desc 'The client private key path for TLS.'
     config_param :tls_client_private_key_path, :string, default: nil
     desc 'The client private key passphrase for TLS.'
-    config_param :tls_client_private_key_passphrase, :string, default: nil
+    config_param :tls_client_private_key_passphrase, :string, default: nil, secret: true
 
     config_section :security, required: false, multi: false do
       desc 'The hostname'

data/lib/fluent/plugin/parser_regexp.rb

@@ -40,6 +40,10 @@ module Fluent
           @expression = Regexp.compile(@expression.source, options)
         end
         @regexp = @expression # For backward compatibility
+
+        if @expression.named_captures.empty?
+          raise Fluent::ConfigError, "No named captures in 'expression' parameter. The regexp must have at least one named capture"
+        end
       end
 
       def parse(text)

data/lib/fluent/supervisor.rb

@@ -716,7 +716,13 @@ module Fluent
     end
 
     def main_process(&block)
-      Process.setproctitle("worker:#{@process_name}") if @process_name
+      if @process_name
+        if @workers > 1
+          Process.setproctitle("worker:#{@process_name}#{ENV['SERVERENGINE_WORKER_ID']}")
+        else
+          Process.setproctitle("worker:#{@process_name}")
+        end
+      end
 
       unrecoverable_error = false
 

data/lib/fluent/version.rb

@@ -16,6 +16,6 @@
 
 module Fluent
 
-  VERSION = '1.4.0'
+  VERSION = '1.4.1'
 
 end

data/test/compat/test_parser.rb

@@ -84,7 +84,7 @@ class TextParserTest < ::Test::Unit::TestCase
        multiline: Regexp::MULTILINE,
        both: Regexp::IGNORECASE & Regexp::MULTILINE)
   def test_regexp_parser_config(options)
-    source = "a"
+    source = "(?<test>.*)"
     parser = Fluent::TextParser::RegexpParser.new(Regexp.new(source, options), { "dummy" => "dummy" })
     regexp = parser.instance_variable_get("@regexp")
     assert_equal(options, regexp.options)

data/test/plugin/test_parser_regexp.rb

@@ -153,6 +153,15 @@ class RegexpParserTest < ::Test::Unit::TestCase
     end
 
     sub_test_case "configure" do
+      def test_bad_expression
+        conf = {
+          'expression' => %q!/.*/!,
+        }
+        assert_raise Fluent::ConfigError do
+          create_driver(conf)
+        end
+      end
+
       def test_default_options
         conf = {
           'expression' => %q!/^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] \[(?<date>[^\]]*)\] "(?<flag>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)$/!,

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fluentd
 version: !ruby/object:Gem::Version
-  version: 1.4.0
+  version: 1.4.1
 platform: ruby
 authors:
 - Sadayuki Furuhashi
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-02-25 00:00:00.000000000 Z
+date: 2019-03-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: msgpack
@@ -329,6 +329,7 @@ extensions: []
 extra_rdoc_files: []
 files:
 - ".github/ISSUE_TEMPLATE.md"
+- ".github/PULL_REQUEST_TEMPLATE.md"
 - ".gitignore"
 - ".travis.yml"
 - ADOPTERS.md
@@ -337,6 +338,7 @@ files:
 - CONTRIBUTING.md
 - GOVERNANCE.md
 - Gemfile
+- GithubWorkflow.md
 - LICENSE
 - MAINTAINERS.md
 - README.md