fluent-plugin-systemd 1.0.0 → 1.0.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 05b2387f068e7c0ec23f74f61bb5d1b031fbd670
-  data.tar.gz: 438b9f07ea3b23d6af05077ec460f51e413002e4
+SHA256:
+  metadata.gz: 4cb4c7536e7d15ebea8d31cab4e9bd1902cfcdf3fcb9adba98b7c957d16a78f0
+  data.tar.gz: 1d18922ec4c903834251e2a325e911c9d4746c164b69d2947aadb0b3c768ab3e
 SHA512:
-  metadata.gz: 23d0e490a9befa2d01ea8fd6092dbed234ea8a31e3433cb458a584195d0ba6a08376881641df75db4fb139887e869b66c935a58ce7daf796d24bea8b533020fa
-  data.tar.gz: 7ac5767e188b7ef944c62415a7e13eb100effa83f05784377b7fa071b9a6cbc52eaf72f7f45a5d593d0302474e4ca858d746a616ae1ec2a8522c62f20cc09dc8
+  metadata.gz: 601f2f07b040378c78a53f8cdce16a53a3d691a9634b4a8e7d85c4c6bf1ec5c2d84f52271ff0a6040d70437f97f975825ff228809e7524921b02bdcd0715c154
+  data.tar.gz: 0f42d023c28dbd644391693c7973ccb34542fc71fcf5e0b2ac97178522549fc7e2dd43b6472622d313a9008073c9712d8b15eb809bc676ca1121e5ec5cd669a9

data/README.md

@@ -1,6 +1,6 @@
 # systemd plugin for [Fluentd](http://github.com/fluent/fluentd)
 
-[![Build Status](https://travis-ci.org/reevoo/fluent-plugin-systemd.svg?branch=master)](https://travis-ci.org/reevoo/fluent-plugin-systemd) [![Code Climate GPA](https://codeclimate.com/github/reevoo/fluent-plugin-systemd/badges/gpa.svg)](https://codeclimate.com/github/reevoo/fluent-plugin-systemd) [![Gem Version](https://badge.fury.io/rb/fluent-plugin-systemd.svg)](https://rubygems.org/gems/fluent-plugin-systemd)
+[![Build Status](https://github.com/fluent-plugin-systemd/fluent-plugin-systemd/actions/workflows/ci.yml/badge.svg)](https://github.com/fluent-plugin-systemd/fluent-plugin-systemd/actions/workflows/ci.yml) [![Maintainability](https://api.codeclimate.com/v1/badges/5c773a4c1c6a6964fa4b/maintainability)](https://codeclimate.com/github/fluent-plugin-systemd/fluent-plugin-systemd/maintainability) [![Gem Version](https://badge.fury.io/rb/fluent-plugin-systemd.svg)](https://rubygems.org/gems/fluent-plugin-systemd)
 
 ## Overview
 
@@ -9,11 +9,8 @@
 
 ## Support
 
-[![Fluentd Slack](http://slack.fluentd.org/badge.svg)](http://slack.fluentd.org/)
-
 Join the #plugin-systemd channel on the [Fluentd Slack](http://slack.fluentd.org/)
 
-
 ## Requirements
 
 |fluent-plugin-systemd|fluentd|td-agent|ruby|
@@ -28,11 +25,11 @@ Join the #plugin-systemd channel on the [Fluentd Slack](http://slack.fluentd.org
 
 Simply use RubyGems:
 
-    gem install fluent-plugin-systemd -v 1.0.0
+    gem install fluent-plugin-systemd -v 1.0.3
 
 or
 
-    td-agent-gem install fluent-plugin-systemd -v 1.0.0
+    td-agent-gem install fluent-plugin-systemd -v 1.0.3
 
 ## Upgrading
 
@@ -42,25 +39,30 @@ If you are upgrading to version 1.0 from a previous version of this plugin take
 
     <source>
       @type systemd
-      tag kube-proxy
+      tag kubelet
       path /var/log/journal
-      matches [{ "_SYSTEMD_UNIT": "kube-proxy.service" }]
+      matches [{ "_SYSTEMD_UNIT": "kubelet.service" }]
       read_from_head true
+
       <storage>
         @type local
-        persistent false
-        path kube-proxy.pos
+        path /var/log/fluentd-journald-kubelet-cursor.json
       </storage>
+
       <entry>
         fields_strip_underscores true
         fields_lowercase true
       </entry>
     </source>
 
-    <match kube-proxy>
+    <match kubelet>
       @type stdout
     </match>
 
+    <system>
+      root_dir /var/log/fluentd
+    </system>
+
 **`path`**
 
 Path to the systemd journal, defaults to `/var/log/journal`
@@ -80,7 +82,7 @@ description of this property and how to use it.
 
 **`storage`**
 
-Configuration for a [storage plugin](http://docs.fluentd.org/v0.14/articles/storage-plugin-overview) used to store the journald cursor.
+Configuration for a [storage plugin](https://docs.fluentd.org/storage) used to store the journald cursor.
 
 **`read_from_head`**
 
@@ -186,6 +188,16 @@ Another strategy would be to use a plugin like [fluent-plugin-concat](https://gi
 * You can use an [offical fluentd docker](https://github.com/fluent/fluentd-docker-image) image as a base, (choose the debian based version, as alpine linux doesn't support systemd).
 * Bind mount `/var/log/journal` into your container.
 
+> ### I am seeing lots of logs being generated very rapidly!
+
+This commonly occurs when a loop is created when fluentd is logging to STDOUT, and the collected logs are then written to the systemd journal. This could happen if you run fluentd as a systemd serivce, or as a docker container with the systemd log driver.
+
+Workarounds include:
+
+* Use another fluentd output
+* Don't read every message from the journal, set some `matches` so you only read the messages you are interested in.
+* Disable the systemd log driver when you launch your fluentd docker container, e.g. by passing `--log-driver json-file`
+
 ### Example
 
 For an example of a full working setup including the plugin, take a look at [the fluentd kubernetes daemonset](https://github.com/fluent/fluentd-kubernetes-daemonset)
@@ -238,14 +250,16 @@ We have adopted the [Contributor Covenant](CODE_OF_CONDUCT.md) and thus expect a
 
 Many thanks to our fantastic contributors
 
-* [Hiroshi Hatake](https://github.com/cosmo0920)
 * [Erik Maciejewski](https://github.com/emacski)
-* [Masahiro Nakagawa](https://github.com/repeatedly)
-* [Richard Megginson](https://github.com/richm)
-* [Mike Kaplinskiy](https://github.com/mikekap)
-* [neko-neko](https://github.com/neko-neko)
-* [Sadayuki Furuhashi](https://github.com/frsyuki)
+* [Ernie Hershey](https://github.com/ehershey)
+* [Hiroshi Hatake](https://github.com/cosmo0920)
 * [Jesus Rafael Carrillo](https://github.com/jescarri)
+* [Joel Gerber](https://github.com/Jitsusama)
 * [John Thomas Wile II](https://github.com/jtwile2)
 * [Kazuhiro Suzuki](https://github.com/ksauzz)
-* [Joel Gerber](https://github.com/Jitsusama)
+* [Marius Grigoriu](https://github.com/mariusgrigoriu)
+* [Masahiro Nakagawa](https://github.com/repeatedly)
+* [Mike Kaplinskiy](https://github.com/mikekap)
+* [Richard Megginson](https://github.com/richm)
+* [Sadayuki Furuhashi](https://github.com/frsyuki)
+* [Seiichi Nishikata](https://github.com/neko-neko)

data/lib/fluent/plugin/in_systemd.rb

@@ -21,7 +21,7 @@ require 'fluent/plugin/systemd/entry_mutator'
 module Fluent
   module Plugin
     # Fluentd plugin for reading from the systemd journal
-    class SystemdInput < Input
+    class SystemdInput < Input # rubocop:disable Metrics/ClassLength
       Fluent::Plugin.register_input('systemd', self)
 
       helpers :timer, :storage
@@ -57,18 +57,25 @@ module Fluent
 
       def start
         super
+        @running = true
         timer_execute(:in_systemd_emit_worker, 1, &method(:run))
       end
 
+      def shutdown
+        @running = false
+        @journal&.close
+        @journal = nil
+        @pos_storage = nil
+        @mutator = nil
+        super
+      end
+
       private
 
       def init_journal
         # TODO: ruby 2.3
         @journal.close if @journal # rubocop:disable Style/SafeNavigation
         @journal = Systemd::Journal.new(path: @path)
-        # make sure initial call to wait doesn't return :invalidate
-        # see https://github.com/ledbettj/systemd-journal/issues/70
-        @journal.wait(0)
         @journal.filter(*(@matches || @filters))
         seek
         true
@@ -95,6 +102,7 @@ module Fluent
       def seek_to(pos)
         @journal.seek(pos)
         return if pos == :head
+
         if pos == :tail
           @journal.move(-2)
         else
@@ -108,6 +116,7 @@ module Fluent
 
       def run
         return unless @journal || init_journal
+
         init_journal if @journal.wait(0) == :invalidate
         watch do |entry|
           emit(entry)
@@ -119,6 +128,7 @@ module Fluent
       rescue Fluent::Plugin::Buffer::BufferOverflowError => e
         retries ||= 0
         raise e if retries > 10
+
         retries += 1
         sleep 1.5**retries + rand(0..3)
         retry
@@ -131,7 +141,9 @@ module Fluent
       end
 
       def watch(&block)
-        yield_current_entry(&block) while @journal.move_next
+        yield_current_entry(&block) while @running && @journal.move_next
+      rescue Systemd::JournalError => e
+        log.warn("Error moving to next Journal entry: #{e.class}: #{e.message}")
       end
 
       def yield_current_entry

data/lib/fluent/plugin/systemd/entry_mutator.rb

@@ -67,6 +67,7 @@ module Fluent
       # Expose config state as read-only instance properties of the mutator.
       def method_missing(sym, *args)
         return @opts[sym] if @opts.members.include?(sym)
+
         super
       end
 
@@ -80,6 +81,7 @@ module Fluent
       def run(entry)
         return entry.to_h if @no_transform
         return map_fields(entry) if @opts.field_map_strict
+
         format_fields(entry, map_fields(entry))
       end
 
@@ -90,6 +92,7 @@ module Fluent
         @map.each_with_object({}) do |(cstm, sysds), mapped|
           vals = sysds.collect { |fld| entry[fld] }.compact
           next if vals.empty? # systemd field does not exist in source entry
+
           mapped[cstm] = join_if_needed(vals)
         end
       end
@@ -103,6 +106,7 @@ module Fluent
         entry.each_with_object(mapped || {}) do |(fld, val), formatted_entry|
           # don't mess with explicitly mapped fields
           next if @map_src_fields.include?(fld)
+
           fld = format_field_name(fld)
           # account for mapping (appending) to an existing systemd field
           formatted_entry[fld] = join_if_needed([val, mapped[fld]])
@@ -111,6 +115,7 @@ module Fluent
 
       def warnings
         return [] unless field_map_strict && field_map.empty?
+
         '`field_map_strict` set to true with empty `field_map`, expect no fields'
       end
 
@@ -119,6 +124,7 @@ module Fluent
       def join_if_needed(values)
         values.compact!
         return values.first if values.length == 1
+
         values.join(' ')
       end
 
@@ -146,7 +152,7 @@ module Fluent
         end
       end
 
-      def validate_all_strings(arr, message, allow_nesting = false)
+      def validate_all_strings(arr, message, allow_nesting = false) # rubocop:disable Style/OptionalBooleanParameter
         valid = arr.all? do |value|
           value.is_a?(String) || allow_nesting && value.is_a?(Array) && value.all? { |key| key.is_a?(String) }
         end

metadata

@@ -1,27 +1,27 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-systemd
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.0.5
 platform: ruby
 authors:
 - Ed Robinson
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2018-05-10 00:00:00.000000000 Z
+date: 2021-05-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">"
       - !ruby/object:Gem::Version
         version: '1.10'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">"
       - !ruby/object:Gem::Version
         version: '1.10'
 - !ruby/object:Gem::Dependency
@@ -44,28 +44,28 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.5'
+        version: '3.4'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.5'
+        version: '3.4'
 - !ruby/object:Gem::Dependency
   name: rubocop
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
-        version: 0.53.0
+        version: 1.13.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
-        version: 0.53.0
+        version: 1.13.0
 - !ruby/object:Gem::Dependency
   name: fluentd
   requirement: !ruby/object:Gem::Requirement
@@ -92,14 +92,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.3'
+        version: 1.4.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.3'
+        version: 1.4.2
 description: This is a fluentd input plugin. It reads logs from the systemd journal.
 email:
 - edward-robinson@cookpad.com
@@ -116,7 +116,7 @@ homepage: https://github.com/reevoo/fluent-plugin-systemd
 licenses:
 - Apache-2.0
 metadata: {}
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -131,9 +131,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.6.13
-signing_key: 
+rubygems_version: 3.2.15
+signing_key:
 specification_version: 4
 summary: Input plugin to read from systemd journal.
 test_files: []