fluent-plugin-insert-id 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3925e1cc5872f73a4f97fa577d614139b07d7ef52803a30bfc4d2fecaac8e2e1
-  data.tar.gz: a298505609f2930adc4ef16a4ed8e8eb19f8149d819f81c7dcd4dc60fa8f5d87
+  metadata.gz: 3f8c9bed82fc7deb1ce5d231f9ec119ab6c581202e1b1549e940528de458a4f3
+  data.tar.gz: 4c4ce9dee089745f4ab6e8b1f07634fa3fd388c6073c6dfcf077608b960df65a
 SHA512:
-  metadata.gz: 3a6af797ba66dcde07c9080e083757cb8ac6695d89a54282ad3731fef616afbe7602a27c0291cb8ed2c4c52225698b3165fbf62700706f7f839c442978f8d2a5
-  data.tar.gz: 8cf9c974ae6705da36acaee38457672c3828e67a859e2d50484968eb3346328657d6e0fa2259a1626a5dbba525cc872ea97019d0b53405dda6e1112d25e93dff
+  metadata.gz: 8ef0f27050c7ec286b60adc83fc82539d6d33375c86a2a26ee9b0e3e55b879084bf8293205d7f3eb710ffb7caaf5e9cf4ebb6bfa7b1bedbcbe07880f025cdffe
+  data.tar.gz: 242363d6a2af8ac79b2b5ca2212ec97d02e01a489f13ce8c5d8a7c2982f0d43a58fee6ba2643778fd56cbcbc793738b803823bf994f33d38070d313ca643173f

data/.gitignore

@@ -1 +1,8 @@
 fluent.conf
+vendor
+.bundle
+Gemfile.lock
+.ruby-version
+.rbenv-version
+tmp
+pkg

data/README.md

@@ -1,9 +1,10 @@
 # fluent-plugin-insert-id
 
-[Fluentd](https://fluentd.org/) filter plugin to insert unique ID string.
+[Fluentd](https://fluentd.org/) filter plugin to insert unique ID string into the message.
 
 The original implementation was copied from [fluent-plugin-google-cloud](https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud)
-which was created by Google to handle GKE's official Stackdriver Logging reporting in sidecar containe for each pod.
+which was created by Google to handle GCP GKE's official Stackdriver Logging reporting in sidecar container for each pod.
+Original source requires fluentd v0.12 and doesn't support fluentd v1.0 but this plugin only supports fluentd v1.0.
 
 ## How it works
 ```
@@ -22,7 +23,7 @@ which was created by Google to handle GKE's official Stackdriver Logging reporti
 2019-08-25 21:20:50.035415329 +0000 message.test: {"a":"foo","b":"bar","insert-id":"nu8a3ptahpbetdea"}
 ```
 
-* It is guaranteed that ID is fixed length string which contains  0-9 and a-z lowercase characters.
+* Inserted ID contains 0-9 and a-z lowercase characters.
 * Initial ID generated is random string like "nu8a3ptahpbetddc".
 * Series ID after the initial ID are 'incremented' string which uses Ruby's `String.next()`.
 * 'incremented' string also has 'carry' feature. Please check below links for more details.
@@ -30,6 +31,40 @@ which was created by Google to handle GKE's official Stackdriver Logging reporti
   * [String.next() (Japanese)](https://docs.ruby-lang.org/ja/2.4.0/class/String.html#I_NEXT)
 * This ordered ID makes debugging easier in most cases.
 
+#### ID string length
+* **From version 1.1.0, it is guaranteed that ID is fixed length string. **
+* **In version 1.0.0, the ID string length is incremented when carry happens at left-most characters.**
+
+```ruby
+# Version 1.0.0
+{"a":"foo","b":"bar","insert-id":"z99999999999999z"}
+{"a":"foo","b":"bar","insert-id":"aa00000000000000z"} # Left most character carry adds new digit.
+
+# Version 1.1.0
+{"a":"foo","b":"bar","insert-id":"z99999999999999z"}
+{"a":"foo","b":"bar","insert-id":"a00000000000000z"} # Left most character carry is ignored.
+```
+
+#### Existing ID protection
+If the message already has the key for inserted ID, the filter doesn't touch it and
+existing value is protected.
+
+```
+2019-08-27 02:10:07.422911774 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsv"}
+2019-08-27 02:10:08.129842499 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsw"}
+2019-08-27 02:10:08.940316454 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsx"}
+2019-08-27 02:11:02.498772740 +0000 message.test: {"a":"foo","b":"bar","insert-id":"existing_ID"}
+2019-08-27 02:11:06.802934944 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsy"}
+```
+
+## Requirements
+
+| fluent-plugin-insert-id  | fluentd | ruby |
+|--------------------------|---------|------|
+| >= 1.0.0 | >= v0.14.x | >= 2.1 |
+
+fluentd v0.12 is not supported.
+
 ## Installation
 
 ### RubyGems
@@ -54,13 +89,26 @@ $ bundle
 
 ## Configuration
 
-You can generate configuration template:
+**insert_id_key** (string) (optional)
 
-```
-$ fluent-plugin-config-format filter insert-id
+The key of inserted-id.
+Default value is `"insert-id"`.
+
+## Usage : Insert ID with default key
+```aconf
+<filter **>
+  @type insert_id
+</filter>
 ```
 
-You can copy and paste generated documents here.
+## Usage : Insert ID with custom key
+In this case, key of ID is changed to 'message_id'
+```aconf
+<filter **>
+  @type insert_id
+  insert_id_key message_id
+</filter>
+```
 
 ## Copyright
 

data/fluent-plugin-insert-id.gemspec

@@ -3,12 +3,12 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name    = "fluent-plugin-insert-id"
-  spec.version = "1.0.0"
+  spec.version = "1.1.0"
   spec.authors = ["Kamome Shido"]
   spec.email   = ["shidokamo@users.noreply.github.com"]
 
-  spec.summary       = %q{fluentd filter plugin to insert unique id}
-  spec.description   = %q{fluentd filter plugin to insert unique id}
+  spec.summary       = %q{fluentd filter plugin to insert unique id into the message}
+  spec.description   = %q{fluentd filter plugin to insert unique id into the message}
   spec.homepage      = "https://github.com/shidokamo/fluent-plugin-insert-id"
   spec.license       = "Apache-2.0"
 
@@ -20,8 +20,7 @@ Gem::Specification.new do |spec|
   spec.test_files    = test_files
   spec.require_paths = ["lib"]
 
-  spec.add_development_dependency "bundler", "~> 1.14"
-  spec.add_development_dependency "rake", "~> 12.0"
-  spec.add_development_dependency "test-unit", "~> 3.0"
+  spec.add_development_dependency "rake", ">= 12.0"
+  spec.add_development_dependency "test-unit", ">= 3.0"
   spec.add_runtime_dependency "fluentd", [">= 0.14.10", "< 2"]
 end

data/lib/fluent/plugin/filter_insert_id.rb

@@ -57,7 +57,7 @@ module Fluent
       def start
         super
         # Initialize the ID
-        log.info "Started the add_insert_ids plugin with #{@insert_id_key} as the insert ID key."
+        log.info "Started the insert-id plugin with #{@insert_id_key} as the insert ID key."
         @insert_id = generate_initial_insert_id
         log.info "Initialized the insert ID key to #{@insert_id}."
       end
@@ -85,7 +85,7 @@ module Fluent
 
       # Increment the insertId and return the new value.
       def increment_insert_id
-        @insert_id = @insert_id.next
+        @insert_id = @insert_id.next.slice(-INSERT_ID_SIZE,INSERT_ID_SIZE) # slice is required to make sure ID has fixed length.
       end
     end
   end

metadata

@@ -1,55 +1,41 @@
 --- !ruby/object:Gem::Specification
 name: fluent-plugin-insert-id
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Kamome Shido
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-08-26 00:00:00.000000000 Z
+date: 2019-08-27 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.14'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.14'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '12.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '12.0'
 - !ruby/object:Gem::Dependency
   name: test-unit
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '3.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '3.0'
 - !ruby/object:Gem::Dependency
@@ -72,7 +58,7 @@ dependencies:
     - - "<"
       - !ruby/object:Gem::Version
         version: '2'
-description: fluentd filter plugin to insert unique id
+description: fluentd filter plugin to insert unique id into the message
 email:
 - shidokamo@users.noreply.github.com
 executables: []
@@ -110,7 +96,7 @@ requirements: []
 rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
-summary: fluentd filter plugin to insert unique id
+summary: fluentd filter plugin to insert unique id into the message
 test_files:
 - test/helper.rb
 - test/plugin/test_filter_insert_id.rb