fluent-plugin-insert-id 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3925e1cc5872f73a4f97fa577d614139b07d7ef52803a30bfc4d2fecaac8e2e1
4
- data.tar.gz: a298505609f2930adc4ef16a4ed8e8eb19f8149d819f81c7dcd4dc60fa8f5d87
3
+ metadata.gz: 3f8c9bed82fc7deb1ce5d231f9ec119ab6c581202e1b1549e940528de458a4f3
4
+ data.tar.gz: 4c4ce9dee089745f4ab6e8b1f07634fa3fd388c6073c6dfcf077608b960df65a
5
5
  SHA512:
6
- metadata.gz: 3a6af797ba66dcde07c9080e083757cb8ac6695d89a54282ad3731fef616afbe7602a27c0291cb8ed2c4c52225698b3165fbf62700706f7f839c442978f8d2a5
7
- data.tar.gz: 8cf9c974ae6705da36acaee38457672c3828e67a859e2d50484968eb3346328657d6e0fa2259a1626a5dbba525cc872ea97019d0b53405dda6e1112d25e93dff
6
+ metadata.gz: 8ef0f27050c7ec286b60adc83fc82539d6d33375c86a2a26ee9b0e3e55b879084bf8293205d7f3eb710ffb7caaf5e9cf4ebb6bfa7b1bedbcbe07880f025cdffe
7
+ data.tar.gz: 242363d6a2af8ac79b2b5ca2212ec97d02e01a489f13ce8c5d8a7c2982f0d43a58fee6ba2643778fd56cbcbc793738b803823bf994f33d38070d313ca643173f
data/.gitignore CHANGED
@@ -1 +1,8 @@
1
1
  fluent.conf
2
+ vendor
3
+ .bundle
4
+ Gemfile.lock
5
+ .ruby-version
6
+ .rbenv-version
7
+ tmp
8
+ pkg
data/README.md CHANGED
@@ -1,9 +1,10 @@
1
1
  # fluent-plugin-insert-id
2
2
 
3
- [Fluentd](https://fluentd.org/) filter plugin to insert unique ID string.
3
+ [Fluentd](https://fluentd.org/) filter plugin to insert unique ID string into the message.
4
4
 
5
5
  The original implementation was copied from [fluent-plugin-google-cloud](https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud)
6
- which was created by Google to handle GKE's official Stackdriver Logging reporting in sidecar containe for each pod.
6
+ which was created by Google to handle GCP GKE's official Stackdriver Logging reporting in sidecar container for each pod.
7
+ Original source requires fluentd v0.12 and doesn't support fluentd v1.0 but this plugin only supports fluentd v1.0.
7
8
 
8
9
  ## How it works
9
10
  ```
@@ -22,7 +23,7 @@ which was created by Google to handle GKE's official Stackdriver Logging reporti
22
23
  2019-08-25 21:20:50.035415329 +0000 message.test: {"a":"foo","b":"bar","insert-id":"nu8a3ptahpbetdea"}
23
24
  ```
24
25
 
25
- * It is guaranteed that ID is fixed length string which contains 0-9 and a-z lowercase characters.
26
+ * Inserted ID contains 0-9 and a-z lowercase characters.
26
27
  * Initial ID generated is random string like "nu8a3ptahpbetddc".
27
28
  * Series ID after the initial ID are 'incremented' string which uses Ruby's `String.next()`.
28
29
  * 'incremented' string also has 'carry' feature. Please check below links for more details.
@@ -30,6 +31,40 @@ which was created by Google to handle GKE's official Stackdriver Logging reporti
30
31
  * [String.next() (Japanese)](https://docs.ruby-lang.org/ja/2.4.0/class/String.html#I_NEXT)
31
32
  * This ordered ID makes debugging easier in most cases.
32
33
 
34
+ #### ID string length
35
+ * **From version 1.1.0, it is guaranteed that ID is fixed length string. **
36
+ * **In version 1.0.0, the ID string length is incremented when carry happens at left-most characters.**
37
+
38
+ ```ruby
39
+ # Version 1.0.0
40
+ {"a":"foo","b":"bar","insert-id":"z99999999999999z"}
41
+ {"a":"foo","b":"bar","insert-id":"aa00000000000000z"} # Left most character carry adds new digit.
42
+
43
+ # Version 1.1.0
44
+ {"a":"foo","b":"bar","insert-id":"z99999999999999z"}
45
+ {"a":"foo","b":"bar","insert-id":"a00000000000000z"} # Left most character carry is ignored.
46
+ ```
47
+
48
+ #### Existing ID protection
49
+ If the message already has the key for inserted ID, the filter doesn't touch it and
50
+ existing value is protected.
51
+
52
+ ```
53
+ 2019-08-27 02:10:07.422911774 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsv"}
54
+ 2019-08-27 02:10:08.129842499 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsw"}
55
+ 2019-08-27 02:10:08.940316454 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsx"}
56
+ 2019-08-27 02:11:02.498772740 +0000 message.test: {"a":"foo","b":"bar","insert-id":"existing_ID"}
57
+ 2019-08-27 02:11:06.802934944 +0000 message.test: {"a":"foo","b":"bar","insert-id":"ehrbwzp772xitjsy"}
58
+ ```
59
+
60
+ ## Requirements
61
+
62
+ | fluent-plugin-insert-id | fluentd | ruby |
63
+ |--------------------------|---------|------|
64
+ | >= 1.0.0 | >= v0.14.x | >= 2.1 |
65
+
66
+ fluentd v0.12 is not supported.
67
+
33
68
  ## Installation
34
69
 
35
70
  ### RubyGems
@@ -54,13 +89,26 @@ $ bundle
54
89
 
55
90
  ## Configuration
56
91
 
57
- You can generate configuration template:
92
+ **insert_id_key** (string) (optional)
58
93
 
59
- ```
60
- $ fluent-plugin-config-format filter insert-id
94
+ The key of inserted-id.
95
+ Default value is `"insert-id"`.
96
+
97
+ ## Usage : Insert ID with default key
98
+ ```aconf
99
+ <filter **>
100
+ @type insert_id
101
+ </filter>
61
102
  ```
62
103
 
63
- You can copy and paste generated documents here.
104
+ ## Usage : Insert ID with custom key
105
+ In this case, key of ID is changed to 'message_id'
106
+ ```aconf
107
+ <filter **>
108
+ @type insert_id
109
+ insert_id_key message_id
110
+ </filter>
111
+ ```
64
112
 
65
113
  ## Copyright
66
114
 
@@ -3,12 +3,12 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
3
3
 
4
4
  Gem::Specification.new do |spec|
5
5
  spec.name = "fluent-plugin-insert-id"
6
- spec.version = "1.0.0"
6
+ spec.version = "1.1.0"
7
7
  spec.authors = ["Kamome Shido"]
8
8
  spec.email = ["shidokamo@users.noreply.github.com"]
9
9
 
10
- spec.summary = %q{fluentd filter plugin to insert unique id}
11
- spec.description = %q{fluentd filter plugin to insert unique id}
10
+ spec.summary = %q{fluentd filter plugin to insert unique id into the message}
11
+ spec.description = %q{fluentd filter plugin to insert unique id into the message}
12
12
  spec.homepage = "https://github.com/shidokamo/fluent-plugin-insert-id"
13
13
  spec.license = "Apache-2.0"
14
14
 
@@ -20,8 +20,7 @@ Gem::Specification.new do |spec|
20
20
  spec.test_files = test_files
21
21
  spec.require_paths = ["lib"]
22
22
 
23
- spec.add_development_dependency "bundler", "~> 1.14"
24
- spec.add_development_dependency "rake", "~> 12.0"
25
- spec.add_development_dependency "test-unit", "~> 3.0"
23
+ spec.add_development_dependency "rake", ">= 12.0"
24
+ spec.add_development_dependency "test-unit", ">= 3.0"
26
25
  spec.add_runtime_dependency "fluentd", [">= 0.14.10", "< 2"]
27
26
  end
@@ -57,7 +57,7 @@ module Fluent
57
57
  def start
58
58
  super
59
59
  # Initialize the ID
60
- log.info "Started the add_insert_ids plugin with #{@insert_id_key} as the insert ID key."
60
+ log.info "Started the insert-id plugin with #{@insert_id_key} as the insert ID key."
61
61
  @insert_id = generate_initial_insert_id
62
62
  log.info "Initialized the insert ID key to #{@insert_id}."
63
63
  end
@@ -85,7 +85,7 @@ module Fluent
85
85
 
86
86
  # Increment the insertId and return the new value.
87
87
  def increment_insert_id
88
- @insert_id = @insert_id.next
88
+ @insert_id = @insert_id.next.slice(-INSERT_ID_SIZE,INSERT_ID_SIZE) # slice is required to make sure ID has fixed length.
89
89
  end
90
90
  end
91
91
  end
metadata CHANGED
@@ -1,55 +1,41 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-insert-id
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0
4
+ version: 1.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kamome Shido
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-08-26 00:00:00.000000000 Z
11
+ date: 2019-08-27 00:00:00.000000000 Z
12
12
  dependencies:
13
- - !ruby/object:Gem::Dependency
14
- name: bundler
15
- requirement: !ruby/object:Gem::Requirement
16
- requirements:
17
- - - "~>"
18
- - !ruby/object:Gem::Version
19
- version: '1.14'
20
- type: :development
21
- prerelease: false
22
- version_requirements: !ruby/object:Gem::Requirement
23
- requirements:
24
- - - "~>"
25
- - !ruby/object:Gem::Version
26
- version: '1.14'
27
13
  - !ruby/object:Gem::Dependency
28
14
  name: rake
29
15
  requirement: !ruby/object:Gem::Requirement
30
16
  requirements:
31
- - - "~>"
17
+ - - ">="
32
18
  - !ruby/object:Gem::Version
33
19
  version: '12.0'
34
20
  type: :development
35
21
  prerelease: false
36
22
  version_requirements: !ruby/object:Gem::Requirement
37
23
  requirements:
38
- - - "~>"
24
+ - - ">="
39
25
  - !ruby/object:Gem::Version
40
26
  version: '12.0'
41
27
  - !ruby/object:Gem::Dependency
42
28
  name: test-unit
43
29
  requirement: !ruby/object:Gem::Requirement
44
30
  requirements:
45
- - - "~>"
31
+ - - ">="
46
32
  - !ruby/object:Gem::Version
47
33
  version: '3.0'
48
34
  type: :development
49
35
  prerelease: false
50
36
  version_requirements: !ruby/object:Gem::Requirement
51
37
  requirements:
52
- - - "~>"
38
+ - - ">="
53
39
  - !ruby/object:Gem::Version
54
40
  version: '3.0'
55
41
  - !ruby/object:Gem::Dependency
@@ -72,7 +58,7 @@ dependencies:
72
58
  - - "<"
73
59
  - !ruby/object:Gem::Version
74
60
  version: '2'
75
- description: fluentd filter plugin to insert unique id
61
+ description: fluentd filter plugin to insert unique id into the message
76
62
  email:
77
63
  - shidokamo@users.noreply.github.com
78
64
  executables: []
@@ -110,7 +96,7 @@ requirements: []
110
96
  rubygems_version: 3.0.3
111
97
  signing_key:
112
98
  specification_version: 4
113
- summary: fluentd filter plugin to insert unique id
99
+ summary: fluentd filter plugin to insert unique id into the message
114
100
  test_files:
115
101
  - test/helper.rb
116
102
  - test/plugin/test_filter_insert_id.rb