fluent-plugin-masking 1.0.6 → 1.0.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 5ba6a8f7cb4b592158ba29e44ad3408995b2e6170030277fc071c8d87558d636
4
- data.tar.gz: bda9061abbb827e79162b0aaee0859b44034d2cee6dfcec448c1ca626ba86509
3
+ metadata.gz: 3829c2b34592e0818d551864f92a43d914285b899f517a36918b4f4ccd06efcc
4
+ data.tar.gz: 9a10456cdd7baef47fcad37a322fc497b5f7261612592822ee710887979da3dc
5
5
  SHA512:
6
- metadata.gz: dc4635b29a99ff1d446b68f1a40937a9469d1919f0d65a14c81a03d3a7e124c9d2cb6ab8ecfe2d1ee40101b009a48d6ae2cf1c314fc23636dd56b7f67d9d58cf
7
- data.tar.gz: 638dee8fc6e7c903cc692cdf5e9d691d1d8901af2390daad53a450e3980e02cdd295ecbcee9c43743c6348be24475b75b1a8881c398371042b747afcd94a26b7
6
+ metadata.gz: 9df328614e49ab9190ba647c28956c19d184471b6cfb332a8d9a5a279dd0f035e75ff17a71e7c51faf8c6d1f8aa2b6500755986c41f262569f05d68fbafc8352
7
+ data.tar.gz: '09ac4052c21d64b735ffc9b44aada2ecaf2cff678a3ca45de4eb1280940456c0c0cc493a3702b0c27d358bb07a045aad72e30f00f98070993eccb09da21efed6'
@@ -1,8 +1,8 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- fluent-plugin-masking (1.0.4)
5
- fluentd (>= 0.14.0, < 2)
4
+ fluent-plugin-masking (1.0.6)
5
+ fluentd (>= 0.14.0)
6
6
 
7
7
  GEM
8
8
  remote: https://rubygems.org/
data/README.md CHANGED
@@ -1 +1,83 @@
1
- fluent-plugin-masking
1
+ # fluent-plugin-masking
2
+
3
+ ## Overview
4
+ Fluentd filter plugin to mask sensitive or privacy records with `*******` in place of the original value. This data masking plugin protects data such as name, email, phonenumber, address, and any other field you would like to mask.
5
+
6
+ ## Requirements
7
+ | fluent-plugin-masking | fluentd | ruby |
8
+ | --------------------- | ---------- | ------ |
9
+ | 1.0.x | >= v0.14.0 | >= 2.1 |
10
+
11
+
12
+ ## Installation
13
+ Install with gem:
14
+
15
+ `gem install fluent-plugin-masking`
16
+
17
+ ## Setup
18
+ In order to setup this plugin, the parameter `fieldsToMaskFilePath` needs to be a valid path to a file containing a list of all the fields to mask. The file should have a unique field on each line. These fields **are** case-sensitive (`Name` != `name`).
19
+
20
+ This is configured as shown below:
21
+ ```
22
+ <filter "**">
23
+ @type masking
24
+ fieldsToMaskFilePath "/path/to/fields-to-mask-file"
25
+ </filter>
26
+ ```
27
+
28
+ Example fields-to-mask-file:
29
+ ```
30
+ name
31
+ email
32
+ phone
33
+ ```
34
+
35
+
36
+ ## Quick Guide
37
+
38
+ ### Configuration:
39
+ ```
40
+ # fluent.conf
41
+ ----------------------------------
42
+ <source>
43
+ @type tail
44
+ path /tmp/test.log
45
+ pos_file /tmp/test.log.pos
46
+ tag maskme
47
+ format none
48
+ </source>
49
+
50
+ <filter "**">
51
+ @type masking
52
+ fieldsToMaskFilePath "/path/to/fields-to-mask-file"
53
+ </filter>
54
+
55
+ <match "**">
56
+ @type stdout
57
+ </match>
58
+
59
+
60
+
61
+ # /path/to/fields-to-mask-file
62
+ ----------------------------------
63
+ first_name
64
+ last_name
65
+ address
66
+ phone
67
+ password
68
+ email
69
+ ```
70
+
71
+ ### Result
72
+
73
+ To run the above configuration, run the following commands:
74
+ ```
75
+ fluentd -c fluent.conf
76
+ echo '{ :body => "{\"first_name\":\"mickey\", \"type\":\"puggle\", \"last_name\":\"the-dog\", \"password\":\"d0g43u39\"}"}' > /tmp/test.log
77
+ ```
78
+
79
+ This sample result is created from the above configuration file `fluent.conf`. As expected, the following fields configured to be masked are masked with `*******` in the output.
80
+
81
+ ```
82
+ 2019-09-15 16:12:50.359191000 +0300 maskme: {"message":"{ :body => \"{\\\"first_name\\\":\\\"*******\\\", \\\"type\\\":\\\"puggle\\\", \\\"last_name\\\":\\\"*******\\\", \\\"password\\\":\\\"*******\\\"}\"}"}
83
+ ```
@@ -9,9 +9,9 @@ Gem::Specification.new do |spec|
9
9
 
10
10
  spec.authors = ["Shai Moria", "Niv Lipetz"]
11
11
  spec.email = ["shai.moria@zooz.com", "niv.lipetz@zooz.com"]
12
- spec.description = "Fluentd Filter plugin to mask given fields in messages"
13
- spec.summary = "Fluentd Filter plugin to mask given fields in messages"
14
- spec.homepage = "https://github.com/zooz"
12
+ spec.description = "Fluentd filter plugin to mask sensitive or privacy records in event messages"
13
+ spec.summary = "Fluentd filter plugin to mask sensitive or privacy records with `*******` in place of the original value. This data masking plugin protects data such as name, email, phonenumber, address, and any other field you would like to mask."
14
+ spec.homepage = "https://github.com/zooz/fluent-plugin-masking"
15
15
 
16
16
  spec.files = `git ls-files`.split($\)
17
17
  spec.require_paths = ["lib"]
@@ -19,7 +19,7 @@ Gem::Specification.new do |spec|
19
19
 
20
20
  spec.required_ruby_version = '>= 2.1'
21
21
 
22
- spec.add_runtime_dependency "fluentd", ">= 0.14.0", "< 2"
22
+ spec.add_runtime_dependency "fluentd", ">= 0.14.0"
23
23
  spec.add_development_dependency "bundler"
24
24
  spec.add_development_dependency "rake", "~> 12.0"
25
25
  spec.add_development_dependency "test-unit", ">= 3.1.0"
@@ -12,16 +12,14 @@ module Fluent
12
12
  end
13
13
 
14
14
  # returns the masked record
15
- # error safe method - if any error occurs
16
- # the original record is return
15
+ # error safe method - if any error occurs the original record is returned
17
16
  def maskRecord(record)
18
17
  maskedRecord = record
19
18
 
20
19
  begin
21
20
  recordStr = record.to_s
22
- @fields_to_mask.each do | fieldToMask |
23
- recordStr = recordStr.gsub(/(?::#{fieldToMask}=>")(.*?)(?:")/m, ":#{fieldToMask}=>\"#{MASK_STRING}\"") # mask element in hash object
24
- recordStr = recordStr.gsub(/(\\+)"#{fieldToMask}\\+":\\+.+?((?=(})|,( *|)(\s|\\+)\")|(?=}"$))/m, "\\1\"#{fieldToMask}\\1\":\\1\"#{MASK_STRING}\\1\"") # mask element in json string
21
+ @fields_to_mask_regex.each do | fieldToMaskRegex, fieldToMaskRegexStringReplacement |
22
+ recordStr = recordStr.gsub(fieldToMaskRegex, fieldToMaskRegexStringReplacement)
25
23
  end
26
24
 
27
25
  maskedRecord = strToHash(recordStr)
@@ -36,6 +34,7 @@ module Fluent
36
34
  def initialize
37
35
  super
38
36
  @fields_to_mask = []
37
+ @fields_to_mask_regex = {}
39
38
  end
40
39
 
41
40
  # this method only called ones (on startup time)
@@ -51,6 +50,14 @@ module Fluent
51
50
  value = value.gsub('\n', '') # remove line breakers
52
51
 
53
52
  @fields_to_mask.push(value)
53
+
54
+ hashObjectRegex = Regexp.new(/(?::#{value}=>")(.*?)(?:")/m) # mask element in hash object
55
+ hashObjectRegexStringReplacement = ":#{value}=>\"#{MASK_STRING}\""
56
+ @fields_to_mask_regex[hashObjectRegex] = hashObjectRegexStringReplacement
57
+
58
+ innerJSONStringRegex = Regexp.new(/(\\+)"#{value}\\+":\\+.+?((?=(})|,( *|)(\s|\\+)\")|(?=}"$))/m) # mask element in json string using capture groups that count the level of escaping inside the json string
59
+ innerJSONStringRegexStringReplacement = "\\1\"#{value}\\1\":\\1\"#{MASK_STRING}\\1\""
60
+ @fields_to_mask_regex[innerJSONStringRegex] = innerJSONStringRegexStringReplacement
54
61
  end
55
62
  end
56
63
 
@@ -1,3 +1,3 @@
1
1
  module FilterMasking
2
- VERSION = "1.0.6"
2
+ VERSION = "1.0.7"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-masking
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.6
4
+ version: 1.0.7
5
5
  platform: ruby
6
6
  authors:
7
7
  - Shai Moria
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2019-08-21 00:00:00.000000000 Z
12
+ date: 2019-09-18 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: fluentd
@@ -18,9 +18,6 @@ dependencies:
18
18
  - - ">="
19
19
  - !ruby/object:Gem::Version
20
20
  version: 0.14.0
21
- - - "<"
22
- - !ruby/object:Gem::Version
23
- version: '2'
24
21
  type: :runtime
25
22
  prerelease: false
26
23
  version_requirements: !ruby/object:Gem::Requirement
@@ -28,9 +25,6 @@ dependencies:
28
25
  - - ">="
29
26
  - !ruby/object:Gem::Version
30
27
  version: 0.14.0
31
- - - "<"
32
- - !ruby/object:Gem::Version
33
- version: '2'
34
28
  - !ruby/object:Gem::Dependency
35
29
  name: bundler
36
30
  requirement: !ruby/object:Gem::Requirement
@@ -87,7 +81,7 @@ dependencies:
87
81
  - - ">="
88
82
  - !ruby/object:Gem::Version
89
83
  version: '0'
90
- description: Fluentd Filter plugin to mask given fields in messages
84
+ description: Fluentd filter plugin to mask sensitive or privacy records in event messages
91
85
  email:
92
86
  - shai.moria@zooz.com
93
87
  - niv.lipetz@zooz.com
@@ -106,7 +100,7 @@ files:
106
100
  - lib/fluent/plugin/version.rb
107
101
  - test/fields-to-mask
108
102
  - test/test_filter_masking.rb
109
- homepage: https://github.com/zooz
103
+ homepage: https://github.com/zooz/fluent-plugin-masking
110
104
  licenses:
111
105
  - Apache-2.0
112
106
  metadata: {}
@@ -128,5 +122,7 @@ requirements: []
128
122
  rubygems_version: 3.0.3
129
123
  signing_key:
130
124
  specification_version: 4
131
- summary: Fluentd Filter plugin to mask given fields in messages
125
+ summary: Fluentd filter plugin to mask sensitive or privacy records with `*******`
126
+ in place of the original value. This data masking plugin protects data such as name,
127
+ email, phonenumber, address, and any other field you would like to mask.
132
128
  test_files: []