logstash-filter-cipher_kms 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/CHANGELOG.md +2 -0
- data/CONTRIBUTORS +10 -0
- data/DEVELOPER.md +2 -0
- data/Gemfile +3 -0
- data/LICENSE +11 -0
- data/README.md +86 -0
- data/lib/logstash/filters/cipher_kms.rb +281 -0
- data/logstash-filter-cipher_kms.gemspec +32 -0
- data/spec/filters/cipher_kms_spec.rb +153 -0
- data/spec/spec_helper.rb +9 -0
- metadata +143 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 120788fc1c7dfcf06d11e79d904c069b42bdca37478818ea2543e0c5ef4aa2e4
|
4
|
+
data.tar.gz: 9bdd3463342e201bd16aad72ad5b51d227408dd410a99d46139eabd599e80385
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 2066e76d16630aed26d9acb86bebd281b70b1d35d3e9bf9ab000a4000b72d0f9f1e8d4a85c9c9e0181a13d325f2fb16445bac0c2ad4a792659dc7976d5024528
|
7
|
+
data.tar.gz: 2e56c12977c50929600d13131a622987f8401a60eb9e35635e0dc3b9e2e41e786ef50d463c231d88239385d519327ba84d97c6cd657a74855f7dff0167e3831c
|
data/CHANGELOG.md
ADDED
data/CONTRIBUTORS
ADDED
@@ -0,0 +1,10 @@
|
|
1
|
+
The following is a list of people who have contributed ideas, code, bug
|
2
|
+
reports, or in general have helped logstash along its way.
|
3
|
+
|
4
|
+
Contributors:
|
5
|
+
* Avihoo Mamka - avihoo.mamka@onfido.com
|
6
|
+
|
7
|
+
Note: If you've sent us patches, bug reports, or otherwise contributed to
|
8
|
+
Logstash, and you aren't on the list above and want to be, please let us know
|
9
|
+
and we'll make sure you're here. Contributions from folks like you are what make
|
10
|
+
open source awesome.
|
data/DEVELOPER.md
ADDED
data/Gemfile
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,11 @@
|
|
1
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
2
|
+
you may not use this file except in compliance with the License.
|
3
|
+
You may obtain a copy of the License at
|
4
|
+
|
5
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
6
|
+
|
7
|
+
Unless required by applicable law or agreed to in writing, software
|
8
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
9
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
10
|
+
See the License for the specific language governing permissions and
|
11
|
+
limitations under the License.
|
data/README.md
ADDED
@@ -0,0 +1,86 @@
|
|
1
|
+
# Logstash Plugin
|
2
|
+
|
3
|
+
This is a plugin for [Logstash](https://github.com/elastic/logstash).
|
4
|
+
|
5
|
+
It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
|
6
|
+
|
7
|
+
## Documentation
|
8
|
+
|
9
|
+
Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elastic.co/guide/en/logstash/current/).
|
10
|
+
|
11
|
+
- For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
|
12
|
+
- For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide
|
13
|
+
|
14
|
+
## Need Help?
|
15
|
+
|
16
|
+
Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
|
17
|
+
|
18
|
+
## Developing
|
19
|
+
|
20
|
+
### 1. Plugin Developement and Testing
|
21
|
+
|
22
|
+
#### Code
|
23
|
+
- To get started, you'll need JRuby with the Bundler gem installed.
|
24
|
+
|
25
|
+
- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
|
26
|
+
|
27
|
+
- Install dependencies
|
28
|
+
```sh
|
29
|
+
bundle install
|
30
|
+
```
|
31
|
+
|
32
|
+
#### Test
|
33
|
+
|
34
|
+
- Update your dependencies
|
35
|
+
|
36
|
+
```sh
|
37
|
+
bundle install
|
38
|
+
```
|
39
|
+
|
40
|
+
- Run tests
|
41
|
+
|
42
|
+
```sh
|
43
|
+
bundle exec rspec
|
44
|
+
```
|
45
|
+
|
46
|
+
### 2. Running your unpublished Plugin in Logstash
|
47
|
+
|
48
|
+
#### 2.1 Run in a local Logstash clone
|
49
|
+
|
50
|
+
- Edit Logstash `Gemfile` and add the local plugin path, for example:
|
51
|
+
```ruby
|
52
|
+
gem "logstash-filter-cipher_kms", :path => "/your/local/logstash-filter-cipher_kms"
|
53
|
+
```
|
54
|
+
- Install plugin
|
55
|
+
```sh
|
56
|
+
bin/logstash-plugin install --no-verify
|
57
|
+
```
|
58
|
+
- Run Logstash with your plugin
|
59
|
+
```sh
|
60
|
+
bin/logstash -e 'filter {cipher_kms {}}'
|
61
|
+
```
|
62
|
+
At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
|
63
|
+
|
64
|
+
#### 2.2 Run in an installed Logstash
|
65
|
+
|
66
|
+
You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
|
67
|
+
|
68
|
+
- Build your plugin gem
|
69
|
+
```sh
|
70
|
+
gem build logstash-filter-cipher_kms.gemspec
|
71
|
+
```
|
72
|
+
- Install the plugin from the Logstash home
|
73
|
+
```sh
|
74
|
+
bin/logstash-plugin install /your/local/plugin/logstash-filter-cipher_kms.gem
|
75
|
+
```
|
76
|
+
- Start Logstash and proceed to test the plugin
|
77
|
+
|
78
|
+
## Contributing
|
79
|
+
|
80
|
+
All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
|
81
|
+
|
82
|
+
Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
|
83
|
+
|
84
|
+
It is more important to the community that you are able to contribute.
|
85
|
+
|
86
|
+
For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.
|
@@ -0,0 +1,281 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
require 'active_support/core_ext'
|
4
|
+
require 'aws-sdk'
|
5
|
+
require 'rubygems'
|
6
|
+
require 'json'
|
7
|
+
require 'logstash/filters/base'
|
8
|
+
require 'logstash/namespace'
|
9
|
+
require 'openssl'
|
10
|
+
|
11
|
+
# This filter parses a source and apply a cipher or decipher before
|
12
|
+
# storing it in the target.
|
13
|
+
# It uses AWS KMS to generate the envelope key for ciphering/deciphering.
|
14
|
+
class LogStash::Filters::CipherKms < LogStash::Filters::Base
|
15
|
+
config_name 'cipher_kms'
|
16
|
+
|
17
|
+
# The field to perform filter
|
18
|
+
#
|
19
|
+
# Example, to use the @message field (default):
|
20
|
+
# [source,ruby]
|
21
|
+
# filter { cipher_kms { source => "message" } }
|
22
|
+
config :source, validate: :string, default: 'message'
|
23
|
+
|
24
|
+
# The name of the container to put the result
|
25
|
+
#
|
26
|
+
# Example, to place the result into crypt :
|
27
|
+
# [source,ruby]
|
28
|
+
# filter { cipher_kms { target => "crypt" } }
|
29
|
+
config :target, validate: :string, default: 'message'
|
30
|
+
|
31
|
+
# The name of the container to put the crypt metadata
|
32
|
+
#
|
33
|
+
# Example, to place the crypt metadata into crypt_metadata :
|
34
|
+
# [source,ruby]
|
35
|
+
# filter { cipher_kms { crypt_metadata => "crypt_metadata" } }
|
36
|
+
config :crypt_metadata, validate: :string, default: 'crypt_metadata'
|
37
|
+
|
38
|
+
# Do we have to perform a `base64` decode or encode?
|
39
|
+
#
|
40
|
+
# If we are decrypting, `base64` decode will be done before.
|
41
|
+
# If we are encrypting, `base64` will be done after.
|
42
|
+
#
|
43
|
+
config :base64, validate: :boolean, default: true
|
44
|
+
|
45
|
+
# The AWS KMS key id to use
|
46
|
+
#
|
47
|
+
config :key_id, validate: :string, required: true
|
48
|
+
|
49
|
+
# The AWS region for KMS (e.g. eu-west-1)
|
50
|
+
# See http://docs.aws.amazon.com/general/latest/gr/rande.html#kms_region
|
51
|
+
config :region, validate: :string, required: true
|
52
|
+
|
53
|
+
# An optional aws access key id to use for AWS.
|
54
|
+
#
|
55
|
+
config :access_key_id, validate: :string, default: nil
|
56
|
+
|
57
|
+
# An optional secret access key to use for AWS.
|
58
|
+
#
|
59
|
+
config :secret_access_key, validate: :string, default: nil
|
60
|
+
|
61
|
+
# An optional AWS profile to use.
|
62
|
+
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-multiple-profiles
|
63
|
+
config :aws_profile, validate: :string, default: nil
|
64
|
+
|
65
|
+
# An optional path for the shared credentials file.
|
66
|
+
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
|
67
|
+
config :aws_shared_credentials_path, validate: :string, default: nil
|
68
|
+
|
69
|
+
# An optional setting whether to use AWS instance profile.
|
70
|
+
# See http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html
|
71
|
+
config :aws_instance_profile, validate: :boolean, default: false
|
72
|
+
|
73
|
+
# An optional setting whether to use AWS ECS credentials.
|
74
|
+
# See http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html
|
75
|
+
config :aws_ecs_credentials, validate: :boolean, default: false
|
76
|
+
|
77
|
+
# An encryption context to use to encrypting and decrypting data.
|
78
|
+
# When encrypting, the encryption context is sent as a part of the payload.
|
79
|
+
# When decrypting, if any of the key-value pairs specified in encryption_context
|
80
|
+
# fails to match the payload's encryption context, then decryption will fail.
|
81
|
+
# Nested hashes are not supported. This is just a flat map of key value pairs.
|
82
|
+
# See: http://docs.aws.amazon.com/kms/latest/developerguide/encryption-context.html
|
83
|
+
config :encryption_context, validate: :hash, default: {}
|
84
|
+
|
85
|
+
# The cipher algorithm
|
86
|
+
#
|
87
|
+
# Due to AWS KMS restrictions the available algorithms are: AES_128, AES_256
|
88
|
+
# The mapping for the equivalent Ruby ciphers are as follows:
|
89
|
+
# {AES_128 => AES-128-cbc, AES_256 => AES-256-cbc}
|
90
|
+
config :algorithm, validate: :string, required: true
|
91
|
+
|
92
|
+
# Encrypting or decrypting some data
|
93
|
+
#
|
94
|
+
# Valid values are encrypt or decrypt
|
95
|
+
config :mode, validate: :string, required: true
|
96
|
+
|
97
|
+
# Force a random IV to be used per encryption invocation and specify
|
98
|
+
# the length of the random IV that will be generated via:
|
99
|
+
#
|
100
|
+
# OpenSSL::Random.random_bytes(int_length)
|
101
|
+
#
|
102
|
+
# Enabling this will force the plugin to generate a unique
|
103
|
+
# random IV for each encryption call. This random IV will be prepended to the
|
104
|
+
# encrypted result bytes and then base64 encoded.
|
105
|
+
# On decryption "iv_random_length" must also be set to utilize this feature.
|
106
|
+
# Random IV's are better than statically hardcoded IVs.
|
107
|
+
#
|
108
|
+
# For AES algorithms you can set this to a 16
|
109
|
+
# [source,ruby]
|
110
|
+
# filter { cipher { iv_random_length => 16 }}
|
111
|
+
config :iv_random_length, validate: :number, required: true
|
112
|
+
|
113
|
+
# If this is set, the internal Cipher instance will be
|
114
|
+
# re-used up to @max_cipher_reuse times before being
|
115
|
+
# reset() and re-created from scratch. This is an option
|
116
|
+
# for efficiency where lots of data is being encrypted
|
117
|
+
# and decrypted using this filter. This lets the filter
|
118
|
+
# avoid creating new Cipher instances over and over
|
119
|
+
# for each encrypt/decrypt operation.
|
120
|
+
#
|
121
|
+
# This is optional, the default is no re-use of the Cipher
|
122
|
+
# instance and max_cipher_reuse = 1 by default
|
123
|
+
# [source,ruby]
|
124
|
+
# filter { cipher { max_cipher_reuse => 1000 }}
|
125
|
+
config :max_cipher_reuse, validate: :number, default: 1
|
126
|
+
|
127
|
+
# Mapping between AWS KMS available ciphers and correlated Ruby ciphers
|
128
|
+
KMS_RUBY_CIPHER_MAP = {"AES_128" => "AES-128-cbc", "AES_256" => "AES-256-cbc"}.freeze
|
129
|
+
|
130
|
+
def register
|
131
|
+
require 'base64' if @base64
|
132
|
+
validate_config
|
133
|
+
init_cipher
|
134
|
+
end
|
135
|
+
|
136
|
+
def filter(event)
|
137
|
+
# If decrypt or encrypt fails, we keep it it intact.
|
138
|
+
begin
|
139
|
+
if event.get(@source).blank?
|
140
|
+
@logger.debug("Event to filter, event 'source' field: " + @source + ' was nil or blank, doing nothing.')
|
141
|
+
return
|
142
|
+
end
|
143
|
+
|
144
|
+
@logger.debug('Event to filter', event: event)
|
145
|
+
data = event.get(@source)
|
146
|
+
|
147
|
+
if @mode == 'encrypt'
|
148
|
+
result, metadata = encrypt(data)
|
149
|
+
event.set(@crypt_metadata, metadata)
|
150
|
+
elsif @mode == 'decrypt'
|
151
|
+
result = decrypt(data, event.get(@crypt_metadata))
|
152
|
+
event.remove(@crypt_metadata)
|
153
|
+
end
|
154
|
+
|
155
|
+
@total_cipher_uses += 1
|
156
|
+
unless result.nil?
|
157
|
+
event.set(@target, result)
|
158
|
+
filter_matched(event)
|
159
|
+
end
|
160
|
+
rescue => e
|
161
|
+
@logger.warn('Exception caught on cipher filter', event: event, error: e)
|
162
|
+
# force a re-initialize on error to be safe
|
163
|
+
init_cipher
|
164
|
+
ensure
|
165
|
+
rotate_cipher_if_needed
|
166
|
+
end
|
167
|
+
end
|
168
|
+
|
169
|
+
def encrypt(data)
|
170
|
+
@random_iv = OpenSSL::Random.random_bytes(@iv_random_length)
|
171
|
+
kms_response = @kms.generate_data_key(key_id: @key_id, key_spec: @algorithm,
|
172
|
+
encryption_context: @encryption_context) # => returns a ciphertext and a plaintext key
|
173
|
+
|
174
|
+
begin
|
175
|
+
data = JSON.generate(data)
|
176
|
+
rescue JSON::GeneratorError
|
177
|
+
# ignored, use as is
|
178
|
+
end
|
179
|
+
result = cipher_process(kms_response.plaintext, data)
|
180
|
+
|
181
|
+
# Prepend padding and base64 encoding if configured
|
182
|
+
result = @random_iv + result unless @random_iv.nil?
|
183
|
+
result = Base64.strict_encode64(result).encode('utf-8') if @base64
|
184
|
+
metadata = Base64.strict_encode64(kms_response.ciphertext_blob).encode('utf-8')
|
185
|
+
[result, metadata]
|
186
|
+
end
|
187
|
+
|
188
|
+
def decrypt(data, metadata)
|
189
|
+
kms_response = @kms.decrypt(ciphertext_blob: Base64.strict_decode64(metadata),
|
190
|
+
encryption_context: @encryption_context)
|
191
|
+
|
192
|
+
data = Base64.strict_decode64(data) if @base64
|
193
|
+
@random_iv = data.byteslice(0, @iv_random_length)
|
194
|
+
data = data.byteslice(@iv_random_length..data.length)
|
195
|
+
|
196
|
+
result = cipher_process(kms_response.plaintext, data)
|
197
|
+
result.force_encoding('utf-8')
|
198
|
+
begin
|
199
|
+
result = JSON.parse(result)
|
200
|
+
rescue JSON::ParserError
|
201
|
+
# ignored, return as is
|
202
|
+
end
|
203
|
+
|
204
|
+
result
|
205
|
+
end
|
206
|
+
|
207
|
+
def cipher_set_required_params(key)
|
208
|
+
@cipher.key = key
|
209
|
+
@cipher.iv = @random_iv
|
210
|
+
end
|
211
|
+
|
212
|
+
def cipher_process(key, data)
|
213
|
+
cipher_set_required_params(key)
|
214
|
+
result = @cipher.update(data) + @cipher.final
|
215
|
+
result
|
216
|
+
end
|
217
|
+
|
218
|
+
def rotate_cipher_if_needed
|
219
|
+
if !@max_cipher_reuse.nil? && @total_cipher_uses >= @max_cipher_reuse
|
220
|
+
@logger.debug('max_cipher_reuse[' + @max_cipher_reuse.to_s + '] reached, total_cipher_uses = ' + @total_cipher_uses.to_s)
|
221
|
+
init_cipher
|
222
|
+
end
|
223
|
+
end
|
224
|
+
|
225
|
+
def init_cipher
|
226
|
+
@logger.debug('Encryption Context: ' + @encryption_context.to_s, plugin: self.class.name)
|
227
|
+
|
228
|
+
credentials = nil
|
229
|
+
if !@access_key_id.blank? && !@secret_access_key.blank?
|
230
|
+
credentials = Aws::Credentials.new(@access_key_id, @secret_access_key)
|
231
|
+
@logger.debug('Using Static Credentials', plugin: self.class.name)
|
232
|
+
elsif !@aws_shared_credentials_path.blank? || !@aws_profile.blank?
|
233
|
+
credentials = Aws::SharedCredentials.new(path: @aws_shared_credentials_path, profile_name: @aws_profile)
|
234
|
+
@logger.debug('Using Shared Credentials', plugin: self.class.name)
|
235
|
+
elsif @aws_instance_profile
|
236
|
+
credentials = Aws::InstanceProfileCredentials.new
|
237
|
+
@logger.debug('Using Instance Profile Credentials', plugin: self.class.name)
|
238
|
+
elsif @aws_ecs_credentials
|
239
|
+
credentials = Aws::ECSCredentials.new
|
240
|
+
@logger.debug('Using ECS Credentials', plugin: self.class.name)
|
241
|
+
end
|
242
|
+
|
243
|
+
@kms = Aws::KMS::Client.new(region: @region, credentials: credentials)
|
244
|
+
|
245
|
+
@total_cipher_uses = 0
|
246
|
+
@cipher = OpenSSL::Cipher.new(KMS_RUBY_CIPHER_MAP[@algorithm])
|
247
|
+
set_cipher_crypt_mode
|
248
|
+
|
249
|
+
@logger.debug('Cipher initialisation done', mode: @mode, iv_random_length: @iv_random_length,
|
250
|
+
algorithm: @algorithm, base64: @base64,
|
251
|
+
max_cipher_reuse: @max_cipher_reuse)
|
252
|
+
end
|
253
|
+
|
254
|
+
def set_cipher_crypt_mode
|
255
|
+
if @mode == 'encrypt'
|
256
|
+
@cipher.encrypt
|
257
|
+
elsif @mode == 'decrypt'
|
258
|
+
@cipher.decrypt
|
259
|
+
end
|
260
|
+
end
|
261
|
+
|
262
|
+
def validate_config
|
263
|
+
@encryption_context.each do |_, value|
|
264
|
+
unless value.is_a?(String)
|
265
|
+
raise LogStash::ConfigurationError, 'Values in encryption_context must be strings, aborting.'
|
266
|
+
end
|
267
|
+
end
|
268
|
+
|
269
|
+
unless KMS_RUBY_CIPHER_MAP.key?(@algorithm)
|
270
|
+
raise LogStash::ConfigurationError, 'You can only use one of the following algorithms: ' + KMS_RUBY_CIPHER_MAP.keys.to_s + ', aborting.'
|
271
|
+
end
|
272
|
+
|
273
|
+
unless @mode == 'encrypt' || @mode == 'decrypt'
|
274
|
+
@logger.error('Invalid cipher mode. Valid values are \"encrypt\" or \"decrypt\"', mode: @mode)
|
275
|
+
raise LogStash::ConfigurationError, 'Invalid cipher mode. Valid values are \"encrypt\" or \"decrypt\", aborting.'
|
276
|
+
end
|
277
|
+
|
278
|
+
true
|
279
|
+
end
|
280
|
+
|
281
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
Gem::Specification.new do |s|
|
4
|
+
s.name = 'logstash-filter-cipher_kms'
|
5
|
+
s.version = '0.1.0'
|
6
|
+
s.licenses = ['Apache License (2.0)']
|
7
|
+
s.summary = 'This is a Logstash plugin to allow data
|
8
|
+
encryption/decryption using AWS KMS.'
|
9
|
+
s.description = 'This is a Logstash plugin to allow data
|
10
|
+
encryption/decryption using AWS KMS.'
|
11
|
+
s.homepage = 'https://github.com/onfido/logstash-filter-cipher_kms'
|
12
|
+
s.authors = ['Onfido']
|
13
|
+
s.email = 'engineering@onfido.com'
|
14
|
+
s.require_paths = ['lib']
|
15
|
+
|
16
|
+
# Files
|
17
|
+
s.files = Dir['lib/**/*', 'spec/**/*', 'vendor/**/*', '*.gemspec', '*.md',
|
18
|
+
'CONTRIBUTORS', 'Gemfile', 'LICENSE', 'NOTICE.TXT']
|
19
|
+
# Tests
|
20
|
+
s.test_files = s.files.grep(%r{^(test|spec|features)/})
|
21
|
+
|
22
|
+
# Special flag to let us know this is actually a logstash plugin
|
23
|
+
s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
|
24
|
+
|
25
|
+
# Gem dependencies
|
26
|
+
s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
|
27
|
+
s.add_development_dependency 'logstash-devutils'
|
28
|
+
s.add_dependency('activesupport')
|
29
|
+
s.add_dependency('aws-sdk', '~> 2')
|
30
|
+
s.add_development_dependency('vcr', '~> 4.0.0')
|
31
|
+
s.add_development_dependency('webmock', '~> 3.1.1')
|
32
|
+
end
|
@@ -0,0 +1,153 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
require_relative '../spec_helper'
|
4
|
+
require 'logstash/filters/cipher_kms'
|
5
|
+
|
6
|
+
describe LogStash::Filters::CipherKms do
|
7
|
+
|
8
|
+
describe 'configuration validations' do
|
9
|
+
|
10
|
+
it 'should pass validation with encrypt mode' do
|
11
|
+
config = described_class.new(
|
12
|
+
'key_id' => 'id',
|
13
|
+
'region' => 'us-west-1',
|
14
|
+
'algorithm' => 'AES_128',
|
15
|
+
'iv_random_length' => 16,
|
16
|
+
'mode' => 'encrypt'
|
17
|
+
)
|
18
|
+
expect(config.validate_config).to eq(true)
|
19
|
+
end
|
20
|
+
|
21
|
+
it 'should pass validation with decrypt mode' do
|
22
|
+
config = described_class.new(
|
23
|
+
'key_id' => 'id',
|
24
|
+
'region' => 'us-west-1',
|
25
|
+
'algorithm' => 'AES_128',
|
26
|
+
'iv_random_length' => 16,
|
27
|
+
'mode' => 'decrypt'
|
28
|
+
)
|
29
|
+
expect(config.validate_config).to eq(true)
|
30
|
+
end
|
31
|
+
|
32
|
+
it 'should fail validation with invalid crypt mode' do
|
33
|
+
config = described_class.new(
|
34
|
+
'key_id' => 'id',
|
35
|
+
'region' => 'us-west-1',
|
36
|
+
'algorithm' => 'AES_128',
|
37
|
+
'iv_random_length' => 16,
|
38
|
+
'mode' => 'foo'
|
39
|
+
)
|
40
|
+
expect {config.validate_config}.to raise_error(LogStash::ConfigurationError)
|
41
|
+
end
|
42
|
+
|
43
|
+
it 'should pass validation with AES_128 algorithm' do
|
44
|
+
config = described_class.new(
|
45
|
+
'key_id' => 'id',
|
46
|
+
'region' => 'us-west-1',
|
47
|
+
'algorithm' => 'AES_128',
|
48
|
+
'iv_random_length' => 16,
|
49
|
+
'mode' => 'encrypt'
|
50
|
+
)
|
51
|
+
expect(config.validate_config).to eq(true)
|
52
|
+
end
|
53
|
+
|
54
|
+
it 'should pass validation with AES_256 algorithm' do
|
55
|
+
config = described_class.new(
|
56
|
+
'key_id' => 'id',
|
57
|
+
'region' => 'us-west-1',
|
58
|
+
'algorithm' => 'AES_256',
|
59
|
+
'iv_random_length' => 16,
|
60
|
+
'mode' => 'encrypt'
|
61
|
+
)
|
62
|
+
expect(config.validate_config).to eq(true)
|
63
|
+
end
|
64
|
+
|
65
|
+
it 'should fail validation with invalid algorithm' do
|
66
|
+
config = described_class.new(
|
67
|
+
'key_id' => 'id',
|
68
|
+
'region' => 'us-west-1',
|
69
|
+
'algorithm' => 'foo',
|
70
|
+
'iv_random_length' => 16,
|
71
|
+
'mode' => 'encrypt'
|
72
|
+
)
|
73
|
+
expect {config.validate_config}.to raise_error(LogStash::ConfigurationError)
|
74
|
+
end
|
75
|
+
|
76
|
+
it 'should pass validation with valid encryption context' do
|
77
|
+
config = described_class.new(
|
78
|
+
'key_id' => 'id',
|
79
|
+
'region' => 'us-west-1',
|
80
|
+
'algorithm' => 'AES_128',
|
81
|
+
'iv_random_length' => 16,
|
82
|
+
'mode' => 'encrypt',
|
83
|
+
'encryption_context' => {'foo' => 'bar'}
|
84
|
+
)
|
85
|
+
expect(config.validate_config).to eq(true)
|
86
|
+
end
|
87
|
+
|
88
|
+
it 'should pass validation with a missing encryption context param' do
|
89
|
+
config = described_class.new(
|
90
|
+
'key_id' => 'id',
|
91
|
+
'region' => 'us-west-1',
|
92
|
+
'algorithm' => 'AES_128',
|
93
|
+
'iv_random_length' => 16,
|
94
|
+
'mode' => 'encrypt'
|
95
|
+
)
|
96
|
+
expect(config.validate_config).to eq(true)
|
97
|
+
end
|
98
|
+
|
99
|
+
it 'should fail validation with an invalid encryption context' do
|
100
|
+
config = described_class.new(
|
101
|
+
'key_id' => 'id',
|
102
|
+
'region' => 'us-west-1',
|
103
|
+
'algorithm' => 'AES_128',
|
104
|
+
'iv_random_length' => 16,
|
105
|
+
'mode' => 'encrypt',
|
106
|
+
'encryption_context' => {'foo' => {'bar' => 'foobar'}}
|
107
|
+
)
|
108
|
+
expect {config.validate_config}.to raise_error(LogStash::ConfigurationError)
|
109
|
+
end
|
110
|
+
end
|
111
|
+
|
112
|
+
describe 'decryption' do
|
113
|
+
it 'returns a plain text version of the input' do
|
114
|
+
encrypter = described_class.new(
|
115
|
+
'algorithm' => 'AES_256',
|
116
|
+
'mode' => 'encrypt',
|
117
|
+
'base64' => true,
|
118
|
+
'key_id' => 'arn:aws:kms:eu-west-1:666666666666:alias/kms-key',
|
119
|
+
'iv_random_length' => 16,
|
120
|
+
'region' => 'eu-west-1',
|
121
|
+
'access_key_id' => 'fake_aws_key',
|
122
|
+
'secret_access_key' => 'fake_aws_secret_key',
|
123
|
+
'encryption_context' => {
|
124
|
+
'kms_cmk_id' => 'arn:aws:kms:eu-west-1:666666666666:alias/kms-key'
|
125
|
+
}
|
126
|
+
)
|
127
|
+
decrypter = described_class.new(
|
128
|
+
'algorithm' => 'AES_256',
|
129
|
+
'mode' => 'decrypt',
|
130
|
+
'base64' => true,
|
131
|
+
'key_id' => 'arn:aws:kms:eu-west-1:666666666666:alias/kms-key',
|
132
|
+
'iv_random_length' => 16,
|
133
|
+
'region' => 'eu-west-1',
|
134
|
+
'access_key_id' => 'fake_aws_key',
|
135
|
+
'secret_access_key' => 'fake_aws_secret_key',
|
136
|
+
'encryption_context' => {
|
137
|
+
'kms_cmk_id' => 'arn:aws:kms:eu-west-1:666666666666:alias/kms-key'
|
138
|
+
}
|
139
|
+
)
|
140
|
+
plain_text = 'foo'
|
141
|
+
event = LogStash::Event.new(LogStash::Json.load("{\"message\":\"#{plain_text}\"}"))
|
142
|
+
encrypter.register
|
143
|
+
decrypter.register
|
144
|
+
|
145
|
+
VCR.use_cassette('aws_kms_communication') do
|
146
|
+
encrypter.filter(event)
|
147
|
+
decrypter.filter(event)
|
148
|
+
end
|
149
|
+
|
150
|
+
expect(event.get('message')).to eq(plain_text)
|
151
|
+
end
|
152
|
+
end
|
153
|
+
end
|
data/spec/spec_helper.rb
ADDED
metadata
ADDED
@@ -0,0 +1,143 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: logstash-filter-cipher_kms
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Onfido
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2018-02-19 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
15
|
+
requirements:
|
16
|
+
- - "~>"
|
17
|
+
- !ruby/object:Gem::Version
|
18
|
+
version: '2.0'
|
19
|
+
name: logstash-core-plugin-api
|
20
|
+
prerelease: false
|
21
|
+
type: :runtime
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '2.0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
29
|
+
requirements:
|
30
|
+
- - ">="
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '0'
|
33
|
+
name: logstash-devutils
|
34
|
+
prerelease: false
|
35
|
+
type: :development
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
43
|
+
requirements:
|
44
|
+
- - ">="
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
version: '0'
|
47
|
+
name: activesupport
|
48
|
+
prerelease: false
|
49
|
+
type: :runtime
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
requirement: !ruby/object:Gem::Requirement
|
57
|
+
requirements:
|
58
|
+
- - "~>"
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: '2'
|
61
|
+
name: aws-sdk
|
62
|
+
prerelease: false
|
63
|
+
type: :runtime
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '2'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
requirement: !ruby/object:Gem::Requirement
|
71
|
+
requirements:
|
72
|
+
- - "~>"
|
73
|
+
- !ruby/object:Gem::Version
|
74
|
+
version: 4.0.0
|
75
|
+
name: vcr
|
76
|
+
prerelease: false
|
77
|
+
type: :development
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: 4.0.0
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
requirement: !ruby/object:Gem::Requirement
|
85
|
+
requirements:
|
86
|
+
- - "~>"
|
87
|
+
- !ruby/object:Gem::Version
|
88
|
+
version: 3.1.1
|
89
|
+
name: webmock
|
90
|
+
prerelease: false
|
91
|
+
type: :development
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - "~>"
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: 3.1.1
|
97
|
+
description: |-
|
98
|
+
This is a Logstash plugin to allow data
|
99
|
+
encryption/decryption using AWS KMS.
|
100
|
+
email: engineering@onfido.com
|
101
|
+
executables: []
|
102
|
+
extensions: []
|
103
|
+
extra_rdoc_files: []
|
104
|
+
files:
|
105
|
+
- CHANGELOG.md
|
106
|
+
- CONTRIBUTORS
|
107
|
+
- DEVELOPER.md
|
108
|
+
- Gemfile
|
109
|
+
- LICENSE
|
110
|
+
- README.md
|
111
|
+
- lib/logstash/filters/cipher_kms.rb
|
112
|
+
- logstash-filter-cipher_kms.gemspec
|
113
|
+
- spec/filters/cipher_kms_spec.rb
|
114
|
+
- spec/spec_helper.rb
|
115
|
+
homepage: https://github.com/onfido/logstash-filter-cipher_kms
|
116
|
+
licenses:
|
117
|
+
- Apache License (2.0)
|
118
|
+
metadata:
|
119
|
+
logstash_plugin: 'true'
|
120
|
+
logstash_group: filter
|
121
|
+
post_install_message:
|
122
|
+
rdoc_options: []
|
123
|
+
require_paths:
|
124
|
+
- lib
|
125
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
126
|
+
requirements:
|
127
|
+
- - ">="
|
128
|
+
- !ruby/object:Gem::Version
|
129
|
+
version: '0'
|
130
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
131
|
+
requirements:
|
132
|
+
- - ">="
|
133
|
+
- !ruby/object:Gem::Version
|
134
|
+
version: '0'
|
135
|
+
requirements: []
|
136
|
+
rubyforge_project:
|
137
|
+
rubygems_version: 2.6.13
|
138
|
+
signing_key:
|
139
|
+
specification_version: 4
|
140
|
+
summary: This is a Logstash plugin to allow data encryption/decryption using AWS KMS.
|
141
|
+
test_files:
|
142
|
+
- spec/filters/cipher_kms_spec.rb
|
143
|
+
- spec/spec_helper.rb
|