blouson 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 82794d470c5403b89c887eb9f7edbbafcc9c3ed8
4
+ data.tar.gz: e49d57431f07c117d82c11937c8b3ae54130ec5e
5
+ SHA512:
6
+ metadata.gz: d4f78d3151d899f6cba3bf747a97ba7594e37234039a90257f45b47d6056e09192275edbc3e3b05b8d851a183465b1b23f52260188ddb3e88c097fcd2872afc5
7
+ data.tar.gz: e3c9c7621074f64028859a09f017d1dff0e53f26a350920beebb4ed17fc2d8dbde444e2fc98c09168968abc4f8e01034a88845ce8f7411abb19ab9c9f096afd4
data/.gitignore ADDED
@@ -0,0 +1,12 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /Gemfile.lock
4
+ /_yardoc/
5
+ /coverage/
6
+ /doc/
7
+ /pkg/
8
+ /spec/reports/
9
+ /tmp/
10
+
11
+ # rspec failure tracking
12
+ .rspec_status
data/.rspec ADDED
@@ -0,0 +1,2 @@
1
+ --format documentation
2
+ --color
data/.travis.yml ADDED
@@ -0,0 +1,7 @@
1
+ sudo: false
2
+ language: ruby
3
+ rvm:
4
+ - 2.4.0
5
+ - 2.3.3
6
+ - 2.2.6
7
+ before_install: gem install bundler -v 1.14.6
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in blouson.gemspec
4
+ gemspec
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2017 Cookpad Inc.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,88 @@
1
+ # Blouson
2
+
3
+ Blouson is a filter tool for Rails to conceal sensitive data from various logs.
4
+
5
+ - HTTP Request parameters in Rails log
6
+ - SQL query in Rails log
7
+ - Exception messages in `ActiveRecord::StatementInvalid`
8
+ - Sentry Raven parameters
9
+
10
+ ## Installation
11
+
12
+ Add this line to your application's Gemfile:
13
+
14
+ ```ruby
15
+ gem 'blouson'
16
+ ```
17
+
18
+ And then execute:
19
+
20
+ $ bundle
21
+
22
+ Or install it yourself as:
23
+
24
+ $ gem install blouson
25
+
26
+ ## Usage
27
+
28
+ ### SensitiveParamsSilencer
29
+ If there is a HTTP request parameter prefixed with ```secure_```, Blouson conceals sensitive data from logging.
30
+ Blouson enables this filter automatically.
31
+
32
+ Example:
33
+ ```
34
+ Started PUT "/employees/1" for 127.0.0.1 at Tue Jan 1 00:00:00 +0900 2013
35
+ Processing by EmployeesController#update as HTML
36
+ Parameters: {"commit"=>"Update Employee", "id"=>"1", "employee"=>{"name"=>"", "secure_personal_information"=>"[FILTERED]"}, "utf8"=>"✓"}
37
+ [Blouson::SensitiveParamsSilencer] SQL Log is skipped for sensitive data
38
+ ```
39
+
40
+ ### SensitiveQueryFilter
41
+ If there is a table prefixed with `secure_`, in exception message of `ActiveRecord::StatementInvalid`, Blouson conceals sensitive data from exception messages.
42
+ Blouson enables this filter automatically.
43
+
44
+ Example:
45
+
46
+ ```
47
+ RuntimeError: error: SELECT `secure_users`.* FROM `secure_users` WHERE `secure_users`.`email` = '[FILTERED]' ORDER BY `secure_users`.`id` ASC LIMIT 1
48
+ ```
49
+
50
+ ### SensitiveTableQueryLogSilencer
51
+ Blouson provides an [Arproxy](https://github.com/cookpad/arproxy) module to suppress query logs for secure_ prefix tables. If there is a query log for `secure_` prefix table, Blouson conceals it.
52
+ This proxy does not works automatically, so that you have to set `Blouson::SensitiveTableQueryLogSilencer` in your Arproxy initializer.
53
+
54
+ ```ruby
55
+ require 'blouson/sensitive_table_query_log_silencer'
56
+ # your initializers
57
+
58
+ Arproxy.configure do |config|
59
+ config.adapter = "mysql2"
60
+ config.use Blouson::SensitiveTableQueryLogSilencer
61
+ end
62
+ Arproxy.enable!
63
+ ```
64
+
65
+ ### RavenParameterFilterProcessor
66
+ Blouson provides an [Raven-Ruby](https://github.com/getsentry/raven-ruby) processor to conceal sensitive data from query string, request body, request headers and cookie values.
67
+
68
+ ```ruby
69
+ require 'blouson/raven_parameter_filter_processor'
70
+
71
+ filter_pattern = Rails.application.config.filter_parameters
72
+ secure_headers = %w(secret_token)
73
+
74
+ Raven.configure do |config|
75
+ ...
76
+ config.processors = [Blouson::RavenParameterFilterProcessor.create(filter_pattern, secure_headers)]
77
+ ...
78
+ end
79
+ ```
80
+
81
+ ## Contributing
82
+
83
+ Bug reports and pull requests are welcome on GitHub at https://github.com/cookpad/blouson.
84
+
85
+
86
+ ## License
87
+
88
+ The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,6 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
data/bin/console ADDED
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "blouson"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ require "irb"
14
+ IRB.start(__FILE__)
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
data/blouson.gemspec ADDED
@@ -0,0 +1,34 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'blouson/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "blouson"
8
+ spec.version = Blouson::VERSION
9
+ spec.authors = ["Cookpad Inc."]
10
+ spec.email = ["kaihatsu@cookpad.com"]
11
+
12
+ spec.summary = %q{Filter tools to mask sensitive data in various logs}
13
+ spec.description = spec.summary
14
+ spec.homepage = "https://github.com/cookpad/blouson"
15
+ spec.license = "MIT"
16
+
17
+ spec.files = `git ls-files -z`.split("\x0").reject do |f|
18
+ f.match(%r{^(test|spec|features)/})
19
+ end
20
+ spec.bindir = "exe"
21
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
22
+ spec.require_paths = ["lib"]
23
+
24
+ spec.add_dependency 'rails', '>= 4.0.0'
25
+ spec.add_dependency 'sentry-raven'
26
+
27
+ spec.add_development_dependency 'arproxy'
28
+ spec.add_development_dependency 'mysql2'
29
+ spec.add_development_dependency 'pry'
30
+
31
+ spec.add_development_dependency "bundler", ">= 1.14"
32
+ spec.add_development_dependency "rake", ">= 10.0"
33
+ spec.add_development_dependency "rspec", ">= 3.0"
34
+ end
@@ -0,0 +1,26 @@
1
+ module Blouson
2
+ class Engine < Rails::Engine
3
+ initializer 'blouson.configure_rails_initialization' do |app|
4
+ app.config.filter_parameters << Blouson::SENSITIVE_PARAMS_REGEXP
5
+ end
6
+
7
+ # We have to prevent logging sensitive data in SQL if production mode and logger level is debug
8
+ initializer 'blouson.load_helpers' do |app|
9
+ if !Rails.env.development? && Rails.logger.level == Logger::DEBUG
10
+ ActiveSupport.on_load(:action_controller) do
11
+ around_action Blouson::SensitiveParamsSilencer
12
+ end
13
+ end
14
+ end
15
+
16
+ initializer 'blouson.set_sensitive_query_filter' do
17
+ if Rails.env.production? || Rails.env.staging? || ENV['ENABLE_SENSITIVE_QUERY_FILTER'] == '1'
18
+ ActiveSupport.on_load(:active_record) do
19
+ ActiveRecord::StatementInvalid.class_eval do
20
+ prepend Blouson::SensitiveQueryFilter::StatementInvalidErrorFilter
21
+ end
22
+ end
23
+ end
24
+ end
25
+ end
26
+ end
@@ -0,0 +1,78 @@
1
+ module Blouson
2
+ class RavenParameterFilterProcessor
3
+ def self.create(filters, header_filters)
4
+ Class.new(Raven::Processor) do
5
+ @filters = filters
6
+ @header_filters = header_filters
7
+
8
+ def self.filters
9
+ @filters
10
+ end
11
+
12
+ def self.header_filters
13
+ @header_filters
14
+ end
15
+
16
+ def initialize(client = nil)
17
+ @parameter_filter = ActionDispatch::Http::ParameterFilter.new(self.class.filters)
18
+ end
19
+
20
+ def process(value)
21
+ process_query_string(value)
22
+ process_request_body(value)
23
+ process_request_header(value)
24
+ process_cookie(value)
25
+ ensure
26
+ return value
27
+ end
28
+
29
+ def process_request_body(value)
30
+ if value[:request] && value[:request][:data].present?
31
+ data = value[:request][:data]
32
+ if data.is_a?(String)
33
+ # Maybe JSON request
34
+ begin
35
+ data = JSON.parse(data)
36
+ value[:request][:data] = JSON.dump(@parameter_filter.filter(data))
37
+ rescue JSON::ParserError => e
38
+ # Record parser error to extra field
39
+ value[:extra]['BlousonError'] = e.message
40
+ end
41
+ else
42
+ value[:request][:data] = @parameter_filter.filter(data)
43
+ end
44
+ end
45
+ end
46
+
47
+ def process_query_string(value)
48
+ if value[:request] && value[:request][:query_string].present?
49
+ query = Rack::Utils.parse_query(value[:request][:query_string])
50
+ filtered = @parameter_filter.filter(query)
51
+
52
+ value[:request][:query_string] = Rack::Utils.build_query(filtered)
53
+ end
54
+ end
55
+
56
+ def process_request_header(value)
57
+ if value[:request] && value[:request][:headers]
58
+ headers = value[:request][:headers]
59
+ headers.each_key do |k|
60
+ if self.class.header_filters.include?(k.downcase)
61
+ headers[k] = 'FILTERED'
62
+ end
63
+ end
64
+ end
65
+ end
66
+
67
+ def process_cookie(value)
68
+ if value[:request] && value[:request][:headers] && value[:request][:headers]['Cookie']
69
+ cookies = Hash[value[:request][:headers]['Cookie'].split('; ').map { |pair| pair.split('=', 2) }]
70
+ filtered = @parameter_filter.filter(cookies)
71
+
72
+ value[:request][:headers]['Cookie'] = filtered.map { |pair| pair.join('=') }.join('; ')
73
+ end
74
+ end
75
+ end
76
+ end
77
+ end
78
+ end
@@ -0,0 +1,40 @@
1
+ module Blouson
2
+ class SensitiveParamsSilencer
3
+ class << self
4
+ def around(controller)
5
+ if include_sensitive_data?(controller)
6
+ begin
7
+ old_level = ActiveRecord::Base.logger.level
8
+ ActiveRecord::Base.logger.level = Logger::INFO
9
+ Rails.logger.info " [Blouson::SensitiveParamsSilencer] SQL Log is skipped for sensitive data"
10
+ yield
11
+ ensure
12
+ ActiveRecord::Base.logger.level = old_level
13
+ end
14
+ else
15
+ yield
16
+ end
17
+ end
18
+
19
+ def include_sensitive_data?(controller)
20
+ nested_params_keys(controller.params).any? { |key, value| Blouson::SENSITIVE_PARAMS_REGEXP === key }
21
+ end
22
+
23
+ private :include_sensitive_data?
24
+
25
+ def nested_params_keys(params)
26
+ if params.respond_to?(:to_unsafe_h)
27
+ params = params.to_unsafe_h
28
+ end
29
+ user_params = params.reject { |key, value| 'controller' == key || 'action' == key }
30
+ user_params.inject([]) do |keys, pair|
31
+ keys << pair.first
32
+ keys += pair.last.keys if pair.last.kind_of? Hash
33
+ keys
34
+ end
35
+ end
36
+
37
+ private :nested_params_keys
38
+ end
39
+ end
40
+ end
@@ -0,0 +1,39 @@
1
+ module Blouson
2
+ module SensitiveQueryFilter
3
+ QUOTED_WORD_REGEXP = /
4
+ (?: '.+?(?<!\\)'
5
+ | ".+?(?<!\\)"
6
+ )
7
+ /x
8
+
9
+ def self.contain_sensitive_query?(message)
10
+ Blouson::SENSITIVE_TABLE_REGEXP === message
11
+ end
12
+
13
+ def self.filter_sensitive_words(message)
14
+ message.gsub(QUOTED_WORD_REGEXP, "'#{Blouson::FILTERED}'")
15
+ end
16
+
17
+ module StatementInvalidErrorFilter
18
+ def initialize(message, original_exception = nil)
19
+ if SensitiveQueryFilter.contain_sensitive_query?(message)
20
+ message = SensitiveQueryFilter.filter_sensitive_words(message)
21
+ if defined?(Mysql2::Error)
22
+ if original_exception.is_a?(Mysql2::Error)
23
+ original_exception.extend(Mysql2Filter)
24
+ elsif $!.is_a?(Mysql2::Error)
25
+ $!.extend(Mysql2Filter)
26
+ end
27
+ end
28
+ end
29
+ super(message, original_exception)
30
+ end
31
+ end
32
+
33
+ module Mysql2Filter
34
+ def message
35
+ SensitiveQueryFilter.filter_sensitive_words(super)
36
+ end
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,17 @@
1
+ module Blouson
2
+ class SensitiveTableQueryLogSilencer < Arproxy::Base
3
+ def execute(sql, name=nil)
4
+ if Rails.logger.level != Logger::DEBUG || !(Blouson::SENSITIVE_TABLE_REGEXP === sql)
5
+ return super(sql, name)
6
+ end
7
+
8
+ begin
9
+ ActiveRecord::Base.logger.level = Logger::INFO
10
+ Rails.logger.info " [Blouson::SensitiveTableQueryLogSilencer] SQL Log is skipped for sensitive table"
11
+ super(sql, name)
12
+ ensure
13
+ ActiveRecord::Base.logger.level = Logger::DEBUG
14
+ end
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,11 @@
1
+ module Blouson
2
+ class TolerantRegexp < Regexp
3
+ def =~(str)
4
+ if str.respond_to?(:valid_encoding?) && !str.valid_encoding?
5
+ nil
6
+ else
7
+ super
8
+ end
9
+ end
10
+ end
11
+ end
@@ -0,0 +1,3 @@
1
+ module Blouson
2
+ VERSION = "1.0.0"
3
+ end
data/lib/blouson.rb ADDED
@@ -0,0 +1,12 @@
1
+ require "blouson/version"
2
+
3
+ require 'blouson/sensitive_params_silener'
4
+ require 'blouson/sensitive_query_filter'
5
+ require 'blouson/engine'
6
+ require 'blouson/tolerant_regexp'
7
+
8
+ module Blouson
9
+ SENSITIVE_PARAMS_REGEXP = TolerantRegexp.new('\Asecure_').freeze
10
+ SENSITIVE_TABLE_REGEXP = /secure_/.freeze
11
+ FILTERED = '[FILTERED]'.freeze
12
+ end
metadata ADDED
@@ -0,0 +1,174 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: blouson
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - Cookpad Inc.
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2017-03-16 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: rails
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: 4.0.0
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: 4.0.0
27
+ - !ruby/object:Gem::Dependency
28
+ name: sentry-raven
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: arproxy
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: mysql2
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: pry
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: bundler
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '1.14'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '1.14'
97
+ - !ruby/object:Gem::Dependency
98
+ name: rake
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '10.0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '10.0'
111
+ - !ruby/object:Gem::Dependency
112
+ name: rspec
113
+ requirement: !ruby/object:Gem::Requirement
114
+ requirements:
115
+ - - ">="
116
+ - !ruby/object:Gem::Version
117
+ version: '3.0'
118
+ type: :development
119
+ prerelease: false
120
+ version_requirements: !ruby/object:Gem::Requirement
121
+ requirements:
122
+ - - ">="
123
+ - !ruby/object:Gem::Version
124
+ version: '3.0'
125
+ description: Filter tools to mask sensitive data in various logs
126
+ email:
127
+ - kaihatsu@cookpad.com
128
+ executables: []
129
+ extensions: []
130
+ extra_rdoc_files: []
131
+ files:
132
+ - ".gitignore"
133
+ - ".rspec"
134
+ - ".travis.yml"
135
+ - Gemfile
136
+ - LICENSE.txt
137
+ - README.md
138
+ - Rakefile
139
+ - bin/console
140
+ - bin/setup
141
+ - blouson.gemspec
142
+ - lib/blouson.rb
143
+ - lib/blouson/engine.rb
144
+ - lib/blouson/raven_parameter_filter_processor.rb
145
+ - lib/blouson/sensitive_params_silener.rb
146
+ - lib/blouson/sensitive_query_filter.rb
147
+ - lib/blouson/sensitive_table_query_log_silencer.rb
148
+ - lib/blouson/tolerant_regexp.rb
149
+ - lib/blouson/version.rb
150
+ homepage: https://github.com/cookpad/blouson
151
+ licenses:
152
+ - MIT
153
+ metadata: {}
154
+ post_install_message:
155
+ rdoc_options: []
156
+ require_paths:
157
+ - lib
158
+ required_ruby_version: !ruby/object:Gem::Requirement
159
+ requirements:
160
+ - - ">="
161
+ - !ruby/object:Gem::Version
162
+ version: '0'
163
+ required_rubygems_version: !ruby/object:Gem::Requirement
164
+ requirements:
165
+ - - ">="
166
+ - !ruby/object:Gem::Version
167
+ version: '0'
168
+ requirements: []
169
+ rubyforge_project:
170
+ rubygems_version: 2.6.10
171
+ signing_key:
172
+ specification_version: 4
173
+ summary: Filter tools to mask sensitive data in various logs
174
+ test_files: []