pii_safe_schema 1.1.0 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (10) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +11 -0
  3. data/README.md +56 -12
  4. data/datadog_example.png +0 -0
  5. data/lib/pii_safe_schema.rb +4 -0
  6. data/lib/pii_safe_schema/configuration.rb +47 -13
  7. data/lib/pii_safe_schema/notifiers/data_dog.rb +11 -10
  8. data/lib/pii_safe_schema/version.rb +1 -1
  9. data/pii_safe_schema.gemspec +6 -4
  10. metadata +45 -10
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9822040f81e24ee9314973cd495c014cf6170a53ed8d8896b0f35575e0be5332
4
- data.tar.gz: e4a0236f9fe223582babe0e27643be0cea1e71d77da7210c7e9260b0cd95df91
3
+ metadata.gz: e10cda981f36b6e67b9964dfbe011856436d48242844ed569dff0d6406607512
4
+ data.tar.gz: dfc71c971b632a0cf124a2bf42a494cd1306e6b4a7178b4d3de085f45a6931cf
5
5
  SHA512:
6
- metadata.gz: 946cb5a5b94dae2dde45f45deb836964b0013f3c3ee0641ebd69379b9470459041777478866e6abc8904f3070185095c820edc7b5c07aab603e32ac235389ae1
7
- data.tar.gz: f34869383b818c33019421edd2c9d7a5ecae4206083c5d73cdfab25232365ede802c5e3c392fa446bfc3d30707e77fdbc43ac5057c818dbe37ec66c105138ad1
6
+ metadata.gz: 0e1c90b37fa8c3a70522bacb74b172855135878af97a355403e79325d8ed32a849f173015fcec844e5f3213715bacc5ba4ce18380550f667b92f9126940f821f
7
+ data.tar.gz: c8044fb8a4d0065ed97625c93e04a7e82dad3ab338d6a878e0fb5839ea26b7d1b7b9f8fa5fddca8413010ddd46306bb4235314f242575ce698c3d8c9e66cbac9
data/CHANGELOG.md CHANGED
@@ -4,6 +4,17 @@ All notable changes to this project will be documented in this file.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
5
5
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
+ ## 1.2.0 - 2019-4-20
8
+ ### Added
9
+ - Can pass Datadog Client object as a configuration option.
10
+
11
+ ### Changed
12
+ - Specs use SQLite3 instead of Postgres, further unblocking local development
13
+ - README got a facelift 😍
14
+
15
+ ### Fixed
16
+ - Development on Windows 10 environments now work
17
+
7
18
  ## 1.1.0 - 2019-4-18
8
19
  ### Added
9
20
  - Added MIT License
data/README.md CHANGED
@@ -1,28 +1,44 @@
1
- ## PiiSafeSchema
1
+ # PII Safe Schema [![CircleCI](https://circleci.com/gh/wealthsimple/pii_safe_schema.svg?style=svg)](https://circleci.com/gh/wealthsimple/pii_safe_schema) [![Coverage Status](https://coveralls.io/repos/github/wealthsimple/pii_safe_schema/badge.svg?branch=master)](https://coveralls.io/github/wealthsimple/pii_safe_schema?branch=master)
2
2
 
3
- this gem serves a few functions:
3
+ Schema migration tool for checking and adding comments on *Personally Identifiable Information* (PII) columns in Rails.
4
+
5
+ Specifically, this gem serves a few functions:
4
6
 
5
7
  * Warning you when you might be missing an annotation on a column
6
- * auto generating your migrations for you
7
- * alerting the security team through datadog events if there are remaining unannotated columns
8
+ * Auto generating your migrations for you
9
+ * Customizable actions through Datadog Events if there are remaining unannotated columns. E.g. alerting your Security Team
10
+
11
+ ![Screenshot of Datadog Event alert](datadog_example.png)
12
+
13
+ ## Why
8
14
 
15
+ Data privacy is an ever increasing concern for users, especially if your project or business is in sensitive industries like healthcare or finance.
9
16
 
17
+ Having structured metadata on the database level of your application ensures Business Intelligence consumers (I.e. Periscope Data) can appropriately filter or obfuscate columns that personally identify your users without impacting business needs.
10
18
 
11
- ### Getting Started
19
+ In other words, as your attack surface increases, the risk of user PII disclosure remains the same.
12
20
 
13
- `gem 'pii_safe_schema'`
21
+ In your data warehousing pipeline, consume the structured metadata this gem provides in order to maintain the privacy of your users.
14
22
 
15
- add the following to `application.rb`
23
+ ## Getting Started
16
24
 
25
+ Add your Rails project Gemfile:
26
+
27
+ ```ruby
28
+ gem 'pii_safe_schema'
17
29
  ```
30
+
31
+ Then, to your `application.rb`
32
+
33
+ ```ruby
18
34
  config.after_initialize do
19
35
  PiiSafeSchema.activate!
20
36
  end
21
37
  ```
22
38
 
23
- if you want to ignore certain columns, add the following initializer:
39
+ If you want to ignore certain columns, add the following initializer:
24
40
 
25
- ```
41
+ ```ruby
26
42
  # initializers/pii_safe_schema.rb
27
43
 
28
44
  PiiSafeSchema.configure do |config|
@@ -30,17 +46,45 @@ PiiSafeSchema.configure do |config|
30
46
  some_table: :*, # ignore the whole table
31
47
  some_other_table: [:column_1, :column_2] # just those columns
32
48
  }
49
+
50
+ # Pass whatever instance you want here, but it must implement the method
51
+ # #event(title, message, opts = {})
52
+ # which is what datadog-statsd does:
53
+ config.datadog_client = Datadog::Statsd.new(
54
+ Rails.application.secrets.fetch(:datadog_host),
55
+ Datadog::Statsd::DEFAULT_PORT,
56
+ # ...
57
+ )
33
58
  end
34
59
  ```
35
60
 
36
- ### Generating Comment Migrations
61
+ ## Generating Comment Migrations
37
62
 
38
- `rake pii_safe_schema:generate_migrations`
63
+ ```ruby
64
+ rake pii_safe_schema:generate_migrations
65
+ ```
39
66
 
40
- this will generate one migration file for each table that should be commented.
67
+ This will generate one migration file for each table that should be commented.
41
68
  it will create a comment field for each column that it warns you about when you start a rails server or console.
42
69
 
70
+ ## Credits
71
+
72
+ Thanks to [Alexi Garrow](https://github.com/AGarrow) for the original code.
73
+
74
+ ## Contributing
43
75
 
76
+ Everyone is encouraged to help improve this project. Here are a few ways you can help:
44
77
 
78
+ * [Report bugs](https://github.com/wealthsimple/pii_safe_schema/issues)
79
+ * Fix bugs and [submit pull requests](https://github.com/wealthsimple/pii_safe_schema/pulls)
80
+ * Write, clarify, or fix documentation
81
+ * Suggest or add new features
45
82
 
83
+ To get started with development and testing:
46
84
 
85
+ ```bash
86
+ git clone https://github.com/wealthsimple/pii_safe_schema.git
87
+ cd pii_safe_schema
88
+ bundle install
89
+ bundle exec rspec
90
+ ```
data/datadog_example.png ADDED
Binary file
data/lib/pii_safe_schema.rb CHANGED
@@ -18,6 +18,10 @@ module PiiSafeSchema
18
18
  @configuration ||= Configuration.new
19
19
  end
20
20
 
21
+ def self.reset_configuration!
22
+ @configuration = Configuration.new
23
+ end
24
+
21
25
  def self.configure
22
26
  yield(configuration)
23
27
  end
data/lib/pii_safe_schema/configuration.rb CHANGED
@@ -5,12 +5,14 @@ module PiiSafeSchema
5
5
  ar_internal_metadata: :*,
6
6
  }.freeze
7
7
 
8
+ KNOWN_DD_CLIENTS = %w[DataDogClient Ws::Railway::Datadog].freeze
9
+
8
10
  def initialize
9
11
  @user_ignore = {}
10
12
  end
11
13
 
12
14
  def ignore=(ignore_params)
13
- validate(ignore_params)
15
+ validate_ignore(ignore_params)
14
16
  @user_ignore = ignore_params
15
17
  end
16
18
 
@@ -18,6 +20,20 @@ module PiiSafeSchema
18
20
  @user_ignore.merge(DEFAULT_IGNORE)
19
21
  end
20
22
 
23
+ def datadog_client=(client)
24
+ raise_config_error(:datadog_client) if client.present? && !client.respond_to?(:event)
25
+
26
+ @datadog_client = client
27
+ end
28
+
29
+ def datadog_client
30
+ @datadog_client ||= begin
31
+ KNOWN_DD_CLIENTS.each do |client|
32
+ return client.safe_constantize if defined?(client)
33
+ end
34
+ end
35
+ end
36
+
21
37
  def ignore_tables
22
38
  ignore.select { |_k, v| v.to_s == '*' }.keys.map(&:to_s)
23
39
  end
@@ -28,11 +44,11 @@ module PiiSafeSchema
28
44
 
29
45
  private
30
46
 
31
- def validate(ignore_params)
32
- raise_config_error unless ignore_params.is_a?(Hash)
47
+ def validate_ignore(ignore_params)
48
+ raise_config_error(:ignore) unless ignore_params.is_a?(Hash)
33
49
 
34
50
  ignore_params.values.each do |ip|
35
- raise_config_error unless valid_column_list?(ip) || ip == :*
51
+ raise_config_error(:ignore) unless valid_column_list?(ip) || ip == :*
36
52
  end
37
53
  true
38
54
  end
@@ -41,19 +57,37 @@ module PiiSafeSchema
41
57
  value.is_a?(Array) && value.all? { |c| c.is_a?(Symbol) }
42
58
  end
43
59
 
44
- def raise_config_error
45
- raise ConfigurationError, ConfigurationError.message
60
+ def raise_config_error(problem)
61
+ raise ConfigurationError, problem
46
62
  end
47
63
  end
48
64
 
49
65
  class ConfigurationError < StandardError
50
- def self.message
51
- <<~HEREDOC
52
- ignore must be a hash where the values are
53
- symbols or arrays of symbols.
54
- e.g. ignore = { some_table: :* } ##ignore the whole some_table
55
- or ignore = { some_table: [:some_column, :some_other_column] }
56
- HEREDOC
66
+ IGNORE_MSG = <<~HEREDOC.freeze
67
+ ignore must be a hash where the values are
68
+ symbols or arrays of symbols.
69
+ e.g. ignore = { some_table: :* } ##ignore the whole some_table
70
+ or ignore = { some_table: [:some_column, :some_other_column] }
71
+ HEREDOC
72
+
73
+ DD_CLIENT_MSG = <<~HEREDOC.freeze
74
+ Datadog client must be implement #event(title, text, opts = {})
75
+
76
+ Consider using dogstatsd-ruby gem and pass in Datadog::Statsd.new(...)
77
+ as the client.
78
+ HEREDOC
79
+
80
+ def initialize(problem)
81
+ super(
82
+ case problem
83
+ when :ignore
84
+ IGNORE_MSG
85
+ when :datadog_client
86
+ DD_CLIENT_MSG
87
+ else
88
+ problem
89
+ end
90
+ )
57
91
  end
58
92
  end
59
93
  end
data/lib/pii_safe_schema/notifiers/data_dog.rb CHANGED
@@ -1,17 +1,20 @@
1
1
  module PiiSafeSchema
2
2
  module Notify
3
3
  module DataDog
4
- KNOWN_CLIENTS = %w[DataDogClient Ws::Railway::Datadog].freeze
4
+ # deprecated
5
+ KNOWN_CLIENTS = PiiSafeSchema::Configuration::KNOWN_DD_CLIENTS
5
6
 
6
7
  class << self
7
8
  def deliver(pii_column)
8
9
  return unless %w[staging production development].include?(Rails.env)
9
- return if dog_client.nil?
10
+ return if datadog_client.nil?
10
11
 
11
- dog_client.event('PII Annotation Warning',
12
- message(pii_column),
13
- msg_title: 'Unannotated PII Column',
14
- alert_type: 'warning')
12
+ datadog_client.event(
13
+ 'PII Annotation Warning',
14
+ message(pii_column),
15
+ msg_title: 'Unannotated PII Column',
16
+ alert_type: 'warning',
17
+ )
15
18
  end
16
19
 
17
20
  private
@@ -20,10 +23,8 @@ module PiiSafeSchema
20
23
  "column #{pii_column.table}.#{pii_column.column.name} is not annotated"
21
24
  end
22
25
 
23
- def dog_client
24
- KNOWN_CLIENTS.each do |client|
25
- return client.safe_constantize if defined?(client)
26
- end
26
+ def datadog_client
27
+ PiiSafeSchema.configuration.datadog_client
27
28
  end
28
29
  end
29
30
  end
data/lib/pii_safe_schema/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module PiiSafeSchema
2
- VERSION = '1.1.0'.freeze
2
+ VERSION = '1.2.0'.freeze
3
3
  end
data/pii_safe_schema.gemspec CHANGED
@@ -23,21 +23,23 @@ Gem::Specification.new do |s|
23
23
  s.add_dependency 'colorize'
24
24
  s.add_dependency 'rails', '>= 5'
25
25
 
26
- s.add_development_dependency 'bundler', '~> 1.16'
26
+ s.add_development_dependency 'bundler', '>= 1.16'
27
27
  s.add_development_dependency 'bundler-audit'
28
+ s.add_development_dependency 'coveralls'
28
29
  s.add_development_dependency 'dogstatsd-ruby'
29
30
  s.add_development_dependency 'git'
30
31
  s.add_development_dependency 'guard-rspec'
31
32
  s.add_development_dependency 'pry'
32
- s.add_development_dependency 'rake', '~> 10.0'
33
- s.add_development_dependency 'rspec', '~> 3.0'
33
+ s.add_development_dependency 'rake', '>= 10.0'
34
+ s.add_development_dependency 'rspec', '< 4', '>= 3.0'
34
35
  s.add_development_dependency 'rspec-collection_matchers'
35
36
  s.add_development_dependency 'rspec-its'
36
37
  s.add_development_dependency 'rubocop'
37
38
  s.add_development_dependency 'simplecov'
39
+ s.add_development_dependency 'sqlite3-ruby'
38
40
  s.add_development_dependency 'ws-style'
39
41
 
40
42
  # Required by activerecord-safer_migrations
41
- s.add_development_dependency 'pg', '~> 0.21'
43
+ s.add_development_dependency 'pg', '>= 0.21'
42
44
  s.add_development_dependency 'strong_migrations'
43
45
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pii_safe_schema
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexi Garrow
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2019-04-18 00:00:00.000000000 Z
11
+ date: 2019-04-20 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activesupport
@@ -56,14 +56,14 @@ dependencies:
56
56
  name: bundler
57
57
  requirement: !ruby/object:Gem::Requirement
58
58
  requirements:
59
- - - "~>"
59
+ - - ">="
60
60
  - !ruby/object:Gem::Version
61
61
  version: '1.16'
62
62
  type: :development
63
63
  prerelease: false
64
64
  version_requirements: !ruby/object:Gem::Requirement
65
65
  requirements:
66
- - - "~>"
66
+ - - ">="
67
67
  - !ruby/object:Gem::Version
68
68
  version: '1.16'
69
69
  - !ruby/object:Gem::Dependency
@@ -80,6 +80,20 @@ dependencies:
80
80
  - - ">="
81
81
  - !ruby/object:Gem::Version
82
82
  version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: coveralls
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
83
97
  - !ruby/object:Gem::Dependency
84
98
  name: dogstatsd-ruby
85
99
  requirement: !ruby/object:Gem::Requirement
@@ -140,30 +154,36 @@ dependencies:
140
154
  name: rake
141
155
  requirement: !ruby/object:Gem::Requirement
142
156
  requirements:
143
- - - "~>"
157
+ - - ">="
144
158
  - !ruby/object:Gem::Version
145
159
  version: '10.0'
146
160
  type: :development
147
161
  prerelease: false
148
162
  version_requirements: !ruby/object:Gem::Requirement
149
163
  requirements:
150
- - - "~>"
164
+ - - ">="
151
165
  - !ruby/object:Gem::Version
152
166
  version: '10.0'
153
167
  - !ruby/object:Gem::Dependency
154
168
  name: rspec
155
169
  requirement: !ruby/object:Gem::Requirement
156
170
  requirements:
157
- - - "~>"
171
+ - - ">="
158
172
  - !ruby/object:Gem::Version
159
173
  version: '3.0'
174
+ - - "<"
175
+ - !ruby/object:Gem::Version
176
+ version: '4'
160
177
  type: :development
161
178
  prerelease: false
162
179
  version_requirements: !ruby/object:Gem::Requirement
163
180
  requirements:
164
- - - "~>"
181
+ - - ">="
165
182
  - !ruby/object:Gem::Version
166
183
  version: '3.0'
184
+ - - "<"
185
+ - !ruby/object:Gem::Version
186
+ version: '4'
167
187
  - !ruby/object:Gem::Dependency
168
188
  name: rspec-collection_matchers
169
189
  requirement: !ruby/object:Gem::Requirement
@@ -220,6 +240,20 @@ dependencies:
220
240
  - - ">="
221
241
  - !ruby/object:Gem::Version
222
242
  version: '0'
243
+ - !ruby/object:Gem::Dependency
244
+ name: sqlite3-ruby
245
+ requirement: !ruby/object:Gem::Requirement
246
+ requirements:
247
+ - - ">="
248
+ - !ruby/object:Gem::Version
249
+ version: '0'
250
+ type: :development
251
+ prerelease: false
252
+ version_requirements: !ruby/object:Gem::Requirement
253
+ requirements:
254
+ - - ">="
255
+ - !ruby/object:Gem::Version
256
+ version: '0'
223
257
  - !ruby/object:Gem::Dependency
224
258
  name: ws-style
225
259
  requirement: !ruby/object:Gem::Requirement
@@ -238,14 +272,14 @@ dependencies:
238
272
  name: pg
239
273
  requirement: !ruby/object:Gem::Requirement
240
274
  requirements:
241
- - - "~>"
275
+ - - ">="
242
276
  - !ruby/object:Gem::Version
243
277
  version: '0.21'
244
278
  type: :development
245
279
  prerelease: false
246
280
  version_requirements: !ruby/object:Gem::Requirement
247
281
  requirements:
248
- - - "~>"
282
+ - - ">="
249
283
  - !ruby/object:Gem::Version
250
284
  version: '0.21'
251
285
  - !ruby/object:Gem::Dependency
@@ -282,6 +316,7 @@ files:
282
316
  - LICENSE
283
317
  - README.md
284
318
  - Rakefile
319
+ - datadog_example.png
285
320
  - lib/pii_safe_schema.rb
286
321
  - lib/pii_safe_schema/annotations.rb
287
322
  - lib/pii_safe_schema/configuration.rb