pii_safe_schema 1.0.4 → 1.3.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.circleci/config.yml +1 -1
- data/.rubocop.yml +4 -0
- data/.ruby-version +1 -1
- data/CHANGELOG.md +23 -0
- data/LICENSE +21 -0
- data/README.md +66 -13
- data/datadog_example.png +0 -0
- data/lib/pii_safe_schema.rb +56 -2
- data/lib/pii_safe_schema/annotations.rb +6 -2
- data/lib/pii_safe_schema/configuration.rb +47 -14
- data/lib/pii_safe_schema/invalid_column_error.rb +4 -0
- data/lib/pii_safe_schema/migration_generator.rb +1 -3
- data/lib/pii_safe_schema/notifiers/data_dog.rb +11 -10
- data/lib/pii_safe_schema/pii_column.rb +14 -4
- data/lib/pii_safe_schema/version.rb +1 -1
- data/lib/tasks/pii_safe_schema.rake +22 -1
- data/pii_safe_schema.gemspec +11 -6
- metadata +83 -13
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 1ebd5e7d63f153bb4cedccaa5345086238c221520e0a9a65e4874c5cdb5e4b0a
|
4
|
+
data.tar.gz: c788254ae8852acc2dc8a8e1b62fb4463cd4133fdec748499715cbfb4132cd40
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3ff11da6f69694f15a90b66d4d9ed8a81beb01db60f6ab119120cae2af2019b82026b0721ffb441ef66d199ab794b6d6212ffae25d276f80fb14e8fb8cd68344
|
7
|
+
data.tar.gz: e3f2cb9127b90fc06ffc56c8a109097dc2dd4b67c2e238ebd3afd4336aca218d2e27606f13936becb4f4e84947c24b0b3db20527baeef6cb4647a345d970e820
|
data/.circleci/config.yml
CHANGED
data/.rubocop.yml
CHANGED
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
2.
|
1
|
+
2.7.2
|
data/CHANGELOG.md
CHANGED
@@ -4,6 +4,29 @@ All notable changes to this project will be documented in this file.
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
5
5
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
6
6
|
|
7
|
+
## 1.3.1 - 2019-11-06
|
8
|
+
### Fixed
|
9
|
+
- Passing arguments to `rake pii_safe_schema:generate_migrations` actually works
|
10
|
+
|
11
|
+
## 1.3.0 - 2019-11-04
|
12
|
+
### Added
|
13
|
+
- Can pass explicitly annotate PII columns from the command line as arguments when using `rake pii_safe_schema:generate_migrations`.
|
14
|
+
|
15
|
+
## 1.2.0 - 2019-4-20
|
16
|
+
### Added
|
17
|
+
- Can pass Datadog Client object as a configuration option.
|
18
|
+
|
19
|
+
### Changed
|
20
|
+
- Specs use SQLite3 instead of Postgres, further unblocking local development
|
21
|
+
- README got a facelift 😍
|
22
|
+
|
23
|
+
### Fixed
|
24
|
+
- Development on Windows 10 environments now work
|
25
|
+
|
26
|
+
## 1.1.0 - 2019-4-18
|
27
|
+
### Added
|
28
|
+
- Added MIT License
|
29
|
+
|
7
30
|
## 1.0.4 - 2019-4-16
|
8
31
|
### Fixed
|
9
32
|
- converted any hyphens to underscores for consistency.
|
data/LICENSE
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
MIT License
|
2
|
+
|
3
|
+
Copyright (c) 2019 Wealthsimple
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
13
|
+
copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
21
|
+
SOFTWARE.
|
data/README.md
CHANGED
@@ -1,28 +1,44 @@
|
|
1
|
-
|
1
|
+
# PII Safe Schema [![CircleCI](https://circleci.com/gh/wealthsimple/pii_safe_schema.svg?style=svg)](https://circleci.com/gh/wealthsimple/pii_safe_schema) [![Coverage Status](https://coveralls.io/repos/github/wealthsimple/pii_safe_schema/badge.svg?branch=master)](https://coveralls.io/github/wealthsimple/pii_safe_schema?branch=master)
|
2
2
|
|
3
|
-
|
3
|
+
Schema migration tool for checking and adding comments on *Personally Identifiable Information* (PII) columns in Rails.
|
4
|
+
|
5
|
+
Specifically, this gem serves a few functions:
|
4
6
|
|
5
7
|
* Warning you when you might be missing an annotation on a column
|
6
|
-
*
|
7
|
-
*
|
8
|
+
* Auto generating your migrations for you
|
9
|
+
* Customizable actions through Datadog Events if there are remaining unannotated columns. E.g. alerting your Security Team
|
10
|
+
|
11
|
+
![Screenshot of Datadog Event alert](datadog_example.png)
|
12
|
+
|
13
|
+
## Why
|
14
|
+
|
15
|
+
Data privacy is an ever increasing concern for users, especially if your project or business is in sensitive industries like healthcare or finance.
|
8
16
|
|
17
|
+
Having structured metadata on the database level of your application ensures Business Intelligence consumers (I.e. Periscope Data) can appropriately filter or obfuscate columns that personally identify your users without impacting business needs.
|
9
18
|
|
19
|
+
In other words, as your attack surface increases, the risk of user PII disclosure remains the same.
|
10
20
|
|
11
|
-
|
21
|
+
In your data warehousing pipeline, consume the structured metadata this gem provides in order to maintain the privacy of your users.
|
12
22
|
|
13
|
-
|
23
|
+
## Getting Started
|
14
24
|
|
15
|
-
|
25
|
+
Add your Rails project Gemfile:
|
16
26
|
|
27
|
+
```ruby
|
28
|
+
gem 'pii_safe_schema'
|
17
29
|
```
|
30
|
+
|
31
|
+
Then, to your `application.rb`
|
32
|
+
|
33
|
+
```ruby
|
18
34
|
config.after_initialize do
|
19
35
|
PiiSafeSchema.activate!
|
20
36
|
end
|
21
37
|
```
|
22
38
|
|
23
|
-
|
39
|
+
If you want to ignore certain columns, add the following initializer:
|
24
40
|
|
25
|
-
```
|
41
|
+
```ruby
|
26
42
|
# initializers/pii_safe_schema.rb
|
27
43
|
|
28
44
|
PiiSafeSchema.configure do |config|
|
@@ -30,17 +46,54 @@ PiiSafeSchema.configure do |config|
|
|
30
46
|
some_table: :*, # ignore the whole table
|
31
47
|
some_other_table: [:column_1, :column_2] # just those columns
|
32
48
|
}
|
49
|
+
|
50
|
+
# Pass whatever instance you want here, but it must implement the method
|
51
|
+
# #event(title, message, opts = {})
|
52
|
+
# which is what datadog-statsd does:
|
53
|
+
config.datadog_client = Datadog::Statsd.new(
|
54
|
+
Rails.application.secrets.fetch(:datadog_host),
|
55
|
+
Datadog::Statsd::DEFAULT_PORT,
|
56
|
+
# ...
|
57
|
+
)
|
33
58
|
end
|
34
59
|
```
|
35
60
|
|
36
|
-
|
61
|
+
## Generating Comment Migrations
|
62
|
+
|
63
|
+
```bash
|
64
|
+
rake pii_safe_schema:generate_migrations
|
65
|
+
```
|
66
|
+
|
67
|
+
This will generate one migration file for each table that should be commented. It will create a comment field for each column that it warns you about when you start a rails server or console.
|
68
|
+
|
69
|
+
### Explicit annotations
|
70
|
+
|
71
|
+
If the generator fails to identify a PII column, you can specify explicitly what columns in what tables are PII. This is particularly useful if you're installed pii_safe_schema into an existing project.
|
72
|
+
|
73
|
+
```bash
|
74
|
+
rake pii_safe_schema:generate_migrations [table:column:annotation_type] ...
|
75
|
+
```
|
76
|
+
|
77
|
+
Run `rake pii_safe_schema:generate_migrations help` for details
|
37
78
|
|
38
|
-
|
79
|
+
## Credits
|
39
80
|
|
40
|
-
|
41
|
-
it will create a comment field for each column that it warns you about when you start a rails server or console.
|
81
|
+
Thanks to [Alexi Garrow](https://github.com/AGarrow) for the original code.
|
42
82
|
|
83
|
+
## Contributing
|
43
84
|
|
85
|
+
Everyone is encouraged to help improve this project. Here are a few ways you can help:
|
44
86
|
|
87
|
+
* [Report bugs](https://github.com/wealthsimple/pii_safe_schema/issues)
|
88
|
+
* Fix bugs and [submit pull requests](https://github.com/wealthsimple/pii_safe_schema/pulls)
|
89
|
+
* Write, clarify, or fix documentation
|
90
|
+
* Suggest or add new features
|
45
91
|
|
92
|
+
To get started with development and testing:
|
46
93
|
|
94
|
+
```bash
|
95
|
+
git clone https://github.com/wealthsimple/pii_safe_schema.git
|
96
|
+
cd pii_safe_schema
|
97
|
+
bundle install
|
98
|
+
bundle exec rspec
|
99
|
+
```
|
data/datadog_example.png
ADDED
Binary file
|
data/lib/pii_safe_schema.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
require 'pii_safe_schema/invalid_column_error'
|
1
2
|
require 'pii_safe_schema/configuration'
|
2
3
|
require 'pii_safe_schema/annotations'
|
3
4
|
require 'pii_safe_schema/notify'
|
@@ -18,6 +19,10 @@ module PiiSafeSchema
|
|
18
19
|
@configuration ||= Configuration.new
|
19
20
|
end
|
20
21
|
|
22
|
+
def self.reset_configuration!
|
23
|
+
@configuration = Configuration.new
|
24
|
+
end
|
25
|
+
|
21
26
|
def self.configure
|
22
27
|
yield(configuration)
|
23
28
|
end
|
@@ -32,7 +37,56 @@ module PiiSafeSchema
|
|
32
37
|
Rails.logger.info('PiiSafeSchema: No DB'.red)
|
33
38
|
end
|
34
39
|
|
35
|
-
def self.generate_migrations
|
36
|
-
PiiSafeSchema::MigrationGenerator.generate_migrations(
|
40
|
+
def self.generate_migrations(additional_pii_columns = [])
|
41
|
+
PiiSafeSchema::MigrationGenerator.generate_migrations(
|
42
|
+
PiiSafeSchema::PiiColumn.all + additional_pii_columns,
|
43
|
+
)
|
44
|
+
end
|
45
|
+
|
46
|
+
def self.parse_additional_columns(arguments)
|
47
|
+
arguments.map do |str|
|
48
|
+
matches = /([a-z_]+):([a-z_]+):([a-z_]+)/i.match(str)
|
49
|
+
return print_help! if matches.blank?
|
50
|
+
|
51
|
+
suggestion = Annotations.comment(matches[3])
|
52
|
+
return print_help! if suggestion.blank?
|
53
|
+
|
54
|
+
PiiColumn.from_column_name(table: matches[1], column: matches[2], suggestion: suggestion)
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
def self.print_help!(do_exit: true) # rubocop:disable Metrics/MethodLength
|
59
|
+
puts <<~HELPMSG # rubocop:disable Rails/Output
|
60
|
+
Usage:
|
61
|
+
rake pii_safe_schema:generate_migrations [table:column:annotation_type] ...
|
62
|
+
|
63
|
+
Arguments:
|
64
|
+
[table:column:annotation_type] # A column to manually annotate. Can be repeated.
|
65
|
+
# annotation_type can be "email", "phone", "ip_address",
|
66
|
+
# "geolocation", "address", "postal_code", "name",
|
67
|
+
# "sensitive_data", or "encrypted_data"
|
68
|
+
|
69
|
+
Description:
|
70
|
+
Generates a migration to add PII annotation comments to appropriate columns on a table.
|
71
|
+
Uses a series of regular expressions to find sensitive fields.
|
72
|
+
|
73
|
+
Optionally supply arguments to annotate columns explicitly
|
74
|
+
|
75
|
+
Example:
|
76
|
+
rake pii_safe_schema:generate_migrations signatures:signatory_name:name signatures:landline:phone
|
77
|
+
|
78
|
+
Will generate a migration with the following, assuming automatic regex had no matches:
|
79
|
+
|
80
|
+
class ChangeCommentsInSignatures < ActiveRecord::Migration[5.2]
|
81
|
+
def change
|
82
|
+
safety_assured do
|
83
|
+
change_column :signatures, :signatory_name, :string, comment: '{"pii":{"obfuscate":"name_obfuscator"}}'
|
84
|
+
change_column :signatures, :landline, :string, comment: '{"pii":{"obfuscate":"phone_obfuscator"}}'
|
85
|
+
end
|
86
|
+
end
|
87
|
+
end
|
88
|
+
HELPMSG
|
89
|
+
|
90
|
+
exit(1) if do_exit # rubocop:disable Rails/Exit
|
37
91
|
end
|
38
92
|
end
|
@@ -70,14 +70,18 @@ module PiiSafeSchema
|
|
70
70
|
nil
|
71
71
|
end
|
72
72
|
|
73
|
+
def self.comment(annotation_type)
|
74
|
+
COLUMNS.dig(annotation_type.to_sym, :comment)
|
75
|
+
end
|
76
|
+
|
73
77
|
def apply_recommendation?(column, pii_info)
|
74
78
|
!encrypted?(column) &&
|
75
|
-
pii_info[:regexp].match(column.name) &&
|
79
|
+
pii_info[:regexp].match?(column.name) &&
|
76
80
|
column.comment != pii_info[:comment].to_json
|
77
81
|
end
|
78
82
|
|
79
83
|
def encrypted?(column)
|
80
|
-
COLUMNS[:encrypted_data][:regexp].match(column.name)
|
84
|
+
COLUMNS[:encrypted_data][:regexp].match?(column.name)
|
81
85
|
end
|
82
86
|
|
83
87
|
def apply_encrypted_recommendation?(column)
|
@@ -5,12 +5,14 @@ module PiiSafeSchema
|
|
5
5
|
ar_internal_metadata: :*,
|
6
6
|
}.freeze
|
7
7
|
|
8
|
+
KNOWN_DD_CLIENTS = %w[DataDogClient Ws::Railway::Datadog].freeze
|
9
|
+
|
8
10
|
def initialize
|
9
11
|
@user_ignore = {}
|
10
12
|
end
|
11
13
|
|
12
14
|
def ignore=(ignore_params)
|
13
|
-
|
15
|
+
validate_ignore(ignore_params)
|
14
16
|
@user_ignore = ignore_params
|
15
17
|
end
|
16
18
|
|
@@ -18,6 +20,19 @@ module PiiSafeSchema
|
|
18
20
|
@user_ignore.merge(DEFAULT_IGNORE)
|
19
21
|
end
|
20
22
|
|
23
|
+
def datadog_client=(client)
|
24
|
+
raise_config_error(:datadog_client) if client.present? && !client.respond_to?(:event)
|
25
|
+
|
26
|
+
@datadog_client = client
|
27
|
+
end
|
28
|
+
|
29
|
+
def datadog_client
|
30
|
+
@datadog_client ||=
|
31
|
+
KNOWN_DD_CLIENTS.find do |client|
|
32
|
+
client.safe_constantize if defined?(client)
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
21
36
|
def ignore_tables
|
22
37
|
ignore.select { |_k, v| v.to_s == '*' }.keys.map(&:to_s)
|
23
38
|
end
|
@@ -28,11 +43,11 @@ module PiiSafeSchema
|
|
28
43
|
|
29
44
|
private
|
30
45
|
|
31
|
-
def
|
32
|
-
raise_config_error unless ignore_params.is_a?(Hash)
|
46
|
+
def validate_ignore(ignore_params)
|
47
|
+
raise_config_error(:ignore) unless ignore_params.is_a?(Hash)
|
33
48
|
|
34
|
-
ignore_params.
|
35
|
-
raise_config_error unless valid_column_list?(ip) || ip == :*
|
49
|
+
ignore_params.each_value do |ip|
|
50
|
+
raise_config_error(:ignore) unless valid_column_list?(ip) || ip == :*
|
36
51
|
end
|
37
52
|
true
|
38
53
|
end
|
@@ -41,19 +56,37 @@ module PiiSafeSchema
|
|
41
56
|
value.is_a?(Array) && value.all? { |c| c.is_a?(Symbol) }
|
42
57
|
end
|
43
58
|
|
44
|
-
def raise_config_error
|
45
|
-
raise ConfigurationError,
|
59
|
+
def raise_config_error(problem)
|
60
|
+
raise ConfigurationError, problem
|
46
61
|
end
|
47
62
|
end
|
48
63
|
|
49
64
|
class ConfigurationError < StandardError
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
65
|
+
IGNORE_MSG = <<~HEREDOC.freeze
|
66
|
+
ignore must be a hash where the values are
|
67
|
+
symbols or arrays of symbols.
|
68
|
+
e.g. ignore = { some_table: :* } ##ignore the whole some_table
|
69
|
+
or ignore = { some_table: [:some_column, :some_other_column] }
|
70
|
+
HEREDOC
|
71
|
+
|
72
|
+
DD_CLIENT_MSG = <<~HEREDOC.freeze
|
73
|
+
Datadog client must be implement #event(title, text, opts = {})
|
74
|
+
|
75
|
+
Consider using dogstatsd-ruby gem and pass in Datadog::Statsd.new(...)
|
76
|
+
as the client.
|
77
|
+
HEREDOC
|
78
|
+
|
79
|
+
def initialize(problem)
|
80
|
+
super(
|
81
|
+
case problem
|
82
|
+
when :ignore
|
83
|
+
IGNORE_MSG
|
84
|
+
when :datadog_client
|
85
|
+
DD_CLIENT_MSG
|
86
|
+
else
|
87
|
+
problem
|
88
|
+
end
|
89
|
+
)
|
57
90
|
end
|
58
91
|
end
|
59
92
|
end
|
@@ -18,9 +18,7 @@ module PiiSafeSchema
|
|
18
18
|
migration_file = generator.create_migration_file
|
19
19
|
file_lines = File.open(migration_file, 'r').read.split("\n")
|
20
20
|
change_line = file_lines.find_index { |i| /def change/.match(i) }
|
21
|
-
new_contents = file_lines[0..change_line] +
|
22
|
-
generated_lines +
|
23
|
-
file_lines[change_line + 1..-1]
|
21
|
+
new_contents = file_lines[0..change_line] + generated_lines + file_lines[change_line + 1..]
|
24
22
|
|
25
23
|
File.open(migration_file, 'w') do |f|
|
26
24
|
f.write(new_contents.join("\n"))
|
@@ -1,17 +1,20 @@
|
|
1
1
|
module PiiSafeSchema
|
2
2
|
module Notify
|
3
3
|
module DataDog
|
4
|
-
|
4
|
+
# deprecated
|
5
|
+
KNOWN_CLIENTS = PiiSafeSchema::Configuration::KNOWN_DD_CLIENTS
|
5
6
|
|
6
7
|
class << self
|
7
8
|
def deliver(pii_column)
|
8
9
|
return unless %w[staging production development].include?(Rails.env)
|
9
|
-
return if
|
10
|
+
return if datadog_client.nil?
|
10
11
|
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
12
|
+
datadog_client.event(
|
13
|
+
'PII Annotation Warning',
|
14
|
+
message(pii_column),
|
15
|
+
msg_title: 'Unannotated PII Column',
|
16
|
+
alert_type: 'warning',
|
17
|
+
)
|
15
18
|
end
|
16
19
|
|
17
20
|
private
|
@@ -20,10 +23,8 @@ module PiiSafeSchema
|
|
20
23
|
"column #{pii_column.table}.#{pii_column.column.name} is not annotated"
|
21
24
|
end
|
22
25
|
|
23
|
-
def
|
24
|
-
|
25
|
-
return client.safe_constantize if defined?(client)
|
26
|
-
end
|
26
|
+
def datadog_client
|
27
|
+
PiiSafeSchema.configuration.datadog_client
|
27
28
|
end
|
28
29
|
end
|
29
30
|
end
|
@@ -3,10 +3,6 @@ module PiiSafeSchema
|
|
3
3
|
extend PiiSafeSchema::Annotations
|
4
4
|
attr_reader :table, :column, :suggestion
|
5
5
|
|
6
|
-
def self.all
|
7
|
-
find_and_create
|
8
|
-
end
|
9
|
-
|
10
6
|
def initialize(table:, column:, suggestion:)
|
11
7
|
@table = table.to_sym
|
12
8
|
@column = column
|
@@ -14,6 +10,20 @@ module PiiSafeSchema
|
|
14
10
|
end
|
15
11
|
|
16
12
|
class << self
|
13
|
+
def all
|
14
|
+
find_and_create
|
15
|
+
end
|
16
|
+
|
17
|
+
def from_column_name(table:, column:, suggestion:)
|
18
|
+
activerecord_column = connection.columns(table.to_s).find { |c| c.name == column.to_s }
|
19
|
+
|
20
|
+
unless activerecord_column
|
21
|
+
raise InvalidColumnError, "column \"#{column}\" does not exist for table \"#{table}\""
|
22
|
+
end
|
23
|
+
|
24
|
+
new(table: table, column: activerecord_column, suggestion: suggestion)
|
25
|
+
end
|
26
|
+
|
17
27
|
private
|
18
28
|
|
19
29
|
def find_and_create
|
@@ -1,5 +1,26 @@
|
|
1
1
|
namespace :pii_safe_schema do
|
2
2
|
task generate_migrations: :environment do
|
3
|
-
PiiSafeSchema.
|
3
|
+
PiiSafeSchema.print_help! if ARGV[2] == 'help'
|
4
|
+
|
5
|
+
if ARGV.length == 1
|
6
|
+
PiiSafeSchema.generate_migrations
|
7
|
+
else
|
8
|
+
additional_columns = PiiSafeSchema.parse_additional_columns(ARGV[1..])
|
9
|
+
PiiSafeSchema.generate_migrations(additional_columns)
|
10
|
+
end
|
11
|
+
|
12
|
+
exit(0) # forces rake to stop after this and not assume args are tasks
|
13
|
+
rescue ActiveRecord::StatementInvalid, PiiSafeSchema::InvalidColumnError => e
|
14
|
+
raise e if e.instance_of?(ActiveRecord::StatementInvalid) && e.cause.class != PG::UndefinedTable
|
15
|
+
|
16
|
+
puts <<~HEREDOC
|
17
|
+
Unable to generate PII annotation migration. Either the underlying table or column does not exist:
|
18
|
+
|
19
|
+
#{e.message}
|
20
|
+
|
21
|
+
Please create the table & columns first, running their migrations, before attempting to use the pii_safe_schema generator.
|
22
|
+
HEREDOC
|
23
|
+
|
24
|
+
exit(1) # forces rake to stop after this and not assume args are tasks
|
4
25
|
end
|
5
26
|
end
|
data/pii_safe_schema.gemspec
CHANGED
@@ -10,6 +10,8 @@ Gem::Specification.new do |s|
|
|
10
10
|
|
11
11
|
s.summary = 'Schema migration tool for checking and adding comments on PII columns.'
|
12
12
|
s.homepage = 'https://github.com/wealthsimple/pii_safe_schema'
|
13
|
+
s.license = "MIT"
|
14
|
+
s.required_ruby_version = Gem::Requirement.new(">= 2.6")
|
13
15
|
|
14
16
|
s.files = `git ls-files -z`.split("\x0").reject do |f|
|
15
17
|
f.match(%r{^(test|spec|features)/})
|
@@ -19,25 +21,28 @@ Gem::Specification.new do |s|
|
|
19
21
|
s.executables = s.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
22
|
s.require_paths = ['lib']
|
21
23
|
|
22
|
-
s.add_dependency 'activesupport', '>= 5'
|
24
|
+
s.add_dependency 'activesupport', '>= 5', '< 7'
|
23
25
|
s.add_dependency 'colorize'
|
24
|
-
s.add_dependency 'rails', '>= 5'
|
26
|
+
s.add_dependency 'rails', '>= 5', '< 7'
|
25
27
|
|
26
|
-
s.add_development_dependency 'bundler', '
|
28
|
+
s.add_development_dependency 'bundler', '>= 1.16'
|
27
29
|
s.add_development_dependency 'bundler-audit'
|
30
|
+
s.add_development_dependency 'coveralls'
|
28
31
|
s.add_development_dependency 'dogstatsd-ruby'
|
29
32
|
s.add_development_dependency 'git'
|
30
33
|
s.add_development_dependency 'guard-rspec'
|
31
34
|
s.add_development_dependency 'pry'
|
32
|
-
s.add_development_dependency '
|
33
|
-
s.add_development_dependency '
|
35
|
+
s.add_development_dependency 'rails', '>= 5.2.3', '< 7'
|
36
|
+
s.add_development_dependency 'rake', '>= 10.0'
|
37
|
+
s.add_development_dependency 'rspec', '< 4', '>= 3.0'
|
34
38
|
s.add_development_dependency 'rspec-collection_matchers'
|
35
39
|
s.add_development_dependency 'rspec-its'
|
36
40
|
s.add_development_dependency 'rubocop'
|
37
41
|
s.add_development_dependency 'simplecov'
|
42
|
+
s.add_development_dependency 'sqlite3'
|
38
43
|
s.add_development_dependency 'ws-style'
|
39
44
|
|
40
45
|
# Required by activerecord-safer_migrations
|
41
|
-
s.add_development_dependency 'pg', '
|
46
|
+
s.add_development_dependency 'pg', '>= 0.21'
|
42
47
|
s.add_development_dependency 'strong_migrations'
|
43
48
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pii_safe_schema
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Alexi Garrow
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2021-03-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -17,6 +17,9 @@ dependencies:
|
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '5'
|
20
|
+
- - "<"
|
21
|
+
- !ruby/object:Gem::Version
|
22
|
+
version: '7'
|
20
23
|
type: :runtime
|
21
24
|
prerelease: false
|
22
25
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -24,6 +27,9 @@ dependencies:
|
|
24
27
|
- - ">="
|
25
28
|
- !ruby/object:Gem::Version
|
26
29
|
version: '5'
|
30
|
+
- - "<"
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '7'
|
27
33
|
- !ruby/object:Gem::Dependency
|
28
34
|
name: colorize
|
29
35
|
requirement: !ruby/object:Gem::Requirement
|
@@ -45,6 +51,9 @@ dependencies:
|
|
45
51
|
- - ">="
|
46
52
|
- !ruby/object:Gem::Version
|
47
53
|
version: '5'
|
54
|
+
- - "<"
|
55
|
+
- !ruby/object:Gem::Version
|
56
|
+
version: '7'
|
48
57
|
type: :runtime
|
49
58
|
prerelease: false
|
50
59
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -52,18 +61,21 @@ dependencies:
|
|
52
61
|
- - ">="
|
53
62
|
- !ruby/object:Gem::Version
|
54
63
|
version: '5'
|
64
|
+
- - "<"
|
65
|
+
- !ruby/object:Gem::Version
|
66
|
+
version: '7'
|
55
67
|
- !ruby/object:Gem::Dependency
|
56
68
|
name: bundler
|
57
69
|
requirement: !ruby/object:Gem::Requirement
|
58
70
|
requirements:
|
59
|
-
- - "
|
71
|
+
- - ">="
|
60
72
|
- !ruby/object:Gem::Version
|
61
73
|
version: '1.16'
|
62
74
|
type: :development
|
63
75
|
prerelease: false
|
64
76
|
version_requirements: !ruby/object:Gem::Requirement
|
65
77
|
requirements:
|
66
|
-
- - "
|
78
|
+
- - ">="
|
67
79
|
- !ruby/object:Gem::Version
|
68
80
|
version: '1.16'
|
69
81
|
- !ruby/object:Gem::Dependency
|
@@ -80,6 +92,20 @@ dependencies:
|
|
80
92
|
- - ">="
|
81
93
|
- !ruby/object:Gem::Version
|
82
94
|
version: '0'
|
95
|
+
- !ruby/object:Gem::Dependency
|
96
|
+
name: coveralls
|
97
|
+
requirement: !ruby/object:Gem::Requirement
|
98
|
+
requirements:
|
99
|
+
- - ">="
|
100
|
+
- !ruby/object:Gem::Version
|
101
|
+
version: '0'
|
102
|
+
type: :development
|
103
|
+
prerelease: false
|
104
|
+
version_requirements: !ruby/object:Gem::Requirement
|
105
|
+
requirements:
|
106
|
+
- - ">="
|
107
|
+
- !ruby/object:Gem::Version
|
108
|
+
version: '0'
|
83
109
|
- !ruby/object:Gem::Dependency
|
84
110
|
name: dogstatsd-ruby
|
85
111
|
requirement: !ruby/object:Gem::Requirement
|
@@ -136,32 +162,58 @@ dependencies:
|
|
136
162
|
- - ">="
|
137
163
|
- !ruby/object:Gem::Version
|
138
164
|
version: '0'
|
165
|
+
- !ruby/object:Gem::Dependency
|
166
|
+
name: rails
|
167
|
+
requirement: !ruby/object:Gem::Requirement
|
168
|
+
requirements:
|
169
|
+
- - ">="
|
170
|
+
- !ruby/object:Gem::Version
|
171
|
+
version: 5.2.3
|
172
|
+
- - "<"
|
173
|
+
- !ruby/object:Gem::Version
|
174
|
+
version: '7'
|
175
|
+
type: :development
|
176
|
+
prerelease: false
|
177
|
+
version_requirements: !ruby/object:Gem::Requirement
|
178
|
+
requirements:
|
179
|
+
- - ">="
|
180
|
+
- !ruby/object:Gem::Version
|
181
|
+
version: 5.2.3
|
182
|
+
- - "<"
|
183
|
+
- !ruby/object:Gem::Version
|
184
|
+
version: '7'
|
139
185
|
- !ruby/object:Gem::Dependency
|
140
186
|
name: rake
|
141
187
|
requirement: !ruby/object:Gem::Requirement
|
142
188
|
requirements:
|
143
|
-
- - "
|
189
|
+
- - ">="
|
144
190
|
- !ruby/object:Gem::Version
|
145
191
|
version: '10.0'
|
146
192
|
type: :development
|
147
193
|
prerelease: false
|
148
194
|
version_requirements: !ruby/object:Gem::Requirement
|
149
195
|
requirements:
|
150
|
-
- - "
|
196
|
+
- - ">="
|
151
197
|
- !ruby/object:Gem::Version
|
152
198
|
version: '10.0'
|
153
199
|
- !ruby/object:Gem::Dependency
|
154
200
|
name: rspec
|
155
201
|
requirement: !ruby/object:Gem::Requirement
|
156
202
|
requirements:
|
157
|
-
- - "
|
203
|
+
- - "<"
|
204
|
+
- !ruby/object:Gem::Version
|
205
|
+
version: '4'
|
206
|
+
- - ">="
|
158
207
|
- !ruby/object:Gem::Version
|
159
208
|
version: '3.0'
|
160
209
|
type: :development
|
161
210
|
prerelease: false
|
162
211
|
version_requirements: !ruby/object:Gem::Requirement
|
163
212
|
requirements:
|
164
|
-
- - "
|
213
|
+
- - "<"
|
214
|
+
- !ruby/object:Gem::Version
|
215
|
+
version: '4'
|
216
|
+
- - ">="
|
165
217
|
- !ruby/object:Gem::Version
|
166
218
|
version: '3.0'
|
167
219
|
- !ruby/object:Gem::Dependency
|
@@ -220,6 +272,20 @@ dependencies:
|
|
220
272
|
- - ">="
|
221
273
|
- !ruby/object:Gem::Version
|
222
274
|
version: '0'
|
275
|
+
- !ruby/object:Gem::Dependency
|
276
|
+
name: sqlite3
|
277
|
+
requirement: !ruby/object:Gem::Requirement
|
278
|
+
requirements:
|
279
|
+
- - ">="
|
280
|
+
- !ruby/object:Gem::Version
|
281
|
+
version: '0'
|
282
|
+
type: :development
|
283
|
+
prerelease: false
|
284
|
+
version_requirements: !ruby/object:Gem::Requirement
|
285
|
+
requirements:
|
286
|
+
- - ">="
|
287
|
+
- !ruby/object:Gem::Version
|
288
|
+
version: '0'
|
223
289
|
- !ruby/object:Gem::Dependency
|
224
290
|
name: ws-style
|
225
291
|
requirement: !ruby/object:Gem::Requirement
|
@@ -238,14 +304,14 @@ dependencies:
|
|
238
304
|
name: pg
|
239
305
|
requirement: !ruby/object:Gem::Requirement
|
240
306
|
requirements:
|
241
|
-
- - "
|
307
|
+
- - ">="
|
242
308
|
- !ruby/object:Gem::Version
|
243
309
|
version: '0.21'
|
244
310
|
type: :development
|
245
311
|
prerelease: false
|
246
312
|
version_requirements: !ruby/object:Gem::Requirement
|
247
313
|
requirements:
|
248
|
-
- - "
|
314
|
+
- - ">="
|
249
315
|
- !ruby/object:Gem::Version
|
250
316
|
version: '0.21'
|
251
317
|
- !ruby/object:Gem::Dependency
|
@@ -279,11 +345,14 @@ files:
|
|
279
345
|
- CHANGELOG.md
|
280
346
|
- Gemfile
|
281
347
|
- Guardfile
|
348
|
+
- LICENSE
|
282
349
|
- README.md
|
283
350
|
- Rakefile
|
351
|
+
- datadog_example.png
|
284
352
|
- lib/pii_safe_schema.rb
|
285
353
|
- lib/pii_safe_schema/annotations.rb
|
286
354
|
- lib/pii_safe_schema/configuration.rb
|
355
|
+
- lib/pii_safe_schema/invalid_column_error.rb
|
287
356
|
- lib/pii_safe_schema/migration_generator.rb
|
288
357
|
- lib/pii_safe_schema/notifiers/data_dog.rb
|
289
358
|
- lib/pii_safe_schema/notifiers/std_out.rb
|
@@ -294,7 +363,8 @@ files:
|
|
294
363
|
- lib/tasks/pii_safe_schema.rake
|
295
364
|
- pii_safe_schema.gemspec
|
296
365
|
homepage: https://github.com/wealthsimple/pii_safe_schema
|
297
|
-
licenses:
|
366
|
+
licenses:
|
367
|
+
- MIT
|
298
368
|
metadata: {}
|
299
369
|
post_install_message:
|
300
370
|
rdoc_options: []
|
@@ -304,14 +374,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
304
374
|
requirements:
|
305
375
|
- - ">="
|
306
376
|
- !ruby/object:Gem::Version
|
307
|
-
version: '
|
377
|
+
version: '2.6'
|
308
378
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
309
379
|
requirements:
|
310
380
|
- - ">="
|
311
381
|
- !ruby/object:Gem::Version
|
312
382
|
version: '0'
|
313
383
|
requirements: []
|
314
|
-
rubygems_version: 3.
|
384
|
+
rubygems_version: 3.1.4
|
315
385
|
signing_key:
|
316
386
|
specification_version: 4
|
317
387
|
summary: Schema migration tool for checking and adding comments on PII columns.
|