pii_safe_schema 1.1.0 → 1.3.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.circleci/config.yml +8 -4
- data/.rubocop.yml +4 -0
- data/.ruby-version +1 -1
- data/CHANGELOG.md +27 -0
- data/README.md +66 -13
- data/datadog_example.png +0 -0
- data/lib/pii_safe_schema.rb +56 -2
- data/lib/pii_safe_schema/annotations.rb +6 -2
- data/lib/pii_safe_schema/configuration.rb +47 -14
- data/lib/pii_safe_schema/invalid_column_error.rb +4 -0
- data/lib/pii_safe_schema/migration_generator.rb +1 -3
- data/lib/pii_safe_schema/notifiers/data_dog.rb +11 -10
- data/lib/pii_safe_schema/pii_column.rb +14 -4
- data/lib/pii_safe_schema/version.rb +1 -1
- data/lib/tasks/pii_safe_schema.rake +22 -1
- data/pii_safe_schema.gemspec +11 -6
- metadata +82 -13
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c846ca865ea611c8d178be2a5ee32ba4cca2d6b955747c72004a561c9124058a
|
4
|
+
data.tar.gz: c696456e1ef21bda144362b1b6fc1c37b87f3869ec0a370415530f9242f7a43a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fb04c0f77566544162769a4f4a83fa352c8c395fa145b3a2de65ada8f94ff80d5695c07ccefc0e62c68ae2f047e049b9040ed517e87e63cbbd0e43b9c5656055
|
7
|
+
data.tar.gz: 8816b7fc9f0446b9fecccd5a9083c785ad1524d272d4ca7d7fb470bcdd82e3fd44c7dc7ea0d9f762ed97f6a2964d5c0d85626e6307ee187fd2c5e2e3e95e3103
|
data/.circleci/config.yml
CHANGED
@@ -3,7 +3,7 @@ version: 2
|
|
3
3
|
defaults: &defaults
|
4
4
|
working_directory: /home/circleci/wealthsimple
|
5
5
|
docker:
|
6
|
-
- image: circleci/ruby:2.
|
6
|
+
- image: circleci/ruby:2.7.2
|
7
7
|
- image: circleci/postgres:9.5.9-alpine
|
8
8
|
environment:
|
9
9
|
POSTGRES_USER: circleci
|
@@ -84,12 +84,15 @@ workflows:
|
|
84
84
|
- checkout_and_bundle:
|
85
85
|
context: wealthsimple
|
86
86
|
- rspec:
|
87
|
+
context: wealthsimple
|
87
88
|
requires:
|
88
89
|
- checkout_and_bundle
|
89
90
|
- lint_check:
|
91
|
+
context: wealthsimple
|
90
92
|
requires:
|
91
93
|
- checkout_and_bundle
|
92
94
|
- vulnerability_check:
|
95
|
+
context: wealthsimple
|
93
96
|
requires:
|
94
97
|
- checkout_and_bundle
|
95
98
|
- release:
|
@@ -98,9 +101,9 @@ workflows:
|
|
98
101
|
branches:
|
99
102
|
only: master
|
100
103
|
requires:
|
101
|
-
|
102
|
-
|
103
|
-
|
104
|
+
- rspec
|
105
|
+
- lint_check
|
106
|
+
- vulnerability_check
|
104
107
|
|
105
108
|
security-audit:
|
106
109
|
triggers:
|
@@ -114,5 +117,6 @@ workflows:
|
|
114
117
|
- checkout_and_bundle:
|
115
118
|
context: wealthsimple
|
116
119
|
- vulnerability_check:
|
120
|
+
context: wealthsimple
|
117
121
|
requires:
|
118
122
|
- checkout_and_bundle
|
data/.rubocop.yml
CHANGED
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
2.
|
1
|
+
2.7.2
|
data/CHANGELOG.md
CHANGED
@@ -4,6 +4,33 @@ All notable changes to this project will be documented in this file.
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
5
5
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
6
6
|
|
7
|
+
## 1.3.3 - 2021-03-15
|
8
|
+
### Changed
|
9
|
+
- Pull CI images from ECR repository
|
10
|
+
|
11
|
+
## 1.3.2 - 2021-03-15
|
12
|
+
### Changed
|
13
|
+
- Update development to Ruby 2.7.2
|
14
|
+
|
15
|
+
## 1.3.1 - 2019-11-06
|
16
|
+
### Fixed
|
17
|
+
- Passing arguments to `rake pii_safe_schema:generate_migrations` actually works
|
18
|
+
|
19
|
+
## 1.3.0 - 2019-11-04
|
20
|
+
### Added
|
21
|
+
- Can pass explicitly annotate PII columns from the command line as arguments when using `rake pii_safe_schema:generate_migrations`.
|
22
|
+
|
23
|
+
## 1.2.0 - 2019-4-20
|
24
|
+
### Added
|
25
|
+
- Can pass Datadog Client object as a configuration option.
|
26
|
+
|
27
|
+
### Changed
|
28
|
+
- Specs use SQLite3 instead of Postgres, further unblocking local development
|
29
|
+
- README got a facelift 😍
|
30
|
+
|
31
|
+
### Fixed
|
32
|
+
- Development on Windows 10 environments now work
|
33
|
+
|
7
34
|
## 1.1.0 - 2019-4-18
|
8
35
|
### Added
|
9
36
|
- Added MIT License
|
data/README.md
CHANGED
@@ -1,28 +1,44 @@
|
|
1
|
-
|
1
|
+
# PII Safe Schema [![CircleCI](https://circleci.com/gh/wealthsimple/pii_safe_schema.svg?style=svg)](https://circleci.com/gh/wealthsimple/pii_safe_schema) [![Coverage Status](https://coveralls.io/repos/github/wealthsimple/pii_safe_schema/badge.svg?branch=master)](https://coveralls.io/github/wealthsimple/pii_safe_schema?branch=master)
|
2
2
|
|
3
|
-
|
3
|
+
Schema migration tool for checking and adding comments on *Personally Identifiable Information* (PII) columns in Rails.
|
4
|
+
|
5
|
+
Specifically, this gem serves a few functions:
|
4
6
|
|
5
7
|
* Warning you when you might be missing an annotation on a column
|
6
|
-
*
|
7
|
-
*
|
8
|
+
* Auto generating your migrations for you
|
9
|
+
* Customizable actions through Datadog Events if there are remaining unannotated columns. E.g. alerting your Security Team
|
10
|
+
|
11
|
+
![Screenshot of Datadog Event alert](datadog_example.png)
|
12
|
+
|
13
|
+
## Why
|
14
|
+
|
15
|
+
Data privacy is an ever increasing concern for users, especially if your project or business is in sensitive industries like healthcare or finance.
|
8
16
|
|
17
|
+
Having structured metadata on the database level of your application ensures Business Intelligence consumers (I.e. Periscope Data) can appropriately filter or obfuscate columns that personally identify your users without impacting business needs.
|
9
18
|
|
19
|
+
In other words, as your attack surface increases, the risk of user PII disclosure remains the same.
|
10
20
|
|
11
|
-
|
21
|
+
In your data warehousing pipeline, consume the structured metadata this gem provides in order to maintain the privacy of your users.
|
12
22
|
|
13
|
-
|
23
|
+
## Getting Started
|
14
24
|
|
15
|
-
|
25
|
+
Add your Rails project Gemfile:
|
16
26
|
|
27
|
+
```ruby
|
28
|
+
gem 'pii_safe_schema'
|
17
29
|
```
|
30
|
+
|
31
|
+
Then, to your `application.rb`
|
32
|
+
|
33
|
+
```ruby
|
18
34
|
config.after_initialize do
|
19
35
|
PiiSafeSchema.activate!
|
20
36
|
end
|
21
37
|
```
|
22
38
|
|
23
|
-
|
39
|
+
If you want to ignore certain columns, add the following initializer:
|
24
40
|
|
25
|
-
```
|
41
|
+
```ruby
|
26
42
|
# initializers/pii_safe_schema.rb
|
27
43
|
|
28
44
|
PiiSafeSchema.configure do |config|
|
@@ -30,17 +46,54 @@ PiiSafeSchema.configure do |config|
|
|
30
46
|
some_table: :*, # ignore the whole table
|
31
47
|
some_other_table: [:column_1, :column_2] # just those columns
|
32
48
|
}
|
49
|
+
|
50
|
+
# Pass whatever instance you want here, but it must implement the method
|
51
|
+
# #event(title, message, opts = {})
|
52
|
+
# which is what datadog-statsd does:
|
53
|
+
config.datadog_client = Datadog::Statsd.new(
|
54
|
+
Rails.application.secrets.fetch(:datadog_host),
|
55
|
+
Datadog::Statsd::DEFAULT_PORT,
|
56
|
+
# ...
|
57
|
+
)
|
33
58
|
end
|
34
59
|
```
|
35
60
|
|
36
|
-
|
61
|
+
## Generating Comment Migrations
|
62
|
+
|
63
|
+
```bash
|
64
|
+
rake pii_safe_schema:generate_migrations
|
65
|
+
```
|
66
|
+
|
67
|
+
This will generate one migration file for each table that should be commented. It will create a comment field for each column that it warns you about when you start a rails server or console.
|
68
|
+
|
69
|
+
### Explicit annotations
|
70
|
+
|
71
|
+
If the generator fails to identify a PII column, you can specify explicitly what columns in what tables are PII. This is particularly useful if you're installed pii_safe_schema into an existing project.
|
72
|
+
|
73
|
+
```bash
|
74
|
+
rake pii_safe_schema:generate_migrations [table:column:annotation_type] ...
|
75
|
+
```
|
76
|
+
|
77
|
+
Run `rake pii_safe_schema:generate_migrations help` for details
|
37
78
|
|
38
|
-
|
79
|
+
## Credits
|
39
80
|
|
40
|
-
|
41
|
-
it will create a comment field for each column that it warns you about when you start a rails server or console.
|
81
|
+
Thanks to [Alexi Garrow](https://github.com/AGarrow) for the original code.
|
42
82
|
|
83
|
+
## Contributing
|
43
84
|
|
85
|
+
Everyone is encouraged to help improve this project. Here are a few ways you can help:
|
44
86
|
|
87
|
+
* [Report bugs](https://github.com/wealthsimple/pii_safe_schema/issues)
|
88
|
+
* Fix bugs and [submit pull requests](https://github.com/wealthsimple/pii_safe_schema/pulls)
|
89
|
+
* Write, clarify, or fix documentation
|
90
|
+
* Suggest or add new features
|
45
91
|
|
92
|
+
To get started with development and testing:
|
46
93
|
|
94
|
+
```bash
|
95
|
+
git clone https://github.com/wealthsimple/pii_safe_schema.git
|
96
|
+
cd pii_safe_schema
|
97
|
+
bundle install
|
98
|
+
bundle exec rspec
|
99
|
+
```
|
data/datadog_example.png
ADDED
Binary file
|
data/lib/pii_safe_schema.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
require 'pii_safe_schema/invalid_column_error'
|
1
2
|
require 'pii_safe_schema/configuration'
|
2
3
|
require 'pii_safe_schema/annotations'
|
3
4
|
require 'pii_safe_schema/notify'
|
@@ -18,6 +19,10 @@ module PiiSafeSchema
|
|
18
19
|
@configuration ||= Configuration.new
|
19
20
|
end
|
20
21
|
|
22
|
+
def self.reset_configuration!
|
23
|
+
@configuration = Configuration.new
|
24
|
+
end
|
25
|
+
|
21
26
|
def self.configure
|
22
27
|
yield(configuration)
|
23
28
|
end
|
@@ -32,7 +37,56 @@ module PiiSafeSchema
|
|
32
37
|
Rails.logger.info('PiiSafeSchema: No DB'.red)
|
33
38
|
end
|
34
39
|
|
35
|
-
def self.generate_migrations
|
36
|
-
PiiSafeSchema::MigrationGenerator.generate_migrations(
|
40
|
+
def self.generate_migrations(additional_pii_columns = [])
|
41
|
+
PiiSafeSchema::MigrationGenerator.generate_migrations(
|
42
|
+
PiiSafeSchema::PiiColumn.all + additional_pii_columns,
|
43
|
+
)
|
44
|
+
end
|
45
|
+
|
46
|
+
def self.parse_additional_columns(arguments)
|
47
|
+
arguments.map do |str|
|
48
|
+
matches = /([a-z_]+):([a-z_]+):([a-z_]+)/i.match(str)
|
49
|
+
return print_help! if matches.blank?
|
50
|
+
|
51
|
+
suggestion = Annotations.comment(matches[3])
|
52
|
+
return print_help! if suggestion.blank?
|
53
|
+
|
54
|
+
PiiColumn.from_column_name(table: matches[1], column: matches[2], suggestion: suggestion)
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
def self.print_help!(do_exit: true) # rubocop:disable Metrics/MethodLength
|
59
|
+
puts <<~HELPMSG # rubocop:disable Rails/Output
|
60
|
+
Usage:
|
61
|
+
rake pii_safe_schema:generate_migrations [table:column:annotation_type] ...
|
62
|
+
|
63
|
+
Arguments:
|
64
|
+
[table:column:annotation_type] # A column to manually annotate. Can be repeated.
|
65
|
+
# annotation_type can be "email", "phone", "ip_address",
|
66
|
+
# "geolocation", "address", "postal_code", "name",
|
67
|
+
# "sensitive_data", or "encrypted_data"
|
68
|
+
|
69
|
+
Description:
|
70
|
+
Generates a migration to add PII annotation comments to appropriate columns on a table.
|
71
|
+
Uses a series of regular expressions to find sensitive fields.
|
72
|
+
|
73
|
+
Optionally supply arguments to annotate columns explicitly
|
74
|
+
|
75
|
+
Example:
|
76
|
+
rake pii_safe_schema:generate_migrations signatures:signatory_name:name signatures:landline:phone
|
77
|
+
|
78
|
+
Will generate a migration with the following, assuming automatic regex had no matches:
|
79
|
+
|
80
|
+
class ChangeCommentsInSignatures < ActiveRecord::Migration[5.2]
|
81
|
+
def change
|
82
|
+
safety_assured do
|
83
|
+
change_column :signatures, :signatory_name, :string, comment: '{"pii":{"obfuscate":"name_obfuscator"}}'
|
84
|
+
change_column :signatures, :landline, :string, comment: '{"pii":{"obfuscate":"phone_obfuscator"}}'
|
85
|
+
end
|
86
|
+
end
|
87
|
+
end
|
88
|
+
HELPMSG
|
89
|
+
|
90
|
+
exit(1) if do_exit # rubocop:disable Rails/Exit
|
37
91
|
end
|
38
92
|
end
|
@@ -70,14 +70,18 @@ module PiiSafeSchema
|
|
70
70
|
nil
|
71
71
|
end
|
72
72
|
|
73
|
+
def self.comment(annotation_type)
|
74
|
+
COLUMNS.dig(annotation_type.to_sym, :comment)
|
75
|
+
end
|
76
|
+
|
73
77
|
def apply_recommendation?(column, pii_info)
|
74
78
|
!encrypted?(column) &&
|
75
|
-
pii_info[:regexp].match(column.name) &&
|
79
|
+
pii_info[:regexp].match?(column.name) &&
|
76
80
|
column.comment != pii_info[:comment].to_json
|
77
81
|
end
|
78
82
|
|
79
83
|
def encrypted?(column)
|
80
|
-
COLUMNS[:encrypted_data][:regexp].match(column.name)
|
84
|
+
COLUMNS[:encrypted_data][:regexp].match?(column.name)
|
81
85
|
end
|
82
86
|
|
83
87
|
def apply_encrypted_recommendation?(column)
|
@@ -5,12 +5,14 @@ module PiiSafeSchema
|
|
5
5
|
ar_internal_metadata: :*,
|
6
6
|
}.freeze
|
7
7
|
|
8
|
+
KNOWN_DD_CLIENTS = %w[DataDogClient Ws::Railway::Datadog].freeze
|
9
|
+
|
8
10
|
def initialize
|
9
11
|
@user_ignore = {}
|
10
12
|
end
|
11
13
|
|
12
14
|
def ignore=(ignore_params)
|
13
|
-
|
15
|
+
validate_ignore(ignore_params)
|
14
16
|
@user_ignore = ignore_params
|
15
17
|
end
|
16
18
|
|
@@ -18,6 +20,19 @@ module PiiSafeSchema
|
|
18
20
|
@user_ignore.merge(DEFAULT_IGNORE)
|
19
21
|
end
|
20
22
|
|
23
|
+
def datadog_client=(client)
|
24
|
+
raise_config_error(:datadog_client) if client.present? && !client.respond_to?(:event)
|
25
|
+
|
26
|
+
@datadog_client = client
|
27
|
+
end
|
28
|
+
|
29
|
+
def datadog_client
|
30
|
+
@datadog_client ||=
|
31
|
+
KNOWN_DD_CLIENTS.find do |client|
|
32
|
+
client.safe_constantize if defined?(client)
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
21
36
|
def ignore_tables
|
22
37
|
ignore.select { |_k, v| v.to_s == '*' }.keys.map(&:to_s)
|
23
38
|
end
|
@@ -28,11 +43,11 @@ module PiiSafeSchema
|
|
28
43
|
|
29
44
|
private
|
30
45
|
|
31
|
-
def
|
32
|
-
raise_config_error unless ignore_params.is_a?(Hash)
|
46
|
+
def validate_ignore(ignore_params)
|
47
|
+
raise_config_error(:ignore) unless ignore_params.is_a?(Hash)
|
33
48
|
|
34
|
-
ignore_params.
|
35
|
-
raise_config_error unless valid_column_list?(ip) || ip == :*
|
49
|
+
ignore_params.each_value do |ip|
|
50
|
+
raise_config_error(:ignore) unless valid_column_list?(ip) || ip == :*
|
36
51
|
end
|
37
52
|
true
|
38
53
|
end
|
@@ -41,19 +56,37 @@ module PiiSafeSchema
|
|
41
56
|
value.is_a?(Array) && value.all? { |c| c.is_a?(Symbol) }
|
42
57
|
end
|
43
58
|
|
44
|
-
def raise_config_error
|
45
|
-
raise ConfigurationError,
|
59
|
+
def raise_config_error(problem)
|
60
|
+
raise ConfigurationError, problem
|
46
61
|
end
|
47
62
|
end
|
48
63
|
|
49
64
|
class ConfigurationError < StandardError
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
65
|
+
IGNORE_MSG = <<~HEREDOC.freeze
|
66
|
+
ignore must be a hash where the values are
|
67
|
+
symbols or arrays of symbols.
|
68
|
+
e.g. ignore = { some_table: :* } ##ignore the whole some_table
|
69
|
+
or ignore = { some_table: [:some_column, :some_other_column] }
|
70
|
+
HEREDOC
|
71
|
+
|
72
|
+
DD_CLIENT_MSG = <<~HEREDOC.freeze
|
73
|
+
Datadog client must be implement #event(title, text, opts = {})
|
74
|
+
|
75
|
+
Consider using dogstatsd-ruby gem and pass in Datadog::Statsd.new(...)
|
76
|
+
as the client.
|
77
|
+
HEREDOC
|
78
|
+
|
79
|
+
def initialize(problem)
|
80
|
+
super(
|
81
|
+
case problem
|
82
|
+
when :ignore
|
83
|
+
IGNORE_MSG
|
84
|
+
when :datadog_client
|
85
|
+
DD_CLIENT_MSG
|
86
|
+
else
|
87
|
+
problem
|
88
|
+
end
|
89
|
+
)
|
57
90
|
end
|
58
91
|
end
|
59
92
|
end
|
@@ -18,9 +18,7 @@ module PiiSafeSchema
|
|
18
18
|
migration_file = generator.create_migration_file
|
19
19
|
file_lines = File.open(migration_file, 'r').read.split("\n")
|
20
20
|
change_line = file_lines.find_index { |i| /def change/.match(i) }
|
21
|
-
new_contents = file_lines[0..change_line] +
|
22
|
-
generated_lines +
|
23
|
-
file_lines[change_line + 1..-1]
|
21
|
+
new_contents = file_lines[0..change_line] + generated_lines + file_lines[change_line + 1..]
|
24
22
|
|
25
23
|
File.open(migration_file, 'w') do |f|
|
26
24
|
f.write(new_contents.join("\n"))
|
@@ -1,17 +1,20 @@
|
|
1
1
|
module PiiSafeSchema
|
2
2
|
module Notify
|
3
3
|
module DataDog
|
4
|
-
|
4
|
+
# deprecated
|
5
|
+
KNOWN_CLIENTS = PiiSafeSchema::Configuration::KNOWN_DD_CLIENTS
|
5
6
|
|
6
7
|
class << self
|
7
8
|
def deliver(pii_column)
|
8
9
|
return unless %w[staging production development].include?(Rails.env)
|
9
|
-
return if
|
10
|
+
return if datadog_client.nil?
|
10
11
|
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
12
|
+
datadog_client.event(
|
13
|
+
'PII Annotation Warning',
|
14
|
+
message(pii_column),
|
15
|
+
msg_title: 'Unannotated PII Column',
|
16
|
+
alert_type: 'warning',
|
17
|
+
)
|
15
18
|
end
|
16
19
|
|
17
20
|
private
|
@@ -20,10 +23,8 @@ module PiiSafeSchema
|
|
20
23
|
"column #{pii_column.table}.#{pii_column.column.name} is not annotated"
|
21
24
|
end
|
22
25
|
|
23
|
-
def
|
24
|
-
|
25
|
-
return client.safe_constantize if defined?(client)
|
26
|
-
end
|
26
|
+
def datadog_client
|
27
|
+
PiiSafeSchema.configuration.datadog_client
|
27
28
|
end
|
28
29
|
end
|
29
30
|
end
|
@@ -3,10 +3,6 @@ module PiiSafeSchema
|
|
3
3
|
extend PiiSafeSchema::Annotations
|
4
4
|
attr_reader :table, :column, :suggestion
|
5
5
|
|
6
|
-
def self.all
|
7
|
-
find_and_create
|
8
|
-
end
|
9
|
-
|
10
6
|
def initialize(table:, column:, suggestion:)
|
11
7
|
@table = table.to_sym
|
12
8
|
@column = column
|
@@ -14,6 +10,20 @@ module PiiSafeSchema
|
|
14
10
|
end
|
15
11
|
|
16
12
|
class << self
|
13
|
+
def all
|
14
|
+
find_and_create
|
15
|
+
end
|
16
|
+
|
17
|
+
def from_column_name(table:, column:, suggestion:)
|
18
|
+
activerecord_column = connection.columns(table.to_s).find { |c| c.name == column.to_s }
|
19
|
+
|
20
|
+
unless activerecord_column
|
21
|
+
raise InvalidColumnError, "column \"#{column}\" does not exist for table \"#{table}\""
|
22
|
+
end
|
23
|
+
|
24
|
+
new(table: table, column: activerecord_column, suggestion: suggestion)
|
25
|
+
end
|
26
|
+
|
17
27
|
private
|
18
28
|
|
19
29
|
def find_and_create
|
@@ -1,5 +1,26 @@
|
|
1
1
|
namespace :pii_safe_schema do
|
2
2
|
task generate_migrations: :environment do
|
3
|
-
PiiSafeSchema.
|
3
|
+
PiiSafeSchema.print_help! if ARGV[2] == 'help'
|
4
|
+
|
5
|
+
if ARGV.length == 1
|
6
|
+
PiiSafeSchema.generate_migrations
|
7
|
+
else
|
8
|
+
additional_columns = PiiSafeSchema.parse_additional_columns(ARGV[1..])
|
9
|
+
PiiSafeSchema.generate_migrations(additional_columns)
|
10
|
+
end
|
11
|
+
|
12
|
+
exit(0) # forces rake to stop after this and not assume args are tasks
|
13
|
+
rescue ActiveRecord::StatementInvalid, PiiSafeSchema::InvalidColumnError => e
|
14
|
+
raise e if e.instance_of?(ActiveRecord::StatementInvalid) && e.cause.class != PG::UndefinedTable
|
15
|
+
|
16
|
+
puts <<~HEREDOC
|
17
|
+
Unable to generate PII annotation migration. Either the underlying table or column does not exist:
|
18
|
+
|
19
|
+
#{e.message}
|
20
|
+
|
21
|
+
Please create the table & columns first, running their migrations, before attempting to use the pii_safe_schema generator.
|
22
|
+
HEREDOC
|
23
|
+
|
24
|
+
exit(1) # forces rake to stop after this and not assume args are tasks
|
4
25
|
end
|
5
26
|
end
|
data/pii_safe_schema.gemspec
CHANGED
@@ -10,6 +10,8 @@ Gem::Specification.new do |s|
|
|
10
10
|
|
11
11
|
s.summary = 'Schema migration tool for checking and adding comments on PII columns.'
|
12
12
|
s.homepage = 'https://github.com/wealthsimple/pii_safe_schema'
|
13
|
+
s.license = "MIT"
|
14
|
+
s.required_ruby_version = Gem::Requirement.new(">= 2.6")
|
13
15
|
|
14
16
|
s.files = `git ls-files -z`.split("\x0").reject do |f|
|
15
17
|
f.match(%r{^(test|spec|features)/})
|
@@ -19,25 +21,28 @@ Gem::Specification.new do |s|
|
|
19
21
|
s.executables = s.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
22
|
s.require_paths = ['lib']
|
21
23
|
|
22
|
-
s.add_dependency 'activesupport', '>= 5'
|
24
|
+
s.add_dependency 'activesupport', '>= 5', '< 7'
|
23
25
|
s.add_dependency 'colorize'
|
24
|
-
s.add_dependency 'rails', '>= 5'
|
26
|
+
s.add_dependency 'rails', '>= 5', '< 7'
|
25
27
|
|
26
|
-
s.add_development_dependency 'bundler', '
|
28
|
+
s.add_development_dependency 'bundler', '>= 1.16'
|
27
29
|
s.add_development_dependency 'bundler-audit'
|
30
|
+
s.add_development_dependency 'coveralls'
|
28
31
|
s.add_development_dependency 'dogstatsd-ruby'
|
29
32
|
s.add_development_dependency 'git'
|
30
33
|
s.add_development_dependency 'guard-rspec'
|
31
34
|
s.add_development_dependency 'pry'
|
32
|
-
s.add_development_dependency '
|
33
|
-
s.add_development_dependency '
|
35
|
+
s.add_development_dependency 'rails', '>= 5.2.3', '< 7'
|
36
|
+
s.add_development_dependency 'rake', '>= 10.0'
|
37
|
+
s.add_development_dependency 'rspec', '< 4', '>= 3.0'
|
34
38
|
s.add_development_dependency 'rspec-collection_matchers'
|
35
39
|
s.add_development_dependency 'rspec-its'
|
36
40
|
s.add_development_dependency 'rubocop'
|
37
41
|
s.add_development_dependency 'simplecov'
|
42
|
+
s.add_development_dependency 'sqlite3'
|
38
43
|
s.add_development_dependency 'ws-style'
|
39
44
|
|
40
45
|
# Required by activerecord-safer_migrations
|
41
|
-
s.add_development_dependency 'pg', '
|
46
|
+
s.add_development_dependency 'pg', '>= 0.21'
|
42
47
|
s.add_development_dependency 'strong_migrations'
|
43
48
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pii_safe_schema
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Alexi Garrow
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2021-03-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -17,6 +17,9 @@ dependencies:
|
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '5'
|
20
|
+
- - "<"
|
21
|
+
- !ruby/object:Gem::Version
|
22
|
+
version: '7'
|
20
23
|
type: :runtime
|
21
24
|
prerelease: false
|
22
25
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -24,6 +27,9 @@ dependencies:
|
|
24
27
|
- - ">="
|
25
28
|
- !ruby/object:Gem::Version
|
26
29
|
version: '5'
|
30
|
+
- - "<"
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '7'
|
27
33
|
- !ruby/object:Gem::Dependency
|
28
34
|
name: colorize
|
29
35
|
requirement: !ruby/object:Gem::Requirement
|
@@ -45,6 +51,9 @@ dependencies:
|
|
45
51
|
- - ">="
|
46
52
|
- !ruby/object:Gem::Version
|
47
53
|
version: '5'
|
54
|
+
- - "<"
|
55
|
+
- !ruby/object:Gem::Version
|
56
|
+
version: '7'
|
48
57
|
type: :runtime
|
49
58
|
prerelease: false
|
50
59
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -52,18 +61,21 @@ dependencies:
|
|
52
61
|
- - ">="
|
53
62
|
- !ruby/object:Gem::Version
|
54
63
|
version: '5'
|
64
|
+
- - "<"
|
65
|
+
- !ruby/object:Gem::Version
|
66
|
+
version: '7'
|
55
67
|
- !ruby/object:Gem::Dependency
|
56
68
|
name: bundler
|
57
69
|
requirement: !ruby/object:Gem::Requirement
|
58
70
|
requirements:
|
59
|
-
- - "
|
71
|
+
- - ">="
|
60
72
|
- !ruby/object:Gem::Version
|
61
73
|
version: '1.16'
|
62
74
|
type: :development
|
63
75
|
prerelease: false
|
64
76
|
version_requirements: !ruby/object:Gem::Requirement
|
65
77
|
requirements:
|
66
|
-
- - "
|
78
|
+
- - ">="
|
67
79
|
- !ruby/object:Gem::Version
|
68
80
|
version: '1.16'
|
69
81
|
- !ruby/object:Gem::Dependency
|
@@ -80,6 +92,20 @@ dependencies:
|
|
80
92
|
- - ">="
|
81
93
|
- !ruby/object:Gem::Version
|
82
94
|
version: '0'
|
95
|
+
- !ruby/object:Gem::Dependency
|
96
|
+
name: coveralls
|
97
|
+
requirement: !ruby/object:Gem::Requirement
|
98
|
+
requirements:
|
99
|
+
- - ">="
|
100
|
+
- !ruby/object:Gem::Version
|
101
|
+
version: '0'
|
102
|
+
type: :development
|
103
|
+
prerelease: false
|
104
|
+
version_requirements: !ruby/object:Gem::Requirement
|
105
|
+
requirements:
|
106
|
+
- - ">="
|
107
|
+
- !ruby/object:Gem::Version
|
108
|
+
version: '0'
|
83
109
|
- !ruby/object:Gem::Dependency
|
84
110
|
name: dogstatsd-ruby
|
85
111
|
requirement: !ruby/object:Gem::Requirement
|
@@ -136,32 +162,58 @@ dependencies:
|
|
136
162
|
- - ">="
|
137
163
|
- !ruby/object:Gem::Version
|
138
164
|
version: '0'
|
165
|
+
- !ruby/object:Gem::Dependency
|
166
|
+
name: rails
|
167
|
+
requirement: !ruby/object:Gem::Requirement
|
168
|
+
requirements:
|
169
|
+
- - ">="
|
170
|
+
- !ruby/object:Gem::Version
|
171
|
+
version: 5.2.3
|
172
|
+
- - "<"
|
173
|
+
- !ruby/object:Gem::Version
|
174
|
+
version: '7'
|
175
|
+
type: :development
|
176
|
+
prerelease: false
|
177
|
+
version_requirements: !ruby/object:Gem::Requirement
|
178
|
+
requirements:
|
179
|
+
- - ">="
|
180
|
+
- !ruby/object:Gem::Version
|
181
|
+
version: 5.2.3
|
182
|
+
- - "<"
|
183
|
+
- !ruby/object:Gem::Version
|
184
|
+
version: '7'
|
139
185
|
- !ruby/object:Gem::Dependency
|
140
186
|
name: rake
|
141
187
|
requirement: !ruby/object:Gem::Requirement
|
142
188
|
requirements:
|
143
|
-
- - "
|
189
|
+
- - ">="
|
144
190
|
- !ruby/object:Gem::Version
|
145
191
|
version: '10.0'
|
146
192
|
type: :development
|
147
193
|
prerelease: false
|
148
194
|
version_requirements: !ruby/object:Gem::Requirement
|
149
195
|
requirements:
|
150
|
-
- - "
|
196
|
+
- - ">="
|
151
197
|
- !ruby/object:Gem::Version
|
152
198
|
version: '10.0'
|
153
199
|
- !ruby/object:Gem::Dependency
|
154
200
|
name: rspec
|
155
201
|
requirement: !ruby/object:Gem::Requirement
|
156
202
|
requirements:
|
157
|
-
- - "
|
203
|
+
- - "<"
|
204
|
+
- !ruby/object:Gem::Version
|
205
|
+
version: '4'
|
206
|
+
- - ">="
|
158
207
|
- !ruby/object:Gem::Version
|
159
208
|
version: '3.0'
|
160
209
|
type: :development
|
161
210
|
prerelease: false
|
162
211
|
version_requirements: !ruby/object:Gem::Requirement
|
163
212
|
requirements:
|
164
|
-
- - "
|
213
|
+
- - "<"
|
214
|
+
- !ruby/object:Gem::Version
|
215
|
+
version: '4'
|
216
|
+
- - ">="
|
165
217
|
- !ruby/object:Gem::Version
|
166
218
|
version: '3.0'
|
167
219
|
- !ruby/object:Gem::Dependency
|
@@ -220,6 +272,20 @@ dependencies:
|
|
220
272
|
- - ">="
|
221
273
|
- !ruby/object:Gem::Version
|
222
274
|
version: '0'
|
275
|
+
- !ruby/object:Gem::Dependency
|
276
|
+
name: sqlite3
|
277
|
+
requirement: !ruby/object:Gem::Requirement
|
278
|
+
requirements:
|
279
|
+
- - ">="
|
280
|
+
- !ruby/object:Gem::Version
|
281
|
+
version: '0'
|
282
|
+
type: :development
|
283
|
+
prerelease: false
|
284
|
+
version_requirements: !ruby/object:Gem::Requirement
|
285
|
+
requirements:
|
286
|
+
- - ">="
|
287
|
+
- !ruby/object:Gem::Version
|
288
|
+
version: '0'
|
223
289
|
- !ruby/object:Gem::Dependency
|
224
290
|
name: ws-style
|
225
291
|
requirement: !ruby/object:Gem::Requirement
|
@@ -238,14 +304,14 @@ dependencies:
|
|
238
304
|
name: pg
|
239
305
|
requirement: !ruby/object:Gem::Requirement
|
240
306
|
requirements:
|
241
|
-
- - "
|
307
|
+
- - ">="
|
242
308
|
- !ruby/object:Gem::Version
|
243
309
|
version: '0.21'
|
244
310
|
type: :development
|
245
311
|
prerelease: false
|
246
312
|
version_requirements: !ruby/object:Gem::Requirement
|
247
313
|
requirements:
|
248
|
-
- - "
|
314
|
+
- - ">="
|
249
315
|
- !ruby/object:Gem::Version
|
250
316
|
version: '0.21'
|
251
317
|
- !ruby/object:Gem::Dependency
|
@@ -282,9 +348,11 @@ files:
|
|
282
348
|
- LICENSE
|
283
349
|
- README.md
|
284
350
|
- Rakefile
|
351
|
+
- datadog_example.png
|
285
352
|
- lib/pii_safe_schema.rb
|
286
353
|
- lib/pii_safe_schema/annotations.rb
|
287
354
|
- lib/pii_safe_schema/configuration.rb
|
355
|
+
- lib/pii_safe_schema/invalid_column_error.rb
|
288
356
|
- lib/pii_safe_schema/migration_generator.rb
|
289
357
|
- lib/pii_safe_schema/notifiers/data_dog.rb
|
290
358
|
- lib/pii_safe_schema/notifiers/std_out.rb
|
@@ -295,7 +363,8 @@ files:
|
|
295
363
|
- lib/tasks/pii_safe_schema.rake
|
296
364
|
- pii_safe_schema.gemspec
|
297
365
|
homepage: https://github.com/wealthsimple/pii_safe_schema
|
298
|
-
licenses:
|
366
|
+
licenses:
|
367
|
+
- MIT
|
299
368
|
metadata: {}
|
300
369
|
post_install_message:
|
301
370
|
rdoc_options: []
|
@@ -305,14 +374,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
305
374
|
requirements:
|
306
375
|
- - ">="
|
307
376
|
- !ruby/object:Gem::Version
|
308
|
-
version: '
|
377
|
+
version: '2.6'
|
309
378
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
310
379
|
requirements:
|
311
380
|
- - ">="
|
312
381
|
- !ruby/object:Gem::Version
|
313
382
|
version: '0'
|
314
383
|
requirements: []
|
315
|
-
rubygems_version: 3.
|
384
|
+
rubygems_version: 3.1.4
|
316
385
|
signing_key:
|
317
386
|
specification_version: 4
|
318
387
|
summary: Schema migration tool for checking and adding comments on PII columns.
|