attr_masker 0.2.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,5 +1,9 @@
1
1
  == Not released yet
2
2
 
3
+ * Drop support for Rails < 4.2
4
+ * Allow masking attributes which span on more than one column (or field),
5
+ supersede `column_name` option with `column_names`
6
+
3
7
  == 0.2.1
4
8
 
5
9
  * Bugfix: preload application to find all models
data/Gemfile CHANGED
@@ -1,3 +1,8 @@
1
1
  source "https://rubygems.org"
2
2
 
3
3
  gemspec
4
+
5
+ gem "mongoid", require: false
6
+
7
+ gem "database_cleaner-active_record", require: false
8
+ gem "database_cleaner-mongoid", require: false
@@ -3,18 +3,33 @@
3
3
  :pygments-style: native
4
4
  :pygments-linenums-mode: inline
5
5
 
6
- image:https://img.shields.io/gem/v/attr_masker.svg["Gem Version", link="https://rubygems.org/gems/attr_masker"]
7
- image:https://img.shields.io/travis/riboseinc/attr_masker/master.svg["Build Status", link="https://travis-ci.org/riboseinc/attr_masker"]
6
+ ifdef::env-github[]
7
+ image:https://img.shields.io/gem/v/attr_masker[
8
+ "Gem Version",
9
+ link="https://rubygems.org/gems/attr_masker"]
10
+ image:https://img.shields.io/github/workflow/status/riboseinc/attr_masker/Tests[
11
+ "Build Status",
12
+ link="https://github.com/riboseinc/attr_masker/actions"]
13
+ image:https://img.shields.io/codeclimate/maintainability/riboseinc/attr_masker[
14
+ "Code Climate",
15
+ link="https://codeclimate.com/github/riboseinc/attr_masker"]
16
+ image:https://img.shields.io/codecov/c/github/riboseinc/attr_masker[
17
+ "Test Coverage",
18
+ link="https://codecov.io/gh/riboseinc/attr_masker"]
19
+ image:https://img.shields.io/badge/documentation-rdoc-informational[
20
+ "Documentation on RubyDoc.info",
21
+ link="https://rubydoc.info/gems/attr_masker"]
22
+ endif::[]
8
23
 
9
24
  Mask ActiveRecord/Mongoid data with ease!
10
25
 
11
26
  == Introduction
12
27
 
13
28
  This gem is intended to mask sensitive data so that production database dumps
14
- can be used in staging or test environments. It works with Rails 4+ and modern
15
- Rubies. It supports Active Record and Mongoid models.
29
+ can be used in staging or test environments. It works with Rails 4.2+ and
30
+ modern Rubies. It supports Active Record and Mongoid models.
16
31
 
17
- == Getting started
32
+ == Usage instructions
18
33
 
19
34
  === Installation
20
35
 
@@ -71,31 +86,9 @@ attr_masker :first_name, :if => :tester_user?
71
86
 
72
87
  The ActiveRecord's `::default_scope` method has no effect on masking. All
73
88
  table records are updated, provided that :if and :unless filters allow that.
74
- For example, if you're using a Paranoia[https://github.com/rubysherpas/paranoia]
89
+ For example, if you're using a https://github.com/rubysherpas/paranoia[Paranoia]
75
90
  gem to soft-delete your data, records marked as deleted will be masked as well.
76
91
 
77
- === Using custom maskers
78
-
79
- By default, data is maksed with `AttrMasker::Maskers::Simple` masker which
80
- always returns `"(redacted)"` string. But anything what responds to `#call`
81
- can be used instead: a lambda, `Method` instance, and more. You can specify it
82
- by setting the `:masker` option.
83
-
84
- For instance, you may want to use https://github.com/ffaker/ffaker[ffaker] or
85
- https://github.com/skalee/well_read_faker[Well Read Faker] to generate random
86
- replacement values:
87
-
88
- [source,ruby]
89
- ----
90
- require "ffaker"
91
-
92
- attr_masker :first_name, :masker => ->(_hash) { FFaker::Name.first_name }
93
- ----
94
-
95
- A hash is passed as an argument, which includes some information about the
96
- record being masked and the attribute value. It can be used to further
97
- customize masker's behaviour.
98
-
99
92
  === Built-in maskers
100
93
 
101
94
  Attr Masker comes with several built-in maskers.
@@ -126,7 +119,7 @@ Can be initialized with options.
126
119
  |===============================================================================
127
120
  |Name|Default|Description
128
121
  |`replacement`|`"*"`|Replacement string, can be empty.
129
- |`alphanum_only`|`false`|When true, only alphanumeric charaters are replaced.
122
+ |`alphanum_only`|`false`|When true, only alphanumeric characters are replaced.
130
123
  |===============================================================================
131
124
  +
132
125
  Example:
@@ -139,7 +132,39 @@ attr_masker :phone, :masker => rm
139
132
  +
140
133
  Would mask "123-456-7890" as "XXX-XXX-XXXX".
141
134
 
135
+ === Using custom maskers
136
+
137
+ Apart from built-in maskers, any object which responds to `#call` can be used,
138
+ e.g. some lambda or `Method` instance. For instance, you may want to produce
139
+ unique values basing on other attributes, to mask selectively, or to use
140
+ tool like https://github.com/skalee/well_read_faker[Well Read Faker] to
141
+ generate random replacement values:
142
+
143
+ [source,ruby]
144
+ ----
145
+ require "well_read_faker"
146
+
147
+ attr_masker :email, masker: ->(model:, **) { "user#{model.id}@example.com" }
148
+ attr_masker :phone, masker: ->(value:, **) { "******" + value[-3..-1] }
149
+ attr_masker :bio, masker: ->(**) { WellReadFaker.paragraph }
150
+ ----
151
+
152
+ Masker is called with following keyword arguments:
153
+
154
+ `value`:: Original value of the field which is about to be masked
155
+
156
+ `model`:: Model instance
157
+
158
+ `attribute_name`:: Name of the attribute which is about to be masked
159
+
160
+ `masking_options`:: Hash of options which were passed in `#attr_masker` call
161
+
162
+ This list is likely to be extended in future versions, and that will not be
163
+ considered a breaking change, therefore it is strongly recommended to always
164
+ use a splat (`**`) at end of argument list of masker's `#call` method.
165
+
142
166
  == Roadmap & TODOs
167
+
143
168
  - documentation
144
169
  - spec tests
145
170
  - Make the `Rails.env` (in which `db:mask` could be run) configurable
@@ -1,13 +1,13 @@
1
1
  # (c) 2017 Ribose Inc.
2
2
  #
3
3
 
4
- lib = File.expand_path("../lib", __FILE__)
4
+ lib = File.expand_path("lib", __dir__)
5
5
  $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
6
6
  require "attr_masker/version"
7
7
 
8
8
  Gem::Specification.new do |gem|
9
9
  gem.name = "attr_masker"
10
- gem.version = AttrMasker::Version.string
10
+ gem.version = AttrMasker::VERSION
11
11
  gem.authors = ["Ribose Inc."]
12
12
  gem.email = ["open.source@ribose.com"]
13
13
  gem.homepage = "https://github.com/riboseinc/attr_masker"
@@ -17,20 +17,23 @@ Gem::Specification.new do |gem|
17
17
  "of certain models by modifying the database."
18
18
 
19
19
  gem.files = `git ls-files`.split($/)
20
- gem.executables = gem.files.grep(%r{^bin/}).map { |f| File.basename(f) }
21
20
  gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
22
21
  gem.require_paths = ["lib"]
23
22
 
24
- gem.add_runtime_dependency("rails", ">= 4.0.0", "< 6.0")
23
+ gem.add_runtime_dependency("rails", ">= 4.0.0", "< 7")
25
24
  gem.add_runtime_dependency("ruby-progressbar", "~> 1.8")
26
25
 
27
- gem.add_development_dependency("bundler", "~> 1.15")
28
- gem.add_development_dependency("combustion", "~> 0.7.0")
29
- gem.add_development_dependency("database_cleaner", "~> 1.6")
26
+ gem.add_development_dependency("bundler", ">= 1.15")
27
+ gem.add_development_dependency("combustion", "~> 1.0")
28
+ gem.add_development_dependency("database_cleaner", "~> 1.8")
29
+ gem.add_development_dependency("database_cleaner-active_record", "~> 1.8")
30
+ gem.add_development_dependency("database_cleaner-mongoid", "~> 1.8")
30
31
  # Older versions aren't needed as we don't support Rails < 4
31
32
  gem.add_development_dependency("mongoid", ">= 5")
32
33
  gem.add_development_dependency("pry")
33
- gem.add_development_dependency("rspec", ">= 3.0")
34
- gem.add_development_dependency("rubocop", "~> 0.49.1")
35
- gem.add_development_dependency("sqlite3", "~> 1.3.13")
34
+ gem.add_development_dependency("rspec", "~> 3.0")
35
+ gem.add_development_dependency("rubocop", "~> 0.54.0")
36
+ gem.add_development_dependency("simplecov")
37
+ gem.add_development_dependency("sqlite3", ">= 1.3.13", "< 2")
38
+ gem.add_development_dependency("warning", "~> 1.1")
36
39
  end
data/bin/rake CHANGED
@@ -12,7 +12,18 @@ require "pathname"
12
12
  ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
13
13
  Pathname.new(__FILE__).realpath)
14
14
 
15
+ bundle_binstub = File.expand_path("../bundle", __FILE__)
16
+
17
+ if File.file?(bundle_binstub)
18
+ if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
19
+ load(bundle_binstub)
20
+ else
21
+ abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
22
+ Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
23
+ end
24
+ end
25
+
15
26
  require "rubygems"
16
27
  require "bundler/setup"
17
28
 
18
- load Gem.bin_path("rake", "rake")
29
+ load Gem.bin_path("rake", "rake") if File.exists?(Gem.bin_path("rake", "rake"))
data/bin/rspec CHANGED
@@ -12,6 +12,17 @@ require "pathname"
12
12
  ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
13
13
  Pathname.new(__FILE__).realpath)
14
14
 
15
+ bundle_binstub = File.expand_path("../bundle", __FILE__)
16
+
17
+ if File.file?(bundle_binstub)
18
+ if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
19
+ load(bundle_binstub)
20
+ else
21
+ abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
22
+ Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
23
+ end
24
+ end
25
+
15
26
  require "rubygems"
16
27
  require "bundler/setup"
17
28
 
@@ -12,6 +12,17 @@ require "pathname"
12
12
  ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
13
13
  Pathname.new(__FILE__).realpath)
14
14
 
15
+ bundle_binstub = File.expand_path("../bundle", __FILE__)
16
+
17
+ if File.file?(bundle_binstub)
18
+ if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
19
+ load(bundle_binstub)
20
+ else
21
+ abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
22
+ Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
23
+ end
24
+ end
25
+
15
26
  require "rubygems"
16
27
  require "bundler/setup"
17
28
 
@@ -1,3 +1,4 @@
1
- eval File.read "gemfiles/common.gemfile"
1
+ eval_gemfile "common.gemfile"
2
2
 
3
3
  gem "activerecord", "~> 4.2.0"
4
+ gem "sqlite3", "~> 1.3.13"
@@ -1,3 +1,4 @@
1
- eval File.read "gemfiles/common.gemfile"
1
+ eval_gemfile "common.gemfile"
2
2
 
3
3
  gem "activerecord", "~> 5.0.0"
4
+ gem "sqlite3", "~> 1.3.13"
@@ -1,3 +1,4 @@
1
- eval File.read "gemfiles/common.gemfile"
1
+ eval_gemfile "common.gemfile"
2
2
 
3
3
  gem "activerecord", "~> 5.1.0"
4
+ gem "sqlite3", "~> 1.3.13"
@@ -0,0 +1,4 @@
1
+ eval_gemfile "common.gemfile"
2
+
3
+ gem "activerecord", "~> 5.2.0"
4
+ gem "sqlite3", "~> 1.3.13"
@@ -0,0 +1,3 @@
1
+ eval_gemfile "common.gemfile"
2
+
3
+ gem "activerecord", "~> 6.0.0"
@@ -1,4 +1,4 @@
1
- eval File.read "gemfiles/common.gemfile"
1
+ eval_gemfile "common.gemfile"
2
2
 
3
3
  gem "activerecord", github: "rails/rails"
4
4
  gem "arel", github: "rails/arel"
@@ -1,5 +1,4 @@
1
- source "https://rubygems.org"
1
+ eval_gemfile "../Gemfile"
2
2
 
3
- gemspec path: ".."
4
-
5
- gem "mongoid", require: false
3
+ gem "codecov", require: false
4
+ gem "simplecov", require: false
@@ -13,7 +13,7 @@ module AttrMasker
13
13
  end
14
14
 
15
15
  # Evaluates the +:if+ and +:unless+ attribute options on given instance.
16
- # Returns +true+ or +fasle+, depending on whether the attribute should be
16
+ # Returns +true+ or +false+, depending on whether the attribute should be
17
17
  # masked for this object or not.
18
18
  def should_mask?(model_instance)
19
19
  not (
@@ -34,8 +34,16 @@ module AttrMasker
34
34
  # returning.
35
35
  def mask(model_instance)
36
36
  value = unmarshal_data(model_instance.send(name))
37
- masker_value = options[:masker].call(options.merge!(value: value))
38
- marshal_data(masker_value)
37
+ masker = options[:masker]
38
+ masker_value = masker.call(value: value, model: model_instance,
39
+ attribute_name: name, masking_options: options)
40
+ model_instance.send("#{name}=", marshal_data(masker_value))
41
+ end
42
+
43
+ # Returns a hash of maskable attribute names, and respective attribute
44
+ # values. Unchanged attributes are skipped.
45
+ def masked_attributes_new_values(model_instance)
46
+ model_instance.changes.slice(*column_names).transform_values(&:second)
39
47
  end
40
48
 
41
49
  # Evaluates option (typically +:if+ or +:unless+) on given model instance.
@@ -55,16 +63,18 @@ module AttrMasker
55
63
 
56
64
  def marshal_data(data)
57
65
  return data unless options[:marshal]
66
+
58
67
  options[:marshaler].send(options[:dump_method], data)
59
68
  end
60
69
 
61
70
  def unmarshal_data(data)
62
71
  return data unless options[:marshal]
72
+
63
73
  options[:marshaler].send(options[:load_method], data)
64
74
  end
65
75
 
66
- def column_name
67
- options[:column_name] || name
76
+ def column_names
77
+ options[:column_names] || [name]
68
78
  end
69
79
  end
70
80
  end
@@ -2,14 +2,30 @@
2
2
  #
3
3
  module AttrMasker
4
4
  module Maskers
5
- # This default masker simply replaces any value with a fixed string.
5
+ # +Replacing+ masker replaces every character of string which is being
6
+ # masked with +replacement+ one, preserving the length of the masked string
7
+ # (provided that a replacement string contains a single character, which is
8
+ # a typical case). Optionally, non-alphanumeric characters like dashes or
9
+ # spaces may be left unchanged.
6
10
  #
7
- # +opts+ is a Hash with the key :value that gives you the current attribute
8
- # value.
11
+ # @example Would mask "Adam West" as "XXXXXXXXX"
12
+ # class User < ActiveRecord::Base
13
+ # m = AttrMasker::Maskers::Replacing.new(replacement: "X")
14
+ # attr_masker :name, :masker => m
15
+ # end
9
16
  #
17
+ # @example Would mask "123-456-789" as "XXX-XXX-XXX"
18
+ # class User < ActiveRecord::Base
19
+ # m = AttrMasker::Maskers::Replacing.new(
20
+ # replacement: "X", alphanum_only: true)
21
+ # attr_masker :phone, :masker => m
22
+ # end
10
23
  class Replacing
11
24
  attr_reader :replacement, :alphanum_only
12
25
 
26
+ # @param replacement [String] replacement string
27
+ # @param alphanum_only [Boolean] whether to leave non-alphanumeric
28
+ # characters unchanged or not
13
29
  def initialize(replacement: "*", alphanum_only: false)
14
30
  replacement = "" if replacement.nil?
15
31
  @replacement = replacement
@@ -1,12 +1,22 @@
1
1
  module AttrMasker
2
2
  module Maskers
3
- # This default masker simply replaces any value with a fixed string.
3
+ # +Simple+ masker replaces values with a predefined +(redacted)+ string.
4
+ # This is a default masker, which is used when no specific +:masker+ is
5
+ # passed in +attr_masker+ method call.
4
6
  #
5
- # +opts+ is a Hash with the key :value that gives you the current attribute
6
- # value.
7
+ # @example Would mask "Adam West" as "(redacted)"
8
+ # class User < ActiveRecord::Base
9
+ # m = AttrMasker::Maskers::Simple.new
10
+ # attr_masker :name, :masker => m
11
+ # end
7
12
  #
13
+ # @example Would mask "Adam West" as "(redacted)"
14
+ # class User < ActiveRecord::Base
15
+ # attr_masker :name
16
+ # end
8
17
  class Simple
9
- def call(_opts)
18
+ # Accepts any keyword arguments, but they all are ignored.
19
+ def call(**_opts)
10
20
  "(redacted)"
11
21
  end
12
22
  end
@@ -73,6 +73,8 @@ module AttrMasker
73
73
  # @user.masker_configuration # returns the masker version of configuration
74
74
  #
75
75
  # See README for more examples
76
+ #--
77
+ # rubocop:disable Metrics/MethodLength
76
78
  def attr_masker(*args)
77
79
  default_options = {
78
80
  if: true,
@@ -94,6 +96,7 @@ module AttrMasker
94
96
  masker_attributes[attribute.name] = attribute
95
97
  end
96
98
  end
99
+ # rubocop:enable Metrics/MethodLength
97
100
 
98
101
  # Default options to use with calls to <tt>attr_masker</tt>
99
102
  # XXX:Keep
@@ -1,5 +1,6 @@
1
1
  # (c) 2017 Ribose Inc.
2
2
  #
3
+
3
4
  module AttrMasker
4
5
  module Performer
5
6
  class Base
@@ -7,25 +8,40 @@ module AttrMasker
7
8
  # Do not want production environment to be masked!
8
9
  #
9
10
  if Rails.env.production?
10
- raise AttrMasker::Error, "Attempted to run in production environment."
11
+ unless ENV["FORCE_MASK"]
12
+ msg = "Attempted to run in production environment."
13
+ raise AttrMasker::Error, msg
14
+ end
11
15
  end
12
16
 
13
17
  all_models.each do |klass|
14
18
  next if klass.masker_attributes.empty?
19
+
15
20
  mask_class(klass)
16
21
  end
17
22
  end
18
23
 
19
24
  private
20
25
 
26
+ # Mask all objects of a class in batches to not run out of memory!
27
+ #--
28
+ # rubocop:todo Metrics/MethodLength
21
29
  def mask_class(klass)
22
30
  progressbar_for_model(klass) do |bar|
23
- klass.all.unscoped.each do |model|
24
- mask_object model
25
- bar.increment
31
+ if klass.all.unscoped.respond_to?(:find_each)
32
+ klass.all.unscoped.find_each(batch_size: 1000) do |model|
33
+ mask_object model
34
+ bar.increment
35
+ end
36
+ else
37
+ klass.all.unscoped.each do |model|
38
+ mask_object model
39
+ bar.increment
40
+ end
26
41
  end
27
42
  end
28
43
  end
44
+ # rubocop:enable Metrics/MethodLength
29
45
 
30
46
  # For each masker attribute, mask it, and save it!
31
47
  #
@@ -35,9 +51,8 @@ module AttrMasker
35
51
  updates = klass.masker_attributes.values.reduce({}) do |acc, attribute|
36
52
  next acc unless attribute.should_mask?(instance)
37
53
 
38
- column_name = attribute.column_name
39
- masker_value = attribute.mask(instance)
40
- acc.merge!(column_name => masker_value)
54
+ attribute.mask(instance)
55
+ acc.merge! attribute.masked_attributes_new_values(instance)
41
56
  end
42
57
 
43
58
  make_update instance, updates unless updates.empty?
@@ -48,7 +63,7 @@ module AttrMasker
48
63
  title: klass.name,
49
64
  total: klass.unscoped.count,
50
65
  throttle_rate: 0.1,
51
- format: %q[%t %c/%C (%j%%) %B %E],
66
+ format: "%t %c/%C (%j%%) %B %E",
52
67
  )
53
68
 
54
69
  yield bar
@@ -66,9 +81,12 @@ module AttrMasker
66
81
  ::ActiveRecord::Base.descendants.select(&:table_exists?)
67
82
  end
68
83
 
84
+ #--
85
+ # rubocop:disable Rails/SkipsModelValidations
69
86
  def make_update(instance, updates)
70
- instance.class.all.unscoped.update(instance.id, updates)
87
+ instance.class.all.unscoped.where(id: instance.id).update_all(updates)
71
88
  end
89
+ # rubocop:enable Rails/SkipsModelValidations
72
90
  end
73
91
 
74
92
  class Mongoid < Base