data-anonymization 0.8.1 → 0.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +21 -9
- data/data-anonymization.gemspec +5 -5
- data/lib/strategy/field/string/random_url.rb +1 -1
- data/lib/strategy/field/string/select_from_database.rb +14 -7
- data/lib/version.rb +1 -1
- metadata +12 -12
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ea71406f4d69385841297c63312abbfe0a1ded34
|
4
|
+
data.tar.gz: 6309e4f2155021c42c80aa561dc5ddd2cd4999f7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6e3fde8664f03c76c8939e14899dc0ded79559e6b016ebdc966b2a6095964b3f1888286f4c09a0a42226d4a475a905e8249f5a9463530c87592455432f36a4f3
|
7
|
+
data.tar.gz: 407caf0cec3c9962efa62272493b80cdf65acdcd49c33e799546419cad94157456733fad89a28f0c8523e1d92902193e2511f6d1fff98dff4a1194d68b4fc45c
|
data/README.md
CHANGED
@@ -1,6 +1,17 @@
|
|
1
1
|
# Data::Anonymization
|
2
2
|
Afraid of using production data due to privacy issues? Data Anonymization is a tool that helps you build anonymized production data dumps which you can use for performance testing, security testing, debugging and development.
|
3
3
|
|
4
|
+
## Java/Kotlin version
|
5
|
+
|
6
|
+
Java/Kotlin version of tool supporting RDBMS databases is available with similar easy to use DSL.
|
7
|
+
* [Kotlin/Java Data Anonymization Tool](https://github.com/dataanon/data-anon)
|
8
|
+
* [Kotlin Maven Sample Project](https://github.com/dataanon/dataanon-kotlin-sample)
|
9
|
+
* [Java Maven Sample Project](https://github.com/dataanon/dataanon-java-sample)
|
10
|
+
|
11
|
+
|
12
|
+
----------------------
|
13
|
+
|
14
|
+
|
4
15
|
[<img src="https://secure.travis-ci.org/sunitparekh/data-anonymization.png?branch=master">](http://travis-ci.org/sunitparekh/data-anonymization)
|
5
16
|
[<img src="https://gemnasium.com/sunitparekh/data-anonymization.png?travis">](https://gemnasium.com/sunitparekh/data-anonymization)
|
6
17
|
[<img src="https://codeclimate.com/badge.png">](https://codeclimate.com/github/sunitparekh/data-anonymization)
|
@@ -71,25 +82,25 @@ Postgresql database having **composite primary key**
|
|
71
82
|
## Changelog
|
72
83
|
|
73
84
|
#### 0.8.1 (Aug 19, 2017)
|
74
|
-
1. Multi-threading support added by [stanislav-tyutin](https://github.com/stanislav-tyutin) using Pull Request.
|
85
|
+
1. Multi-threading support added by [stanislav-tyutin](https://github.com/stanislav-tyutin) using Pull Request.
|
75
86
|
2. Fixed to work with Ruby 2.4.x, issue with Integer data type
|
76
87
|
|
77
88
|
#### 0.8.0 (Oct 31, 2016)
|
78
|
-
1. Upgraded to rails 5.x
|
89
|
+
1. Upgraded to rails 5.x
|
79
90
|
|
80
91
|
#### 0.7.4 (Oct 29, 2016)
|
81
|
-
1. Continue to work on rails 4.x. Minor changes based on feedback.
|
92
|
+
1. Continue to work on rails 4.x. Minor changes based on feedback.
|
82
93
|
|
83
94
|
#### 0.8.0.rc1 (Sep 5, 2016)
|
84
|
-
1. Upgraded to rails 5.0, please report any issue or use case not working.
|
95
|
+
1. Upgraded to rails 5.0, please report any issue or use case not working.
|
85
96
|
|
86
97
|
#### 0.7.3 (Feb 5, 2016)
|
87
|
-
1. Fixed issue with batchsize. Thanks to [Jan Raasch](https://github.com/janraasch) for sending pull request.
|
98
|
+
1. Fixed issue with batchsize. Thanks to [Jan Raasch](https://github.com/janraasch) for sending pull request.
|
88
99
|
|
89
100
|
#### 0.7.2 (Sep 26, 2015)
|
90
101
|
1. Upgraded MongoDB to latest gem version 2.1.0 and tested with MongoDB 3.x version.
|
91
|
-
2. Upgraded gems to latest version
|
92
|
-
3. Adding limit functionality - Merge pull request #27 from yanismydj/master
|
102
|
+
2. Upgraded gems to latest version
|
103
|
+
3. Adding limit functionality - Merge pull request #27 from yanismydj/master
|
93
104
|
|
94
105
|
#### 0.7.1 (Jun 13, 2015)
|
95
106
|
1. Fixed issues with empty array data for MongoDB
|
@@ -228,6 +239,7 @@ Read more about [blacklist and whitelist here](http://sunitspace.blogspot.in/201
|
|
228
239
|
5. Make sure to give proper case for fields and table names.
|
229
240
|
6. Use skip and continue to apply different strategies for records.
|
230
241
|
7. Use 'limit' to limit the number of rows that will be imported in whitelist
|
242
|
+
8. RDBMS databases utilizing schemas can be specified via `schema_search_path`: `source_db { ... schema_search_path: 'public,my_special_schema' }`
|
231
243
|
|
232
244
|
## DSL Generation
|
233
245
|
|
@@ -273,7 +285,7 @@ datanon generate_rdbms_dsl -a postgresql -h 123.456.7.8 -d production_db
|
|
273
285
|
|
274
286
|
```
|
275
287
|
|
276
|
-
The relevant db gems must be installed so that AR has the adapters required to establish the connection to the databases. The script generates a file named **rdbms_whitelist_generated.rb** in the same location as the project.
|
288
|
+
The relevant db gems must be installed so that AR has the adapters required to establish the connection to the databases. The script generates a file named **rdbms_whitelist_generated.rb** in the same location as the project.
|
277
289
|
|
278
290
|
### MongoDB whitelist generation
|
279
291
|
|
@@ -378,7 +390,7 @@ has following attribute accessor
|
|
378
390
|
</tr>
|
379
391
|
<tr>
|
380
392
|
<td align="left">Text</td>
|
381
|
-
<td align="left"><a href="http://rubydoc.info/github/sunitparekh/data-anonymization/DataAnon/Strategy/Field/RandomUrl">
|
393
|
+
<td align="left"><a href="http://rubydoc.info/github/sunitparekh/data-anonymization/DataAnon/Strategy/Field/RandomUrl">RandomUrl</a></td>
|
382
394
|
<td align="left">Anonymizes a URL while mainting the structure</td>
|
383
395
|
</tr>
|
384
396
|
</table><table>
|
data/data-anonymization.gemspec
CHANGED
@@ -19,11 +19,11 @@ Gem::Specification.new do |gem|
|
|
19
19
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
20
20
|
gem.require_paths = ['lib']
|
21
21
|
|
22
|
-
gem.add_dependency('activerecord', '~> 5.
|
23
|
-
gem.add_dependency('composite_primary_keys', '~>
|
24
|
-
gem.add_dependency('activesupport', '~> 5.
|
25
|
-
gem.add_dependency('rgeo', '~> 0
|
26
|
-
gem.add_dependency('rgeo-geojson', '~> 0
|
22
|
+
gem.add_dependency('activerecord', '~> 5.1')
|
23
|
+
gem.add_dependency('composite_primary_keys', '~> 10.0')
|
24
|
+
gem.add_dependency('activesupport', '~> 5.1')
|
25
|
+
gem.add_dependency('rgeo', '~> 1.0')
|
26
|
+
gem.add_dependency('rgeo-geojson', '~> 2.0')
|
27
27
|
gem.add_dependency('powerbar', '~> 1.0')
|
28
28
|
gem.add_dependency('parallel', '~> 1.12')
|
29
29
|
gem.add_dependency('thor', '~> 0.20')
|
@@ -12,16 +12,23 @@ module DataAnon
|
|
12
12
|
include Utils::Logging
|
13
13
|
|
14
14
|
def initialize table_name, field_name, connection_spec
|
15
|
-
|
16
|
-
|
17
|
-
@
|
18
|
-
logger.debug "For field strategy #{table_name}:#{field_name} using values #{@values} "
|
19
|
-
|
15
|
+
@table_name = table_name
|
16
|
+
@field_name = field_name
|
17
|
+
@connection_spec = connection_spec
|
20
18
|
end
|
21
19
|
|
22
|
-
|
23
|
-
|
20
|
+
def anonymize field
|
21
|
+
@values ||= begin
|
22
|
+
DataAnon::Utils::SourceDatabase.establish_connection @connection_spec
|
23
|
+
source = Utils::SourceTable.create @table_name, []
|
24
|
+
values = source.select(@field_name).distinct.collect { |record| record[@field_name]}
|
25
|
+
logger.debug "For field strategy #{@table_name}:#{@field_name} using values #{values} "
|
26
|
+
values
|
27
|
+
end
|
24
28
|
|
29
|
+
super
|
30
|
+
end
|
31
|
+
end
|
25
32
|
end
|
26
33
|
end
|
27
34
|
end
|
data/lib/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: data-anonymization
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.8.
|
4
|
+
version: 0.8.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Sunit Parekh
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2018-03-07 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: activerecord
|
@@ -18,70 +18,70 @@ dependencies:
|
|
18
18
|
requirements:
|
19
19
|
- - "~>"
|
20
20
|
- !ruby/object:Gem::Version
|
21
|
-
version: '5.
|
21
|
+
version: '5.1'
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
24
|
version_requirements: !ruby/object:Gem::Requirement
|
25
25
|
requirements:
|
26
26
|
- - "~>"
|
27
27
|
- !ruby/object:Gem::Version
|
28
|
-
version: '5.
|
28
|
+
version: '5.1'
|
29
29
|
- !ruby/object:Gem::Dependency
|
30
30
|
name: composite_primary_keys
|
31
31
|
requirement: !ruby/object:Gem::Requirement
|
32
32
|
requirements:
|
33
33
|
- - "~>"
|
34
34
|
- !ruby/object:Gem::Version
|
35
|
-
version: '
|
35
|
+
version: '10.0'
|
36
36
|
type: :runtime
|
37
37
|
prerelease: false
|
38
38
|
version_requirements: !ruby/object:Gem::Requirement
|
39
39
|
requirements:
|
40
40
|
- - "~>"
|
41
41
|
- !ruby/object:Gem::Version
|
42
|
-
version: '
|
42
|
+
version: '10.0'
|
43
43
|
- !ruby/object:Gem::Dependency
|
44
44
|
name: activesupport
|
45
45
|
requirement: !ruby/object:Gem::Requirement
|
46
46
|
requirements:
|
47
47
|
- - "~>"
|
48
48
|
- !ruby/object:Gem::Version
|
49
|
-
version: '5.
|
49
|
+
version: '5.1'
|
50
50
|
type: :runtime
|
51
51
|
prerelease: false
|
52
52
|
version_requirements: !ruby/object:Gem::Requirement
|
53
53
|
requirements:
|
54
54
|
- - "~>"
|
55
55
|
- !ruby/object:Gem::Version
|
56
|
-
version: '5.
|
56
|
+
version: '5.1'
|
57
57
|
- !ruby/object:Gem::Dependency
|
58
58
|
name: rgeo
|
59
59
|
requirement: !ruby/object:Gem::Requirement
|
60
60
|
requirements:
|
61
61
|
- - "~>"
|
62
62
|
- !ruby/object:Gem::Version
|
63
|
-
version: '0
|
63
|
+
version: '1.0'
|
64
64
|
type: :runtime
|
65
65
|
prerelease: false
|
66
66
|
version_requirements: !ruby/object:Gem::Requirement
|
67
67
|
requirements:
|
68
68
|
- - "~>"
|
69
69
|
- !ruby/object:Gem::Version
|
70
|
-
version: '0
|
70
|
+
version: '1.0'
|
71
71
|
- !ruby/object:Gem::Dependency
|
72
72
|
name: rgeo-geojson
|
73
73
|
requirement: !ruby/object:Gem::Requirement
|
74
74
|
requirements:
|
75
75
|
- - "~>"
|
76
76
|
- !ruby/object:Gem::Version
|
77
|
-
version: '0
|
77
|
+
version: '2.0'
|
78
78
|
type: :runtime
|
79
79
|
prerelease: false
|
80
80
|
version_requirements: !ruby/object:Gem::Requirement
|
81
81
|
requirements:
|
82
82
|
- - "~>"
|
83
83
|
- !ruby/object:Gem::Version
|
84
|
-
version: '0
|
84
|
+
version: '2.0'
|
85
85
|
- !ruby/object:Gem::Dependency
|
86
86
|
name: powerbar
|
87
87
|
requirement: !ruby/object:Gem::Requirement
|