data-anonymization 0.8.1 → 0.8.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +21 -9
- data/data-anonymization.gemspec +5 -5
- data/lib/strategy/field/string/random_url.rb +1 -1
- data/lib/strategy/field/string/select_from_database.rb +14 -7
- data/lib/version.rb +1 -1
- metadata +12 -12
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ea71406f4d69385841297c63312abbfe0a1ded34
|
4
|
+
data.tar.gz: 6309e4f2155021c42c80aa561dc5ddd2cd4999f7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6e3fde8664f03c76c8939e14899dc0ded79559e6b016ebdc966b2a6095964b3f1888286f4c09a0a42226d4a475a905e8249f5a9463530c87592455432f36a4f3
|
7
|
+
data.tar.gz: 407caf0cec3c9962efa62272493b80cdf65acdcd49c33e799546419cad94157456733fad89a28f0c8523e1d92902193e2511f6d1fff98dff4a1194d68b4fc45c
|
data/README.md
CHANGED
@@ -1,6 +1,17 @@
|
|
1
1
|
# Data::Anonymization
|
2
2
|
Afraid of using production data due to privacy issues? Data Anonymization is a tool that helps you build anonymized production data dumps which you can use for performance testing, security testing, debugging and development.
|
3
3
|
|
4
|
+
## Java/Kotlin version
|
5
|
+
|
6
|
+
Java/Kotlin version of tool supporting RDBMS databases is available with similar easy to use DSL.
|
7
|
+
* [Kotlin/Java Data Anonymization Tool](https://github.com/dataanon/data-anon)
|
8
|
+
* [Kotlin Maven Sample Project](https://github.com/dataanon/dataanon-kotlin-sample)
|
9
|
+
* [Java Maven Sample Project](https://github.com/dataanon/dataanon-java-sample)
|
10
|
+
|
11
|
+
|
12
|
+
----------------------
|
13
|
+
|
14
|
+
|
4
15
|
[<img src="https://secure.travis-ci.org/sunitparekh/data-anonymization.png?branch=master">](http://travis-ci.org/sunitparekh/data-anonymization)
|
5
16
|
[<img src="https://gemnasium.com/sunitparekh/data-anonymization.png?travis">](https://gemnasium.com/sunitparekh/data-anonymization)
|
6
17
|
[<img src="https://codeclimate.com/badge.png">](https://codeclimate.com/github/sunitparekh/data-anonymization)
|
@@ -71,25 +82,25 @@ Postgresql database having **composite primary key**
|
|
71
82
|
## Changelog
|
72
83
|
|
73
84
|
#### 0.8.1 (Aug 19, 2017)
|
74
|
-
1. Multi-threading support added by [stanislav-tyutin](https://github.com/stanislav-tyutin) using Pull Request.
|
85
|
+
1. Multi-threading support added by [stanislav-tyutin](https://github.com/stanislav-tyutin) using Pull Request.
|
75
86
|
2. Fixed to work with Ruby 2.4.x, issue with Integer data type
|
76
87
|
|
77
88
|
#### 0.8.0 (Oct 31, 2016)
|
78
|
-
1. Upgraded to rails 5.x
|
89
|
+
1. Upgraded to rails 5.x
|
79
90
|
|
80
91
|
#### 0.7.4 (Oct 29, 2016)
|
81
|
-
1. Continue to work on rails 4.x. Minor changes based on feedback.
|
92
|
+
1. Continue to work on rails 4.x. Minor changes based on feedback.
|
82
93
|
|
83
94
|
#### 0.8.0.rc1 (Sep 5, 2016)
|
84
|
-
1. Upgraded to rails 5.0, please report any issue or use case not working.
|
95
|
+
1. Upgraded to rails 5.0, please report any issue or use case not working.
|
85
96
|
|
86
97
|
#### 0.7.3 (Feb 5, 2016)
|
87
|
-
1. Fixed issue with batchsize. Thanks to [Jan Raasch](https://github.com/janraasch) for sending pull request.
|
98
|
+
1. Fixed issue with batchsize. Thanks to [Jan Raasch](https://github.com/janraasch) for sending pull request.
|
88
99
|
|
89
100
|
#### 0.7.2 (Sep 26, 2015)
|
90
101
|
1. Upgraded MongoDB to latest gem version 2.1.0 and tested with MongoDB 3.x version.
|
91
|
-
2. Upgraded gems to latest version
|
92
|
-
3. Adding limit functionality - Merge pull request #27 from yanismydj/master
|
102
|
+
2. Upgraded gems to latest version
|
103
|
+
3. Adding limit functionality - Merge pull request #27 from yanismydj/master
|
93
104
|
|
94
105
|
#### 0.7.1 (Jun 13, 2015)
|
95
106
|
1. Fixed issues with empty array data for MongoDB
|
@@ -228,6 +239,7 @@ Read more about [blacklist and whitelist here](http://sunitspace.blogspot.in/201
|
|
228
239
|
5. Make sure to give proper case for fields and table names.
|
229
240
|
6. Use skip and continue to apply different strategies for records.
|
230
241
|
7. Use 'limit' to limit the number of rows that will be imported in whitelist
|
242
|
+
8. RDBMS databases utilizing schemas can be specified via `schema_search_path`: `source_db { ... schema_search_path: 'public,my_special_schema' }`
|
231
243
|
|
232
244
|
## DSL Generation
|
233
245
|
|
@@ -273,7 +285,7 @@ datanon generate_rdbms_dsl -a postgresql -h 123.456.7.8 -d production_db
|
|
273
285
|
|
274
286
|
```
|
275
287
|
|
276
|
-
The relevant db gems must be installed so that AR has the adapters required to establish the connection to the databases. The script generates a file named **rdbms_whitelist_generated.rb** in the same location as the project.
|
288
|
+
The relevant db gems must be installed so that AR has the adapters required to establish the connection to the databases. The script generates a file named **rdbms_whitelist_generated.rb** in the same location as the project.
|
277
289
|
|
278
290
|
### MongoDB whitelist generation
|
279
291
|
|
@@ -378,7 +390,7 @@ has following attribute accessor
|
|
378
390
|
</tr>
|
379
391
|
<tr>
|
380
392
|
<td align="left">Text</td>
|
381
|
-
<td align="left"><a href="http://rubydoc.info/github/sunitparekh/data-anonymization/DataAnon/Strategy/Field/RandomUrl">
|
393
|
+
<td align="left"><a href="http://rubydoc.info/github/sunitparekh/data-anonymization/DataAnon/Strategy/Field/RandomUrl">RandomUrl</a></td>
|
382
394
|
<td align="left">Anonymizes a URL while mainting the structure</td>
|
383
395
|
</tr>
|
384
396
|
</table><table>
|
data/data-anonymization.gemspec
CHANGED
@@ -19,11 +19,11 @@ Gem::Specification.new do |gem|
|
|
19
19
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
20
20
|
gem.require_paths = ['lib']
|
21
21
|
|
22
|
-
gem.add_dependency('activerecord', '~> 5.
|
23
|
-
gem.add_dependency('composite_primary_keys', '~>
|
24
|
-
gem.add_dependency('activesupport', '~> 5.
|
25
|
-
gem.add_dependency('rgeo', '~> 0
|
26
|
-
gem.add_dependency('rgeo-geojson', '~> 0
|
22
|
+
gem.add_dependency('activerecord', '~> 5.1')
|
23
|
+
gem.add_dependency('composite_primary_keys', '~> 10.0')
|
24
|
+
gem.add_dependency('activesupport', '~> 5.1')
|
25
|
+
gem.add_dependency('rgeo', '~> 1.0')
|
26
|
+
gem.add_dependency('rgeo-geojson', '~> 2.0')
|
27
27
|
gem.add_dependency('powerbar', '~> 1.0')
|
28
28
|
gem.add_dependency('parallel', '~> 1.12')
|
29
29
|
gem.add_dependency('thor', '~> 0.20')
|
@@ -12,16 +12,23 @@ module DataAnon
|
|
12
12
|
include Utils::Logging
|
13
13
|
|
14
14
|
def initialize table_name, field_name, connection_spec
|
15
|
-
|
16
|
-
|
17
|
-
@
|
18
|
-
logger.debug "For field strategy #{table_name}:#{field_name} using values #{@values} "
|
19
|
-
|
15
|
+
@table_name = table_name
|
16
|
+
@field_name = field_name
|
17
|
+
@connection_spec = connection_spec
|
20
18
|
end
|
21
19
|
|
22
|
-
|
23
|
-
|
20
|
+
def anonymize field
|
21
|
+
@values ||= begin
|
22
|
+
DataAnon::Utils::SourceDatabase.establish_connection @connection_spec
|
23
|
+
source = Utils::SourceTable.create @table_name, []
|
24
|
+
values = source.select(@field_name).distinct.collect { |record| record[@field_name]}
|
25
|
+
logger.debug "For field strategy #{@table_name}:#{@field_name} using values #{values} "
|
26
|
+
values
|
27
|
+
end
|
24
28
|
|
29
|
+
super
|
30
|
+
end
|
31
|
+
end
|
25
32
|
end
|
26
33
|
end
|
27
34
|
end
|
data/lib/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: data-anonymization
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.8.
|
4
|
+
version: 0.8.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Sunit Parekh
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2018-03-07 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: activerecord
|
@@ -18,70 +18,70 @@ dependencies:
|
|
18
18
|
requirements:
|
19
19
|
- - "~>"
|
20
20
|
- !ruby/object:Gem::Version
|
21
|
-
version: '5.
|
21
|
+
version: '5.1'
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
24
|
version_requirements: !ruby/object:Gem::Requirement
|
25
25
|
requirements:
|
26
26
|
- - "~>"
|
27
27
|
- !ruby/object:Gem::Version
|
28
|
-
version: '5.
|
28
|
+
version: '5.1'
|
29
29
|
- !ruby/object:Gem::Dependency
|
30
30
|
name: composite_primary_keys
|
31
31
|
requirement: !ruby/object:Gem::Requirement
|
32
32
|
requirements:
|
33
33
|
- - "~>"
|
34
34
|
- !ruby/object:Gem::Version
|
35
|
-
version: '
|
35
|
+
version: '10.0'
|
36
36
|
type: :runtime
|
37
37
|
prerelease: false
|
38
38
|
version_requirements: !ruby/object:Gem::Requirement
|
39
39
|
requirements:
|
40
40
|
- - "~>"
|
41
41
|
- !ruby/object:Gem::Version
|
42
|
-
version: '
|
42
|
+
version: '10.0'
|
43
43
|
- !ruby/object:Gem::Dependency
|
44
44
|
name: activesupport
|
45
45
|
requirement: !ruby/object:Gem::Requirement
|
46
46
|
requirements:
|
47
47
|
- - "~>"
|
48
48
|
- !ruby/object:Gem::Version
|
49
|
-
version: '5.
|
49
|
+
version: '5.1'
|
50
50
|
type: :runtime
|
51
51
|
prerelease: false
|
52
52
|
version_requirements: !ruby/object:Gem::Requirement
|
53
53
|
requirements:
|
54
54
|
- - "~>"
|
55
55
|
- !ruby/object:Gem::Version
|
56
|
-
version: '5.
|
56
|
+
version: '5.1'
|
57
57
|
- !ruby/object:Gem::Dependency
|
58
58
|
name: rgeo
|
59
59
|
requirement: !ruby/object:Gem::Requirement
|
60
60
|
requirements:
|
61
61
|
- - "~>"
|
62
62
|
- !ruby/object:Gem::Version
|
63
|
-
version: '0
|
63
|
+
version: '1.0'
|
64
64
|
type: :runtime
|
65
65
|
prerelease: false
|
66
66
|
version_requirements: !ruby/object:Gem::Requirement
|
67
67
|
requirements:
|
68
68
|
- - "~>"
|
69
69
|
- !ruby/object:Gem::Version
|
70
|
-
version: '0
|
70
|
+
version: '1.0'
|
71
71
|
- !ruby/object:Gem::Dependency
|
72
72
|
name: rgeo-geojson
|
73
73
|
requirement: !ruby/object:Gem::Requirement
|
74
74
|
requirements:
|
75
75
|
- - "~>"
|
76
76
|
- !ruby/object:Gem::Version
|
77
|
-
version: '0
|
77
|
+
version: '2.0'
|
78
78
|
type: :runtime
|
79
79
|
prerelease: false
|
80
80
|
version_requirements: !ruby/object:Gem::Requirement
|
81
81
|
requirements:
|
82
82
|
- - "~>"
|
83
83
|
- !ruby/object:Gem::Version
|
84
|
-
version: '0
|
84
|
+
version: '2.0'
|
85
85
|
- !ruby/object:Gem::Dependency
|
86
86
|
name: powerbar
|
87
87
|
requirement: !ruby/object:Gem::Requirement
|