strike 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,20 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
18
+ .rbx
19
+ pkg
20
+ *.db
data/Gemfile ADDED
@@ -0,0 +1,7 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
4
+
5
+ group :test do
6
+ gem 'minitest'
7
+ end
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Juan Hernández
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,95 @@
1
+ # Strike
2
+
3
+ Command line script to generate mysql dump with encrypted data.
4
+ This is a wrapper arround the [my_obfuscate][my_obfuscate] gem.
5
+
6
+ ## Installation
7
+
8
+ To install it:
9
+
10
+ $ gem install strike
11
+
12
+ Or install it yourself:
13
+
14
+ $ bundle install
15
+ $ bundle exec rake install
16
+
17
+ ## Usage
18
+
19
+ To see all the options execute:
20
+
21
+ $ strike help
22
+
23
+ To generate a new dump, use the following command:
24
+
25
+ $ strike dump mysql://root@localhost/db_production --profile=tables.rb > obfuscated_dump.sql
26
+
27
+ This command dumps the `database_url` following the tables defined in the `profile`
28
+ file (defaults to `Strikefile`). The default dump output is STDOUT.
29
+
30
+ The `database_url` must have one of the following formats:
31
+
32
+ mysql://user:password@host/database
33
+ mysql://user@host/database
34
+
35
+ It uses the same database url format than [Sequel][sequel].
36
+
37
+ ## Profile file
38
+
39
+ The profile file is a ruby file with the definitions of the tables that hold
40
+ the data that needs to be obfuscated or encrypted. The definitions has
41
+ the same format that is specified by the [my_obfuscate][my_obfuscate] gem,
42
+ but with a convenient DSL. Not all the data of all the tables needs
43
+ to be defined.
44
+
45
+ By default, the command will search for a `Strikefile` in the current directory.
46
+
47
+ Example profile file:
48
+
49
+ ```ruby
50
+ # tables_definition.rb
51
+
52
+ table :users do |t|
53
+ # t.column_name :obfuscation_type
54
+ t.name :first_name
55
+ t.email :email
56
+ end
57
+
58
+ def some_method
59
+ 'Fixed title'
60
+ end
61
+
62
+ table :movies do |t|
63
+ t.title type: fixed, string: some_method
64
+ t.date type: fixed, string: proc { |row| DateTime.now }
65
+ end
66
+ ```
67
+
68
+ ## Dependencies
69
+
70
+ * `mysqldump`: to create the dump to manipulate.
71
+ * [my_obfuscate][my_obfuscate]: the core of this utility.
72
+ * [Sequel][sequel]: extracts the info for the non defined tables.
73
+ * [Thor][thor]: cli utilities.
74
+
75
+ ## Notes
76
+
77
+ * It is only 1.9 compliant.
78
+ * Only supports `mysql`.
79
+
80
+ ## Why that name?
81
+
82
+ Comes from [S.T.R.I.K.E][strike].
83
+
84
+ ## Contributing
85
+
86
+ 1. Fork it
87
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
88
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
89
+ 4. Push to the branch (`git push origin my-new-feature`)
90
+ 5. Create new Pull Request
91
+
92
+ [my_obfuscate]: https://github.com/jbraeuer/my_obfuscate/
93
+ [sequel]: http://sequel.rubyforge.org/
94
+ [thor]: https://github.com/wycats/thor
95
+ [strike]: http://en.wikipedia.org/wiki/S.T.R.I.K.E
data/Rakefile ADDED
@@ -0,0 +1,7 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+ require "rake/testtask"
4
+
5
+ Rake::TestTask.new do |t|
6
+ t.pattern = "test/**/*_test.rb"
7
+ end
data/TODO.md ADDED
@@ -0,0 +1,6 @@
1
+ # TODO:
2
+
3
+ * Log info in /var/log/strike/`database`-`timestamp`.log
4
+ * Options
5
+ - log: alternative log file
6
+ * BETTER TESTS!
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.3.1
data/bin/strike ADDED
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ $:.unshift File.dirname(__FILE__) + '/../lib'
4
+ require 'strike'
5
+
6
+ Strike.start(ARGV.dup)
data/lib/strike.rb ADDED
@@ -0,0 +1,78 @@
1
+ # encoding: utf-8
2
+
3
+ require 'rubygems'
4
+ require 'bundler'
5
+ Bundler.require(:default)
6
+
7
+ require 'tempfile'
8
+ require 'thor'
9
+
10
+ class Strike < Thor
11
+ require 'strike/interpreter'
12
+ require 'strike/agent'
13
+
14
+ include Thor::Actions
15
+
16
+ desc 'version', 'Show version'
17
+ def version
18
+ $stdout.puts "v#{IO.read(File.expand_path('../../VERSION', __FILE__))}"
19
+ end
20
+
21
+ desc 'dump <database_url>', 'Dump the <database_url> to STDOUT.'
22
+ long_desc <<-DESC
23
+ Dump the <database_url> following the table definitions defined in the <profile>
24
+ (defaults to `Strikefile`). The default dump output is STDOUT.
25
+
26
+ The <database_url> must have one of the following formats:
27
+
28
+ \x5\tmysql://user:password@host/database
29
+ \x5\tmysql://user@host/database
30
+
31
+ Usage example:
32
+
33
+ $ strike dump mysql://root@localhost/db_production > dump.sql
34
+
35
+ $ strike dump mysql://root:secret@localhost/db_production --profile=tables.rb > dump.sql
36
+
37
+ The tables are defined with a DSL, which is a wrapper
38
+ arround the obfuscation types defined in the MyObfuscate gem.
39
+
40
+ Example:
41
+ \x5\t# tables.rb
42
+ \x5\ttable :users do |t|
43
+ \x5\t # t.column_name :obfuscation_type
44
+ \x5\t t.name :first_name
45
+ \x5\t t.email :email
46
+ \x5\tend
47
+ DESC
48
+ method_option :profile,
49
+ aliases: '-p',
50
+ type: :string,
51
+ default: 'Strikefile',
52
+ required: true,
53
+ desc: 'Profile with the table definitions.'
54
+ method_option :output,
55
+ aliases: '-o',
56
+ type: :string,
57
+ required: false,
58
+ desc: 'Output file. If none is given, outputs to STDOUT.'
59
+ def dump(database_url)
60
+ file = options[:profile]
61
+
62
+ if options[:output]
63
+ modes = File::CREAT|File::TRUNC|File::RDWR
64
+ output = File.new(options[:output], modes, 0644)
65
+ end
66
+
67
+ if file && File.exist?(file)
68
+ File.open(file) do |profile|
69
+ tables = Interpreter.new.parse(profile.read)
70
+ Agent.new.call(self, database_url, tables, output || $stdout)
71
+ end
72
+ else
73
+ $stdout.puts "Profile Error: No such file #{file}"
74
+ end
75
+ ensure
76
+ output.close if output
77
+ end
78
+ end
@@ -0,0 +1,108 @@
1
+ # encoding: utf-8
2
+
3
+ require 'bundler'
4
+ Bundler.require(:default)
5
+
6
+ require 'mysql2'
7
+ require 'sequel'
8
+ require 'my_obfuscate'
9
+
10
+ class Strike
11
+ class Agent
12
+ def initialize(config = {})
13
+ @db_connector = config[:db_connector]
14
+ @dumpfile_source = config[:dumpfile_source]
15
+ @obfuscator_source = config[:obfuscator_source]
16
+ end
17
+
18
+ def connect_to_db(database_url)
19
+ db_connector.call(database_url.gsub(/^mysql:/, 'mysql2:'))
20
+ end
21
+ protected :connect_to_db
22
+
23
+ def db_connector
24
+ @db_connector ||= Sequel.public_method(:connect)
25
+ end
26
+ protected :db_connector
27
+
28
+ def call(cli, database_url, tables, output = $stdout)
29
+ @cli = cli
30
+ @db = connect_to_db(database_url)
31
+ @tables = tables
32
+
33
+ tempfile do |file|
34
+ dump_data(@db.opts, file)
35
+ obfuscate_data(file, output)
36
+ end
37
+ end
38
+
39
+ def tempfile(&block)
40
+ tmp = dumpfile_source.call(['original_dump', 'sql']) do |file|
41
+ block.call(file)
42
+ file
43
+ end
44
+ ensure
45
+ tmp.unlink if tmp
46
+ end
47
+ protected :tempfile
48
+
49
+ def dumpfile_source
50
+ @dumpfile_source ||= Tempfile.public_method(:open)
51
+ end
52
+ protected :dumpfile_source
53
+
54
+ # TODO: support more databases
55
+ def dump_data(db_config, file)
56
+ dump_options = %w(-c
57
+ --add-drop-table
58
+ --add-locks
59
+ --single-transaction
60
+ --set-charset
61
+ --create-options
62
+ --disable-keys
63
+ --quick).join(' ')
64
+ dump_options << " -u #{db_config[:user]}" if db_config[:user]
65
+ dump_options << " -h #{db_config[:host]}" if db_config[:host]
66
+ dump_options << " -P #{db_config[:port]}" if db_config[:port]
67
+ dump_options << " -p#{db_config[:password]}" if db_config[:password]
68
+ dump_options << " #{db_config[:database]}"
69
+
70
+ run dump_cmd(dump_options, file)
71
+ end
72
+
73
+ def dump_cmd(options, file)
74
+ "mysqldump #{options} > #{file.path}"
75
+ end
76
+ protected :dump_cmd
77
+
78
+ def run(cmd)
79
+ @cli.run cmd, verbose: false, capture: true
80
+ end
81
+ protected :run
82
+
83
+ def obfuscate_data(input, output)
84
+ obfuscator = obfuscator_source.call(table_definitions)
85
+ obfuscator.globally_kept_columns = %w(id created_at updated_at)
86
+
87
+ obfuscator.obfuscate(input, output)
88
+ end
89
+
90
+ def obfuscator_source
91
+ @obfuscator_source ||= MyObfuscate.public_method(:new)
92
+ end
93
+ protected :obfuscator_source
94
+
95
+ def table_definitions
96
+ db_tables.reduce({}) do |acc, table|
97
+ acc[table] = @tables[table].call
98
+ acc
99
+ end
100
+ end
101
+ protected :table_definitions
102
+
103
+ def db_tables
104
+ @db.tables
105
+ end
106
+ protected :db_tables
107
+ end
108
+ end
@@ -0,0 +1,38 @@
1
+ # encoding: utf-8
2
+
3
+ require 'strike/table'
4
+
5
+ class Strike
6
+ class Interpreter
7
+ attr_reader :tables
8
+
9
+ def initialize(table_source = nil)
10
+ @table_source = table_source
11
+ @tables ||= Hash.new { |h, k| h[k] = -> { :keep } }
12
+ end
13
+
14
+ # Parse the given profile and generate the tables defined in it.
15
+ #
16
+ # @param [String] profile the profile with the definitions.
17
+ # @return [Hash] all the tables defined in the profile.
18
+ def parse(profile)
19
+ instance_eval(profile)
20
+ tables
21
+ end
22
+
23
+ # Define a table and its tables.
24
+ #
25
+ # @param [String, Symbol] name the name of the table.
26
+ # @param [Proc] block the block to declare the definitions for the tables.
27
+ def table(name, &block)
28
+ table = table_source.call(&block)
29
+
30
+ @tables[name.to_sym] = table
31
+ end
32
+
33
+ def table_source
34
+ @table_source ||= Strike::Table.public_method(:new)
35
+ end
36
+ protected :table_source
37
+ end
38
+ end
@@ -0,0 +1,24 @@
1
+ # encoding: utf-8
2
+
3
+ class Strike
4
+ class Table
5
+ def initialize(&block)
6
+ @definition ||= {}
7
+ yield self if block_given?
8
+ end
9
+
10
+ def method_missing(method, *args, &block)
11
+ @definition[method] = block_given? ? yield(self) : args.first
12
+
13
+ true
14
+ end
15
+
16
+ def call
17
+ to_hash
18
+ end
19
+
20
+ def to_hash
21
+ @definition
22
+ end
23
+ end
24
+ end
data/strike.gemspec ADDED
@@ -0,0 +1,23 @@
1
+ # encoding: utf-8
2
+
3
+ Gem::Specification.new do |gem|
4
+ gem.authors = ['Juan Hernández']
5
+ gem.email = ['juan.hernandez@wuaki.tv']
6
+ gem.description = %q{Dump a mysql database with sensitive data encrypted}
7
+ gem.summary = %q{Dump a mysql database with sensitive data encrypted}
8
+ gem.homepage = ''
9
+
10
+ gem.files = `git ls-files`.split($\)
11
+ gem.executables = ['strike']
12
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
13
+ gem.name = 'strike'
14
+ gem.require_paths = ['lib']
15
+ gem.version = IO.read(File.expand_path('../VERSION', __FILE__))
16
+
17
+ gem.add_runtime_dependency 'rake', '~> 0.9'
18
+ gem.add_runtime_dependency 'my_obfuscate', '~> 0.3.7'
19
+ gem.add_runtime_dependency 'sequel', '~> 3.39'
20
+ gem.add_runtime_dependency 'mysql2', '~> 0.3.11'
21
+ gem.add_runtime_dependency 'sqlite3', '~> 1.3.6'
22
+ gem.add_runtime_dependency 'thor', '~> 0.16.0'
23
+ end
@@ -0,0 +1,44 @@
1
+ # encoding: utf-8
2
+
3
+ require_relative '../../minitest_helper'
4
+ require 'strike/interpreter'
5
+
6
+ class Strike::InterpreterTest < MiniTest::Unit::TestCase
7
+ def setup
8
+ @interpreter = Strike::Interpreter.new(table_source)
9
+ @profile = <<-PROFILE
10
+ table :users do |t|
11
+ t.name :first_name
12
+ end
13
+
14
+ table :movies
15
+ PROFILE
16
+ end
17
+
18
+ def test_should_parse_a_profile_into_tables
19
+ tables = @interpreter.parse(@profile)
20
+
21
+ assert_equal 2, tables.count
22
+ assert tables[:users]
23
+ assert tables[:movies]
24
+
25
+ table_mock.verify
26
+ end
27
+
28
+ def test_should_have_default_tables
29
+ tables = @interpreter.tables
30
+
31
+ assert_equal :keep, tables[:test].call
32
+ assert_equal :keep, tables[:test2].call
33
+ end
34
+
35
+ private
36
+
37
+ def table_source
38
+ ->(&block) { block ? block.call(table_mock) : table_mock }
39
+ end
40
+
41
+ def table_mock
42
+ @table_mock ||= MiniTest::Mock.new.expect(:name, true, [:first_name])
43
+ end
44
+ end
@@ -0,0 +1,35 @@
1
+ # encoding: utf-8
2
+
3
+ require_relative '../../minitest_helper'
4
+ require 'strike/table'
5
+
6
+ class Strike::TableTest < MiniTest::Unit::TestCase
7
+ def setup
8
+ @table = Strike::Table.new
9
+ end
10
+
11
+ def test_should_respond_to_missing_methods
12
+ assert @table.name(:test)
13
+ assert @table.test
14
+ end
15
+
16
+ def test_should_save_method_calls_as_hash
17
+ expected = { name: :test }
18
+ @table.name(:test)
19
+
20
+ assert_equal expected, @table.to_hash
21
+ end
22
+
23
+ def test_should_respond_to_call
24
+ assert_equal Hash.new, @table.call
25
+ end
26
+
27
+ def test_should_accept_a_block_when_its_initialized
28
+ table = Strike::Table.new do |t|
29
+ t.name :test
30
+ end
31
+ expected = { name: :test }
32
+
33
+ assert_equal expected, table.to_hash
34
+ end
35
+ end
@@ -0,0 +1,4 @@
1
+ # encoding: utf-8
2
+
3
+ $:.unshift File.dirname(__FILE__) + '/../lib'
4
+ require 'minitest/autorun'
metadata ADDED
@@ -0,0 +1,167 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: strike
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.3.1
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Juan Hernández
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2012-09-19 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ~>
20
+ - !ruby/object:Gem::Version
21
+ version: '0.9'
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ~>
28
+ - !ruby/object:Gem::Version
29
+ version: '0.9'
30
+ - !ruby/object:Gem::Dependency
31
+ name: my_obfuscate
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - ~>
36
+ - !ruby/object:Gem::Version
37
+ version: 0.3.7
38
+ type: :runtime
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ~>
44
+ - !ruby/object:Gem::Version
45
+ version: 0.3.7
46
+ - !ruby/object:Gem::Dependency
47
+ name: sequel
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - ~>
52
+ - !ruby/object:Gem::Version
53
+ version: '3.39'
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - ~>
60
+ - !ruby/object:Gem::Version
61
+ version: '3.39'
62
+ - !ruby/object:Gem::Dependency
63
+ name: mysql2
64
+ requirement: !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - ~>
68
+ - !ruby/object:Gem::Version
69
+ version: 0.3.11
70
+ type: :runtime
71
+ prerelease: false
72
+ version_requirements: !ruby/object:Gem::Requirement
73
+ none: false
74
+ requirements:
75
+ - - ~>
76
+ - !ruby/object:Gem::Version
77
+ version: 0.3.11
78
+ - !ruby/object:Gem::Dependency
79
+ name: sqlite3
80
+ requirement: !ruby/object:Gem::Requirement
81
+ none: false
82
+ requirements:
83
+ - - ~>
84
+ - !ruby/object:Gem::Version
85
+ version: 1.3.6
86
+ type: :runtime
87
+ prerelease: false
88
+ version_requirements: !ruby/object:Gem::Requirement
89
+ none: false
90
+ requirements:
91
+ - - ~>
92
+ - !ruby/object:Gem::Version
93
+ version: 1.3.6
94
+ - !ruby/object:Gem::Dependency
95
+ name: thor
96
+ requirement: !ruby/object:Gem::Requirement
97
+ none: false
98
+ requirements:
99
+ - - ~>
100
+ - !ruby/object:Gem::Version
101
+ version: 0.16.0
102
+ type: :runtime
103
+ prerelease: false
104
+ version_requirements: !ruby/object:Gem::Requirement
105
+ none: false
106
+ requirements:
107
+ - - ~>
108
+ - !ruby/object:Gem::Version
109
+ version: 0.16.0
110
+ description: Dump a mysql database with sensitive data encrypted
111
+ email:
112
+ - juan.hernandez@wuaki.tv
113
+ executables:
114
+ - strike
115
+ extensions: []
116
+ extra_rdoc_files: []
117
+ files:
118
+ - .gitignore
119
+ - Gemfile
120
+ - LICENSE
121
+ - README.md
122
+ - Rakefile
123
+ - TODO.md
124
+ - VERSION
125
+ - bin/strike
126
+ - lib/strike.rb
127
+ - lib/strike/agent.rb
128
+ - lib/strike/interpreter.rb
129
+ - lib/strike/table.rb
130
+ - strike.gemspec
131
+ - test/lib/strike/interpreter_test.rb
132
+ - test/lib/strike/table_test.rb
133
+ - test/minitest_helper.rb
134
+ homepage: ''
135
+ licenses: []
136
+ post_install_message:
137
+ rdoc_options: []
138
+ require_paths:
139
+ - lib
140
+ required_ruby_version: !ruby/object:Gem::Requirement
141
+ none: false
142
+ requirements:
143
+ - - ! '>='
144
+ - !ruby/object:Gem::Version
145
+ version: '0'
146
+ segments:
147
+ - 0
148
+ hash: -1697264933312532693
149
+ required_rubygems_version: !ruby/object:Gem::Requirement
150
+ none: false
151
+ requirements:
152
+ - - ! '>='
153
+ - !ruby/object:Gem::Version
154
+ version: '0'
155
+ segments:
156
+ - 0
157
+ hash: -1697264933312532693
158
+ requirements: []
159
+ rubyforge_project:
160
+ rubygems_version: 1.8.24
161
+ signing_key:
162
+ specification_version: 3
163
+ summary: Dump a mysql database with sensitive data encrypted
164
+ test_files:
165
+ - test/lib/strike/interpreter_test.rb
166
+ - test/lib/strike/table_test.rb
167
+ - test/minitest_helper.rb