rmre 0.0.5 → 0.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore CHANGED
@@ -1,4 +1,7 @@
1
1
  *.gem
2
2
  tmp
3
- vendor/bundle
4
3
  .bundle
4
+ vendor/
5
+ .projectile
6
+ *.sublime-project
7
+ *.sublime-workspace
@@ -1,43 +1,45 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- rmre (0.0.5)
4
+ rmre (0.0.6)
5
5
  activerecord (>= 3.0.0)
6
6
  erubis
7
7
 
8
8
  GEM
9
9
  remote: http://rubygems.org/
10
10
  specs:
11
- activemodel (3.2.8)
12
- activesupport (= 3.2.8)
11
+ activemodel (3.2.13)
12
+ activesupport (= 3.2.13)
13
13
  builder (~> 3.0.0)
14
- activerecord (3.2.8)
15
- activemodel (= 3.2.8)
16
- activesupport (= 3.2.8)
14
+ activerecord (3.2.13)
15
+ activemodel (= 3.2.13)
16
+ activesupport (= 3.2.13)
17
17
  arel (~> 3.0.2)
18
18
  tzinfo (~> 0.3.29)
19
- activesupport (3.2.8)
20
- i18n (~> 0.6)
19
+ activesupport (3.2.13)
20
+ i18n (= 0.6.1)
21
21
  multi_json (~> 1.0)
22
22
  arel (3.0.2)
23
23
  builder (3.0.4)
24
24
  diff-lcs (1.1.3)
25
25
  erubis (2.7.0)
26
26
  i18n (0.6.1)
27
- multi_json (1.3.6)
28
- rspec (2.8.0)
29
- rspec-core (~> 2.8.0)
30
- rspec-expectations (~> 2.8.0)
31
- rspec-mocks (~> 2.8.0)
32
- rspec-core (2.8.0)
33
- rspec-expectations (2.8.0)
34
- diff-lcs (~> 1.1.2)
35
- rspec-mocks (2.8.0)
36
- tzinfo (0.3.34)
27
+ multi_json (1.7.3)
28
+ rake (10.0.3)
29
+ rspec (2.12.0)
30
+ rspec-core (~> 2.12.0)
31
+ rspec-expectations (~> 2.12.0)
32
+ rspec-mocks (~> 2.12.0)
33
+ rspec-core (2.12.2)
34
+ rspec-expectations (2.12.1)
35
+ diff-lcs (~> 1.1.3)
36
+ rspec-mocks (2.12.1)
37
+ tzinfo (0.3.37)
37
38
 
38
39
  PLATFORMS
39
40
  ruby
40
41
 
41
42
  DEPENDENCIES
43
+ rake
42
44
  rmre!
43
45
  rspec
@@ -2,6 +2,21 @@ Rmre is utility gem for creating Ruby on Rails models for legacy databases.
2
2
 
3
3
  Changes:
4
4
 
5
+ === 0.0.6 / 2013-xx-xx
6
+
7
+ * Enhancements
8
+
9
+ * Added support for mysql2 gem with foreign keys ({Daniel Garajau}[https://github.com/kriansa])
10
+ * Added support for non-standard foreign key names in has_many relationship ({Gui Weinmann}[https://github.com/alphabet])
11
+ * Changed readme file to use mysql2 gem in samples ({Ketan Deshmukh}[https://github.com/ketan21])
12
+ * Experimental support for copying databases through db_copy binary or by using Rmre::Migrator class
13
+
14
+ === 0.0.5 / 2012-11-04
15
+
16
+ * Enhancements
17
+
18
+ * Drop explicit erubis dependency which prevented rmre usage along with newest Rails versions
19
+
5
20
  === 0.0.4 / 2012-06-07
6
21
 
7
22
  * Enhancements
@@ -8,7 +8,7 @@ sets models relationships on a basic level through belongs_to and has_many decla
8
8
 
9
9
  = Installation
10
10
 
11
- Rmre gem is on the Rubygems and you can install it with
11
+ Rmre can be installed with
12
12
 
13
13
  gem install rmre
14
14
 
@@ -16,7 +16,7 @@ Rmre gem is on the Rubygems and you can install it with
16
16
 
17
17
  Rmre is very simple to use:
18
18
 
19
- rmre -a mysql -d my_database -u my_username -p my_password -o /path/where/models/will/be/created
19
+ rmre -a mysql2 -d my_database -u my_username -p my_password -o /path/where/models/will/be/created
20
20
 
21
21
  That's all! Of course there is standard help which you can print at any time:
22
22
 
@@ -63,6 +63,83 @@ If you want to try *Rmre* and you do not have sample database you can use
63
63
  _Sakila_ at http://dev.mysql.com/doc/sakila/en/sakila.html#sakila-installation for MySQL and
64
64
  _Pagila_ at http://pgfoundry.org/projects/dbsamples for PostgreSQL.
65
65
 
66
+ = Copying databases
67
+
68
+ Rmre gem has built-in support for copying databases (structure and data). This feature is
69
+ currently experimental.
70
+
71
+ During copy, Rmre will create primary keys on target tables. Since Rmre uses ActiveRecord,
72
+ composite primary keys are not supported.
73
+
74
+ Copying structure between different RDBMS can be tricky due to different data types.
75
+ Some adapters do not convert all types to Rails value. Example is oracle_enhanced adapter
76
+ which for 'LONG' column type sets column's type to nil but keeps sql_type as 'LONG'. Rmre
77
+ handles these cases through Rmre::DbUtils module. Currently it properly converts Oracle's
78
+ +raw+ and +LONG+ types to MySQL's +binary+ and +text+. Support for more conversions will
79
+ be added (if I find or get info about needed conversions). However if you write your own
80
+ script and do not use +db_copy+ runner you can set additional conversion rules by adding
81
+ values to Rmre::DbUtils::COLUMN_CONVERSIONS hash.
82
+
83
+ Keys in this hash are target sdapter names and values are hashes with source column type
84
+ as key and target column type as value:
85
+
86
+ COLUMN_CONVERSIONS = {
87
+ "Mysql2" => {
88
+ :raw => :binary,
89
+ "LONG" => :text
90
+ }
91
+ }
92
+
93
+ In order to copy one database to another RDBMS you must start db_copy with +-f+ option
94
+ and with path to YAML configuration file. Full sample of configuration file is:
95
+
96
+ :source:
97
+ adapter: sqlserver
98
+ mode: dblib
99
+ dataserver:
100
+ host: localhost
101
+ port: 1433
102
+ database: source_db
103
+ username: source_username
104
+ password: source_pass
105
+ timeout: 5000
106
+ :target:
107
+ adapter: mysql2
108
+ encoding: utf8
109
+ reconnect: false
110
+ database: target_db
111
+ pool: 5
112
+ username: target_username
113
+ password: target_pass
114
+ host: localhost
115
+ :verbose: true
116
+ :force: true
117
+ :skip_existing: false
118
+ :skip:
119
+ - do_not_copy_table_1
120
+ - do_not_copy_table_2
121
+
122
+ Source and target options are standard Rails configurations for source and target databases.
123
+ Parameter +:verbose+ is optional and can be omitted. If set to try will db_copy will pring out
124
+ progress during copy. This parameter can be set also by passing +-v+ options to db_copy
125
+
126
+ db_copy -f /path/to/config/file.yml -v
127
+
128
+ Value from file will override the one given on the command line.
129
+
130
+ Next optional parameter is +:force+. If it is set to true Migrator class will force table creation
131
+ Similar to +verbose+ parameter this value can be set by passing +-o+ option to db_copy and value
132
+ from configuration file will override the value given on the command line.
133
+
134
+ Parameter +:skip_existing+ signals db_copy to skip tables that already exist in target database.
135
+
136
+ If you do not want to copy some tables add them to the array +:skip+ in configuration file.
137
+
138
+ There is a big probability that db_copy will not be able to copy database which is not Rails compliant.
139
+ Reasons for this are numerous: unsupported column types, composite primary keys, etc. If you face such
140
+ a problem create an issue and I will try to implement support for various special cases. However if used
141
+ on Rails compliant databases db_copy should be able to peform full copy between any of supported RDBMS.
142
+
66
143
  = TODO
67
144
 
68
145
  * Improve filtering
@@ -0,0 +1,73 @@
1
+ #!/usr/bin/env ruby
2
+ $LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
3
+
4
+ require "yaml"
5
+ require "optparse"
6
+ require "rmre"
7
+
8
+ options = {verbose: false, force: false}
9
+ sample_cfg = {source: '<Source DB connection settings>',
10
+ target: '<Target DB connection settings>',
11
+ verbose: true,
12
+ force: true,
13
+ skip_existing: false,
14
+ skip: ["skip_one", "skip_two"]
15
+ }
16
+
17
+ optparse = OptionParser.new do |opts|
18
+ opts.banner = "Usage: db_copy -f CONFIGURATION_FILE"
19
+
20
+ opts.on('-h', '--help', 'Display this screen') do
21
+ puts opts
22
+ puts "Format of configuration file:\n#{sample_cfg.to_yaml}"
23
+ exit
24
+ end
25
+
26
+ opts.on('-f', '--file CONFIGURATION_FILE',
27
+ 'File with source and target DB connection settings') do |f|
28
+ options[:file] = f
29
+ end
30
+
31
+ opts.on('-v', '--verbose',
32
+ 'Verbose mode (show progress)') do |v|
33
+ options[:verbose] = true
34
+ end
35
+
36
+ opts.on('-o', '--force',
37
+ 'Force creating tables in target database') do |o|
38
+ options[:force] = true
39
+ end
40
+ end
41
+
42
+ begin
43
+ optparse.parse!
44
+ rescue OptionParser::ParseError => pe
45
+ puts pe.message
46
+ puts optparse
47
+ exit
48
+ end
49
+
50
+ options.merge! YAML.load_file(options[:file]) if options[:file] && File.exists?(options[:file])
51
+
52
+ unless options[:source] && options[:target]
53
+ puts "Missing configurations for source and target database"
54
+ puts optparse
55
+ puts "Format of configuration file:\n#{sample_cfg.to_yaml}"
56
+ exit
57
+ end
58
+
59
+ mig = Rmre::Migrator.new(options[:source],
60
+ options[:target],
61
+ verbose: options[:verbose],
62
+ skip_existing: options[:skip_existing])
63
+ mig.before_copy = lambda { |table_name|
64
+ return false if options[:skip].include?(table_name)
65
+ true
66
+ }
67
+
68
+ begin
69
+ mig.copy(options[:force])
70
+ rescue Exception => e
71
+ puts e.message
72
+ exit
73
+ end
data/bin/rmre CHANGED
@@ -107,7 +107,7 @@ if options[:file] == true
107
107
  :file_name => "#{file_name}")
108
108
  end
109
109
  else
110
- options = YAML.load_file(options[:file]) if options[:file] && File.exists?(options[:file])
110
+ options.merge! YAML.load_file(options[:file]) if options[:file] && File.exists?(options[:file])
111
111
 
112
112
  unless options[:db][:adapter]
113
113
  puts "Missing required arguments -a (--adapter) and -d (--database)"
@@ -115,7 +115,7 @@ else
115
115
  exit
116
116
  end
117
117
 
118
- generator = Rmre::Generator.new(options[:db], options[:out_path], options[:include])
118
+ generator = Rmre::Generator.new(options[:db], options[:out_path], options[:include], options[:inflections])
119
119
 
120
120
  begin
121
121
  generator.connect
@@ -0,0 +1,237 @@
1
+ # = progressbar.rb
2
+ #
3
+ # == Copyright (C) 2001 Satoru Takabayashi
4
+ #
5
+ # Ruby License
6
+ #
7
+ # This module is free software. You may use, modify, and/or redistribute this
8
+ # software under the same terms as Ruby.
9
+ #
10
+ # This program is distributed in the hope that it will be useful, but WITHOUT
11
+ # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
12
+ # FOR A PARTICULAR PURPOSE.
13
+ #
14
+ # == Author(s)
15
+ #
16
+ # * Satoru Takabayashi
17
+
18
+ # Author:: Satoru Takabayashi
19
+ # Copyright:: Copyright (c) 2001 Satoru Takabayashi
20
+ # License:: Ruby License
21
+
22
+ # = Console Progress Bar
23
+ #
24
+ # Console::ProgressBar is a terminal-based progress bar library.
25
+ #
26
+ # == Usage
27
+ #
28
+ # pbar = ConsoleProgressBar.new( "Demo", 100 )
29
+ # 100.times { pbar.inc }
30
+ # pbar.finish
31
+ #
32
+
33
+ module Console; end
34
+
35
+ class Console::ProgressBar
36
+
37
+ def initialize(title, total, out = STDERR)
38
+ @title = title
39
+ @total = total
40
+ @out = out
41
+ @bar_length = 80
42
+ @bar_mark = "o"
43
+ @total_overflow = true
44
+ @current = 0
45
+ @previous = 0
46
+ @is_finished = false
47
+ @start_time = Time.now
48
+ @format = "%-14s %3d%% %s %s"
49
+ @format_arguments = [:title, :percentage, :bar, :stat]
50
+ show_progress
51
+ end
52
+
53
+ private
54
+ def convert_bytes (bytes)
55
+ if bytes < 1024
56
+ sprintf("%6dB", bytes)
57
+ elsif bytes < 1024 * 1000 # 1000kb
58
+ sprintf("%5.1fKB", bytes.to_f / 1024)
59
+ elsif bytes < 1024 * 1024 * 1000 # 1000mb
60
+ sprintf("%5.1fMB", bytes.to_f / 1024 / 1024)
61
+ else
62
+ sprintf("%5.1fGB", bytes.to_f / 1024 / 1024 / 1024)
63
+ end
64
+ end
65
+
66
+ def transfer_rate
67
+ bytes_per_second = @current.to_f / (Time.now - @start_time)
68
+ sprintf("%s/s", convert_bytes(bytes_per_second))
69
+ end
70
+
71
+ def bytes
72
+ convert_bytes(@current)
73
+ end
74
+
75
+ def format_time (t)
76
+ t = t.to_i
77
+ sec = t % 60
78
+ min = (t / 60) % 60
79
+ hour = t / 3600
80
+ sprintf("%02d:%02d:%02d", hour, min, sec);
81
+ end
82
+
83
+ # ETA stands for Estimated Time of Arrival.
84
+ def eta
85
+ if @current == 0
86
+ "ETA: --:--:--"
87
+ else
88
+ elapsed = Time.now - @start_time
89
+ eta = elapsed * @total / @current - elapsed;
90
+ sprintf("ETA: %s", format_time(eta))
91
+ end
92
+ end
93
+
94
+ def elapsed
95
+ elapsed = Time.now - @start_time
96
+ sprintf("Time: %s", format_time(elapsed))
97
+ end
98
+
99
+ def stat
100
+ if @is_finished then elapsed else eta end
101
+ end
102
+
103
+ def stat_for_file_transfer
104
+ if @is_finished then
105
+ sprintf("%s %s %s", bytes, transfer_rate, elapsed)
106
+ else
107
+ sprintf("%s %s %s", bytes, transfer_rate, eta)
108
+ end
109
+ end
110
+
111
+ def eol
112
+ if @is_finished then "\n" else "\r" end
113
+ end
114
+
115
+ def bar
116
+ len = percentage * @bar_length / 100
117
+ sprintf("|%s%s|", @bar_mark * len, " " * (@bar_length - len))
118
+ end
119
+
120
+ def percentage
121
+ if @total.zero?
122
+ 100
123
+ else
124
+ @current * 100 / @total
125
+ end
126
+ end
127
+
128
+ def title
129
+ @title[0,13] + ":"
130
+ end
131
+
132
+ def get_width
133
+ # FIXME: I don't know how portable it is.
134
+ default_width = 80
135
+ begin
136
+ tiocgwinsz = 0x5413
137
+ data = [0, 0, 0, 0].pack("SSSS")
138
+ if @out.ioctl(tiocgwinsz, data) >= 0 then
139
+ rows, cols, xpixels, ypixels = data.unpack("SSSS")
140
+ if cols >= 0 then cols else default_width end
141
+ else
142
+ default_width
143
+ end
144
+ rescue Exception
145
+ default_width
146
+ end
147
+ end
148
+
149
+ def show
150
+ arguments = @format_arguments.map {|method| send(method) }
151
+ line = sprintf(@format, *arguments)
152
+
153
+ width = get_width
154
+ if line.length == width - 1
155
+ @out.print(line + eol)
156
+ elsif line.length >= width
157
+ @bar_length = [@bar_length - (line.length - width + 1), 0].max
158
+ if @bar_length == 0 then @out.print(line + eol) else show end
159
+ else #line.length < width - 1
160
+ @bar_length += width - line.length + 1
161
+ show
162
+ end
163
+ end
164
+
165
+ def show_progress
166
+ if @total.zero?
167
+ cur_percentage = 100
168
+ prev_percentage = 0
169
+ else
170
+ cur_percentage = (@current * 100 / @total).to_i
171
+ prev_percentage = (@previous * 100 / @total).to_i
172
+ end
173
+
174
+ if cur_percentage > prev_percentage || @is_finished
175
+ show
176
+ end
177
+ end
178
+
179
+ public
180
+ def file_transfer_mode
181
+ @format_arguments = [:title, :percentage, :bar, :stat_for_file_transfer]
182
+ end
183
+
184
+ def bar_mark= (mark)
185
+ @bar_mark = String(mark)[0..0]
186
+ end
187
+
188
+ def total_overflow= (boolv)
189
+ @total_overflow = boolv ? true : false
190
+ end
191
+
192
+ def format= (format)
193
+ @format = format
194
+ end
195
+
196
+ def format_arguments= (arguments)
197
+ @format_arguments = arguments
198
+ end
199
+
200
+ def finish
201
+ @current = @total
202
+ @is_finished = true
203
+ show_progress
204
+ end
205
+
206
+ def halt
207
+ @is_finished = true
208
+ show_progress
209
+ end
210
+
211
+ def set (count)
212
+ if count < 0
213
+ raise "invalid count less than zero: #{count}"
214
+ elsif count > @total
215
+ if @total_overflow
216
+ @total = count + 1
217
+ else
218
+ raise "invalid count greater than total: #{count}"
219
+ end
220
+ end
221
+ @current = count
222
+ show_progress
223
+ @previous = @current
224
+ end
225
+
226
+ def inc (step = 1)
227
+ @current += step
228
+ @current = @total if @current > @total
229
+ show_progress
230
+ @previous = @current
231
+ end
232
+
233
+ def inspect
234
+ "(ProgressBar: #{@current}/#{@total})"
235
+ end
236
+
237
+ end
@@ -1,2 +1,3 @@
1
1
  require 'rmre/generator'
2
+ require "rmre/migrator"
2
3
  require 'rmre/version'
@@ -0,0 +1,18 @@
1
+ module Rmre
2
+ module DbUtils
3
+ COLUMN_CONVERSIONS = {
4
+ "Mysql2" => {
5
+ :raw => :binary,
6
+ "LONG" => :text
7
+ }
8
+ }
9
+
10
+ def self.convert_column_type(target_adapter_name, start_type)
11
+ if COLUMN_CONVERSIONS[target_adapter_name] &&
12
+ COLUMN_CONVERSIONS[target_adapter_name][start_type]
13
+ return COLUMN_CONVERSIONS[target_adapter_name][start_type]
14
+ end
15
+ return start_type
16
+ end
17
+ end
18
+ end
@@ -0,0 +1,30 @@
1
+ require 'active_record'
2
+
3
+ module Rmre
4
+ module DynamicDb
5
+ def self.included(base)
6
+ base.send :extend, Rmre::DynamicDb
7
+ end
8
+
9
+ def connection_options
10
+ @connection_options
11
+ end
12
+
13
+ def connection_options= v
14
+ @connection_options = v
15
+ end
16
+
17
+ def create_model_for(table_name, primary_key_name)
18
+ model_name = table_name.classify
19
+ module_eval <<-ruby_src, __FILE__, __LINE__ + 1
20
+ class #{model_name} < Db
21
+ self.table_name = '#{table_name}'
22
+ establish_connection(#{connection_options})
23
+ end
24
+ ruby_src
25
+ klass = const_get model_name
26
+ klass.primary_key = primary_key_name if primary_key_name && primary_key_name != 'id'
27
+ klass
28
+ end
29
+ end
30
+ end
@@ -10,11 +10,17 @@ module Rmre
10
10
 
11
11
  SETTINGS_ROOT = File.expand_path('../../../../db', __FILE__)
12
12
 
13
- def initialize(options, out_path, include)
13
+ def initialize(options, out_path, include, custom_inflections)
14
14
  @connection_options = options
15
15
  @connection = nil
16
16
  @output_path = out_path
17
17
  @include_prefixes = include
18
+ ActiveSupport::Inflector.inflections do |inflect|
19
+ custom_inflections.each do |ci|
20
+ inflect.plural(/#{ci[:plural].first}/, ci[:plural].second)
21
+ inflect.singular(/#{ci[:singular].first}/, ci[:singular].second)
22
+ end if custom_inflections.is_a? Array
23
+ end
18
24
  end
19
25
 
20
26
  def connect
@@ -70,6 +76,7 @@ module Rmre
70
76
  fk = []
71
77
  case @connection_options[:adapter]
72
78
  when 'mysql'
79
+ when 'mysql2'
73
80
  fk = mysql_foreign_keys
74
81
  when 'postgresql'
75
82
  fk = psql_foreign_keys
@@ -87,6 +94,9 @@ module Rmre
87
94
  src = "belongs_to :#{fk['to_table'].downcase.singularize}, :class_name => '#{fk['to_table'].tableize.classify}', :foreign_key => :#{fk['from_column']}"
88
95
  elsif fk['to_table'] == table_name
89
96
  src = "has_many :#{fk['from_table'].downcase.pluralize}, :class_name => '#{fk['from_table'].tableize.classify}'"
97
+ if connection.primary_key(table_name) == fk['from_column']
98
+ src += ", :foreign_key => :#{fk['from_column']}"
99
+ end
90
100
  end
91
101
  src
92
102
  end
@@ -0,0 +1,141 @@
1
+ require "rmre/db_utils"
2
+ require "rmre/dynamic_db"
3
+ require "contrib/progressbar"
4
+
5
+ module Rmre
6
+ module Source
7
+ include DynamicDb
8
+
9
+ class Db < ActiveRecord::Base
10
+ self.abstract_class = true
11
+ end
12
+ end
13
+
14
+ module Target
15
+ include DynamicDb
16
+
17
+ class Db < ActiveRecord::Base
18
+ self.abstract_class = true
19
+ end
20
+ end
21
+
22
+ class Migrator
23
+ attr_accessor :before_copy
24
+
25
+ def initialize(source_db_options, target_db_options, options = {})
26
+ # If set to true will call AR create_table with force (table will be dropped if exists)
27
+ @force_table_create = false
28
+ @skip_existing_tables = options[:skip_existing]
29
+ @verbose = options[:verbose]
30
+ @before_copy = nil
31
+
32
+ Rmre::Source.connection_options = source_db_options
33
+ Rmre::Target.connection_options = target_db_options
34
+ Rmre::Source::Db.establish_connection(Rmre::Source.connection_options)
35
+ Rmre::Target::Db.establish_connection(Rmre::Target.connection_options)
36
+ end
37
+
38
+ # Before we start copying we call block if it is given so some additional options
39
+ # can be set. For example MS SQL adapter has option to use lowercase names for
40
+ # all entities. We can set this options in a following way:
41
+ #
42
+ # mig = Migrator.new(..)
43
+ # mig.copy(true) do
44
+ # ActiveRecord::ConnectionAdapters::SQLServerAdapter.lowercase_schema_reflection = true
45
+ # end
46
+ def copy(force = false)
47
+ yield if block_given?
48
+
49
+ @force_table_create = force
50
+ tables_count = Rmre::Source::Db.connection.tables.length
51
+ Rmre::Source::Db.connection.tables.sort.each_with_index do |table, idx|
52
+ info "Copying table #{table} (#{idx + 1}/#{tables_count})..."
53
+ copy_table(table)
54
+ end
55
+ end
56
+
57
+ def copy_table(table)
58
+ if @skip_existing_tables && Rmre::Target::Db.connection.table_exists?(table)
59
+ info "Skipping #{table}"
60
+ return
61
+ end
62
+
63
+ if before_copy && before_copy.is_a?(Proc)
64
+ return unless before_copy.call(table)
65
+ end
66
+
67
+ if !Rmre::Target::Db.connection.table_exists?(table) || @force_table_create
68
+ create_table(table, Rmre::Source::Db.connection.columns(table))
69
+ end
70
+ copy_data(table)
71
+ end
72
+
73
+ def create_table(table, source_columns)
74
+ primary_key = Rmre::Source::Db.connection.primary_key(table)
75
+
76
+ # Create primary key if source table has primary key
77
+ opts = { :id => !primary_key.nil?, :force => @force_table_create }
78
+ # If primary key is not 'id' then set option to create proper primary key
79
+ opts[:primary_key] = primary_key unless primary_key == "id"
80
+
81
+ Rmre::Target::Db.connection.create_table(table, opts) do |t|
82
+ # Skip 'id' column if it is already created as primary key
83
+ source_columns.reject {|col| col.name == 'id' && opts[:id] && opts[:primary_key].nil? }.each do |sc|
84
+ options = {
85
+ :null => sc.null,
86
+ :default => sc.default
87
+ }
88
+
89
+ # Some adapters do not convert all types to Rails value. Example is oracle_enhanced adapter
90
+ # which for 'LONG' column type sets column's type to nil but keeps sql_type as 'LONG'.
91
+ # Therefore we will use one of these values so we can, in DbUtils, handle all possible
92
+ # column type mappings when we are migrating from one DB to anohter (Oracle -> MySQL,
93
+ # MS SQL -> PostgreSQL, etc).
94
+ source_type = sc.type.nil? ? sc.sql_type : sc.type
95
+ col_type = Rmre::DbUtils.convert_column_type(Rmre::Target::Db.connection.adapter_name, source_type)
96
+ case col_type
97
+ when :decimal
98
+ options.merge!({
99
+ :limit => sc.limit,
100
+ :precision => sc.precision,
101
+ :scale => sc.scale,
102
+ })
103
+ when :string
104
+ options.merge!({
105
+ :limit => sc.limit
106
+ })
107
+ end
108
+
109
+ t.column(sc.name, col_type, options)
110
+ end
111
+ end
112
+ end
113
+
114
+ def table_has_type_column(table)
115
+ Rmre::Source::Db.connection.columns(table).find {|col| col.name == 'type'}
116
+ end
117
+
118
+ def copy_data(table_name)
119
+ primary_key = Rmre::Source::Db.connection.primary_key(table_name)
120
+
121
+ src_model = Rmre::Source.create_model_for(table_name, primary_key)
122
+ src_model.inheritance_column = 'ruby_type' if table_has_type_column(table_name)
123
+ tgt_model = Rmre::Target.create_model_for(table_name, primary_key)
124
+
125
+ rec_count = src_model.count
126
+ # We will always copy attributes without protection because we
127
+ # are the ones initiating DB copy (no need to preform additional checks)
128
+ progress_bar = Console::ProgressBar.new(table_name, rec_count) if @verbose
129
+ src_model.all.each do |src_rec|
130
+ tgt_model.create!(src_rec.attributes, :without_protection => true)
131
+ progress_bar.inc if @verbose
132
+ end
133
+ end
134
+
135
+ private
136
+
137
+ def info(msg)
138
+ puts msg if @verbose
139
+ end
140
+ end
141
+ end
@@ -1,3 +1,3 @@
1
1
  module Rmre
2
- VERSION = "0.0.5" unless defined?(::Rmre::VERSION)
2
+ VERSION = "0.0.6" unless defined?(::Rmre::VERSION)
3
3
  end
@@ -18,6 +18,7 @@ Gem::Specification.new do |s|
18
18
 
19
19
  s.add_dependency "activerecord", ">= 3.0.0"
20
20
  s.add_dependency "erubis"
21
+ s.add_development_dependency "rake"
21
22
  s.add_development_dependency "rspec"
22
23
 
23
24
  s.files = `git ls-files`.split("\n")
@@ -8,17 +8,19 @@ module Rmre
8
8
  :username => 'user',
9
9
  :password => 'pass'},
10
10
  :out_path => File.join(Dir.tmpdir, 'gne-test'),
11
- :include => ['incl1_', 'incl2_']}
11
+ :include => ['incl1_', 'incl2_'],
12
+ :inflections => [{plural: ["(.*)_des$", '\1_des'], singular: ["(.*)_des$", '\1_des']}]
13
+ }
12
14
  end
13
15
 
14
16
  let(:generator) do |gen|
15
- gen = Generator.new(settings[:db], settings[:out_path], settings[:include])
17
+ gen = Generator.new(settings[:db], settings[:out_path], settings[:include], settings[:inflections])
16
18
  connection = double("db_connection")
17
19
  connection.stub(:columns).and_return([])
18
20
  gen.stub(:connection).and_return(connection)
19
21
  gen
20
22
  end
21
-
23
+
22
24
  let(:tables) { %w(incl1_tbl1 incl1_tbl2 incl2_tbl1 user processes) }
23
25
 
24
26
  it "should flag table incl1_tbl1 for processing" do
@@ -28,7 +30,7 @@ module Rmre
28
30
  it "should not flag table 'processes' for processing" do
29
31
  generator.process?('processes').should be_false
30
32
  end
31
-
33
+
32
34
  it "should process three tables from the passed array of tables" do
33
35
  generator.stub(:create_model)
34
36
 
@@ -53,7 +55,7 @@ module Rmre
53
55
  file.stub(:write)
54
56
 
55
57
  generator.connection.stub(:primary_key).and_return('')
56
-
58
+
57
59
  File.stub(:open).and_yield(file)
58
60
  File.should_receive(:open).with(/tbl_user/, "w")
59
61
  file.should_receive(:write).with(/class TblUser/)
@@ -61,16 +63,50 @@ module Rmre
61
63
  generator.create_model("TBL_USERS")
62
64
  end
63
65
 
64
- it "should set primary key if PK column is not id" do
65
- file = double("model_file")
66
- file.stub(:write)
66
+ context 'with non standard keys' do
67
+ before(:each) do
68
+ @file = double('model_file')
69
+ @file.stub(:write)
70
+ end
67
71
 
68
- generator.connection.stub(:primary_key).and_return("usr_id")
72
+ it "should set primary key if PK column is not id" do
73
+ generator.connection.stub(:primary_key).and_return('usr_id')
69
74
 
70
- File.stub(:open).and_yield(file)
71
- file.should_receive(:write).with(/self\.primary_key = :usr_id/)
75
+ File.stub(:open).and_yield(@file)
76
+ @file.should_receive(:write).with(/self\.primary_key = :usr_id/)
77
+
78
+ generator.create_model('users')
79
+ end
80
+
81
+ it "should set foreign key if FK column is not id" do
82
+ generator.connection.stub(:primary_key).and_return('pst_id')
83
+ generator.stub(:foreign_keys).and_return([
84
+ { 'from_table' => 'posts',
85
+ 'from_column' => 'pst_id',
86
+ 'to_table'=>'user',
87
+ 'to_column'=>'user_id'}
88
+ ])
89
+
90
+ File.stub(:open).and_yield(@file)
91
+ @file.should_receive(:write).with(/:foreign_key => :pst_id/)
92
+
93
+ generator.create_model('posts')
94
+ end
95
+ end
96
+
97
+ context 'irregular plural table names' do
98
+ it "should create correct file and class names" do
99
+ file = double("model_file")
100
+ file.stub(:write)
101
+
102
+ generator.connection.stub(:primary_key).and_return('')
103
+
104
+ File.stub(:open).and_yield(file)
105
+ File.should_receive(:open).with(/status_des/, "w")
106
+ file.should_receive(:write).with(/class StatusDes/)
72
107
 
73
- generator.create_model("users")
108
+ generator.create_model("status_des")
109
+ end
74
110
  end
75
111
  end
76
112
  end
@@ -0,0 +1,178 @@
1
+ require "spec_helper"
2
+
3
+ module Rmre
4
+ describe Migrator do
5
+ let(:src_connection) do |src_con|
6
+ src_con = double("source_connection")
7
+ end
8
+
9
+ let(:tgt_connection) do |tgt_con|
10
+ tgt_con = double("target_connection")
11
+ end
12
+
13
+ let(:src_db_opts) do |opts|
14
+ opts = { :adapter => "fake_adapter", :database => "source_db" }
15
+ end
16
+
17
+ let(:tgt_db_opts) do |opts|
18
+ opts = { :adapter => "fake_adapter", :database => "target_db" }
19
+ end
20
+
21
+ let(:id_column) do |col|
22
+ col = double("id_column")
23
+ col.stub!(:name).and_return("id")
24
+ col.stub!(:null).and_return(false)
25
+ col.stub!(:default).and_return(nil)
26
+ col.stub!(:type).and_return("integer")
27
+ col
28
+ end
29
+
30
+ let(:name_column) do |col|
31
+ col = double("name_column")
32
+ col.stub!(:name).and_return("name")
33
+ col.stub!(:null).and_return(false)
34
+ col.stub!(:default).and_return(nil)
35
+ col.stub!(:type).and_return("integer")
36
+ col
37
+ end
38
+
39
+ let(:table) do |tbl|
40
+ tbl = double("created_table")
41
+ tbl.stub!(:column)
42
+ tbl
43
+ end
44
+
45
+ before(:each) do
46
+ Source::Db.stub!(:establish_connection).and_return(true)
47
+ Source::Db.stub!(:connection).and_return(src_connection)
48
+
49
+ Target::Db.stub!(:establish_connection).and_return(true)
50
+ Target::Db.stub!(:connection).and_return(tgt_connection)
51
+ end
52
+
53
+ context "initialization" do
54
+ it "stores connection options in source and target modules" do
55
+ Migrator.new(src_db_opts, tgt_db_opts)
56
+ Source.connection_options.should be_eql(src_db_opts)
57
+ Target.connection_options.should be_eql(tgt_db_opts)
58
+ end
59
+
60
+ it "passes connection options to source and target connections" do
61
+ Source::Db.should_receive(:establish_connection).with(src_db_opts)
62
+ Target::Db.should_receive(:establish_connection).with(tgt_db_opts)
63
+ Migrator.new(src_db_opts, tgt_db_opts)
64
+ end
65
+ end
66
+
67
+ context "copying tables" do
68
+ before(:each) do
69
+ src_connection.stub(:tables).and_return %w{parent_table child_table}
70
+ src_connection.stub!(:columns).and_return([id_column, name_column])
71
+ src_connection.stub!(:primary_key).and_return("id")
72
+
73
+ @migrator = Migrator.new(src_db_opts, tgt_db_opts)
74
+ @migrator.stub!(:copy_data)
75
+ end
76
+
77
+ it "copies all tables if they do not exist" do
78
+ tgt_connection.should_receive(:table_exists?).exactly(2).times.and_return(false)
79
+ tgt_connection.should_receive(:create_table).exactly(2).times
80
+ @migrator.copy
81
+ end
82
+
83
+ it "doesn't copy tables if they exist" do
84
+ tgt_connection.should_receive(:table_exists?).exactly(2).times.and_return(true)
85
+ tgt_connection.should_not_receive(:create_table)
86
+ @migrator.copy
87
+ end
88
+
89
+ it "copies existing tables if it is forced to recreate them" do
90
+ tgt_connection.should_receive(:table_exists?).exactly(2).times.and_return(true)
91
+ tgt_connection.should_receive(:create_table).exactly(2).times
92
+ @migrator.copy(true)
93
+ end
94
+
95
+ context "with before_copy filter" do
96
+ before(:each) do
97
+ @migrator.before_copy = lambda { |table_name|
98
+ return false if table_name == "child_table"
99
+ true
100
+ }
101
+ end
102
+
103
+ it "does not copy table if before copy filter returns false" do
104
+ tgt_connection.should_receive(:table_exists?).with("parent_table").and_return(false)
105
+ tgt_connection.should_receive(:create_table).once
106
+ @migrator.copy
107
+ end
108
+ end
109
+ end
110
+
111
+ context "copying tables with 'skip existing' turned on" do
112
+ before(:each) do
113
+ src_connection.stub(:tables).and_return %w{parent_table child_table}
114
+ src_connection.stub!(:columns).and_return([id_column, name_column])
115
+
116
+ @migrator = Migrator.new(src_db_opts, tgt_db_opts, {:skip_existing => true})
117
+ end
118
+
119
+ it "skips existing tables" do
120
+ tgt_connection.should_receive(:table_exists?).exactly(2).times.and_return(true)
121
+ tgt_connection.should_not_receive(:create_table)
122
+ @migrator.copy
123
+ end
124
+ end
125
+
126
+ context "table creation" do
127
+ before(:each) do
128
+ @source_columns = [id_column, name_column]
129
+ end
130
+
131
+ context "Rails copy mode" do
132
+ before(:each) do
133
+ @migrator = Migrator.new(src_db_opts, tgt_db_opts)
134
+ src_connection.stub!(:primary_key).and_return("id")
135
+ tgt_connection.stub!(:adapter_name).and_return("fake adapter")
136
+ end
137
+
138
+ it "does not explicitely create ID column" do
139
+ tgt_connection.should_receive(:create_table).with("parent", {:id => true, :force => false}).
140
+ and_yield(table)
141
+ table.should_not_receive(:column).with("id", anything(), anything())
142
+ @migrator.create_table("parent", @source_columns)
143
+ end
144
+
145
+ it "creates other columns but ID column" do
146
+ tgt_connection.should_receive(:create_table).with("parent", {:id => true, :force => false}).
147
+ and_yield(table)
148
+ table.should_receive(:column).with("name", anything(), anything())
149
+ @migrator.create_table("parent", @source_columns)
150
+ end
151
+ end
152
+
153
+ context "non-Rails copy mode" do
154
+ before(:each) do
155
+ @migrator = Migrator.new(src_db_opts, tgt_db_opts, {:rails_copy_mode => false})
156
+ tgt_connection.stub!(:adapter_name).times.and_return("fake adapter")
157
+ src_connection.stub!(:primary_key).and_return("primaryIdColumn")
158
+ end
159
+
160
+ it "explicitely creates ID column" do
161
+ tgt_connection.should_receive(:create_table).with("parent",
162
+ {:id => true, :force => false, :primary_key => "primaryIdColumn" }).
163
+ and_yield(table)
164
+ table.should_receive(:column).with("id", anything(), anything())
165
+ @migrator.create_table("parent", @source_columns)
166
+ end
167
+
168
+ it "creates other columns too" do
169
+ tgt_connection.should_receive(:create_table).with("parent",
170
+ {:id => true, :force => false, :primary_key => "primaryIdColumn"}).
171
+ and_yield(table)
172
+ table.should_receive(:column).with("name", anything(), anything())
173
+ @migrator.create_table("parent", @source_columns)
174
+ end
175
+ end
176
+ end
177
+ end
178
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rmre
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.5
4
+ version: 0.0.6
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2012-11-04 00:00:00.000000000 Z
12
+ date: 2013-05-09 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: activerecord
@@ -43,6 +43,22 @@ dependencies:
43
43
  - - ! '>='
44
44
  - !ruby/object:Gem::Version
45
45
  version: '0'
46
+ - !ruby/object:Gem::Dependency
47
+ name: rake
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - ! '>='
52
+ - !ruby/object:Gem::Version
53
+ version: '0'
54
+ type: :development
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - ! '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
46
62
  - !ruby/object:Gem::Dependency
47
63
  name: rspec
48
64
  requirement: !ruby/object:Gem::Requirement
@@ -64,6 +80,7 @@ description: Rmre creates ActiveRecord models for legacy database with all const
64
80
  email:
65
81
  - bosko.ivanisevic@gmail.com
66
82
  executables:
83
+ - db_copy
67
84
  - rmre
68
85
  extensions: []
69
86
  extra_rdoc_files:
@@ -77,15 +94,21 @@ files:
77
94
  - LICENSE.txt
78
95
  - README.rdoc
79
96
  - Rakefile
97
+ - bin/db_copy
80
98
  - bin/rmre
99
+ - lib/contrib/progressbar.rb
81
100
  - lib/rmre.rb
82
101
  - lib/rmre/active_record/schema_dumper.rb
102
+ - lib/rmre/db_utils.rb
103
+ - lib/rmre/dynamic_db.rb
83
104
  - lib/rmre/generator.rb
84
105
  - lib/rmre/load_file.eruby
106
+ - lib/rmre/migrator.rb
85
107
  - lib/rmre/model.eruby
86
108
  - lib/rmre/version.rb
87
109
  - rmre.gemspec
88
110
  - spec/rmre/generator_spec.rb
111
+ - spec/rmre/migrator_spec.rb
89
112
  - spec/spec_helper.rb
90
113
  homepage: http://github.com/bosko/rmre
91
114
  licenses: []
@@ -105,7 +128,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
105
128
  version: '0'
106
129
  segments:
107
130
  - 0
108
- hash: -146509643511923988
131
+ hash: -360534453122830700
109
132
  required_rubygems_version: !ruby/object:Gem::Requirement
110
133
  none: false
111
134
  requirements:
@@ -120,4 +143,5 @@ specification_version: 3
120
143
  summary: The easiest way to create ActiveRecord models for legacy database
121
144
  test_files:
122
145
  - spec/rmre/generator_spec.rb
146
+ - spec/rmre/migrator_spec.rb
123
147
  - spec/spec_helper.rb