ironmin_yaml_db 0.0.2

Sign up to get free protection for your applications and to get access to all the features.
data/.travis.yml ADDED
@@ -0,0 +1,5 @@
1
+ language: ruby
2
+ rvm:
3
+ - 1.8.7
4
+ - 1.9.2
5
+ - 1.9.3
data/README.markdown ADDED
@@ -0,0 +1,42 @@
1
+ # IronmineYamlDb
2
+
3
+ IronmineYamlDb focked YamlDb, it is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
4
+
5
+ This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
6
+
7
+ Any database that has an ActiveRecord adapter should work. This gem is now Rails 3 only. For Rails 2, clone and checkout the Rails2 branch.
8
+
9
+ [![Build Status](https://secure.travis-ci.org/ludicast/yaml_db.png)](http://travis-ci.org/ludicast/yaml_db)
10
+
11
+ ## Installation
12
+
13
+ Simply add to your Gemfile:
14
+
15
+ gem 'ironmine_yaml_db'
16
+
17
+ All rake tasks will then be available to you.
18
+
19
+ ## Usage
20
+
21
+ rake db:data:dump -> Dump contents of Rails database to db/data.yml
22
+ rake db:data:load -> Load contents of db/data.yml into the database
23
+
24
+ Further, there are tasks db:dump and db:load which do the entire database (the equivalent of running db:schema:dump followed by db:data:load). Also, there are other tasks recently added that allow the export of the database contents to/from multiple files (each one named after the table being dumped or loaded).
25
+
26
+ rake db:data:dump_dir -> Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)
27
+ rake db:data:load_dir -> Load contents of db/data_dir into database
28
+
29
+ In addition, we have plugins whereby you can export your database to/from various formats. We only deal with yaml and csv right now, but you can easily write tools for your own formats (such as Excel or XML). To use another format, just load setting the "class" parameter to the class you are using. This defaults to "YamlDb::Helper" which is a refactoring of the old yaml_db code. We'll shorten this to use class nicknames in a little bit.
30
+
31
+ ## Examples
32
+
33
+ One common use would be to switch your data from one database backend to another. For example, let's say you wanted to switch from SQLite to MySQL. You might execute the following steps:
34
+
35
+ 1. rake db:dump
36
+
37
+ 2. Edit config/database.yml and change your adapter to mysql, set up database params
38
+
39
+ 3. mysqladmin create [database name]
40
+
41
+ 4. rake db:load
42
+
data/Rakefile ADDED
@@ -0,0 +1,29 @@
1
+ require 'rake'
2
+ require "rspec/core/rake_task"
3
+
4
+ begin
5
+ require 'jeweler'
6
+ Jeweler::Tasks.new do |gem|
7
+ gem.name = "ironmine_yaml_db"
8
+ gem.summary = %Q{ironmine_yaml_db allows export/import of database into/from yaml files}
9
+ gem.description = %Q{
10
+ IronmineYamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
11
+ This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
12
+ Any database that has an ActiveRecord adapter should work
13
+ }
14
+ gem.email = "442453867@qq.com"
15
+ gem.homepage = "http://github.com/uudui/ironmine_yaml_db"
16
+ gem.authors = ["uudui"]
17
+ end
18
+ Jeweler::GemcutterTasks.new
19
+ rescue LoadError
20
+ puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
21
+ end
22
+
23
+
24
+ RSpec::Core::RakeTask.new(:spec) do |spec|
25
+ spec.pattern = 'spec/*_spec.rb'
26
+ spec.rspec_opts = ['--backtrace']
27
+ end
28
+
29
+ task :default => :spec
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.0.2
data/about.yml ADDED
@@ -0,0 +1,4 @@
1
+ author: uudui
2
+ summary: Dumps and loads a database-independent data dump format in db/data.yml.
3
+ license: MIT
4
+ rails_version: 3.0+
data/init.rb ADDED
@@ -0,0 +1 @@
1
+ require 'ironmine_yaml_db'
@@ -0,0 +1,53 @@
1
+ # Generated by jeweler
2
+ # DO NOT EDIT THIS FILE DIRECTLY
3
+ # Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec'
4
+ # -*- encoding: utf-8 -*-
5
+
6
+ Gem::Specification.new do |s|
7
+ s.name = "ironmin_yaml_db"
8
+ s.version = "0.0.2"
9
+
10
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11
+ s.authors = ["uudui"]
12
+ s.date = "2013-07-15"
13
+ s.description = "\nIronmineYamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.\nThis can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.\nAny database that has an ActiveRecord adapter should work\n"
14
+ s.email = "442453867@qq.com"
15
+ s.extra_rdoc_files = [
16
+ "README.markdown"
17
+ ]
18
+ s.files = [
19
+ ".travis.yml",
20
+ "README.markdown",
21
+ "Rakefile",
22
+ "VERSION",
23
+ "about.yml",
24
+ "init.rb",
25
+ "lib/csv_db.rb",
26
+ "lib/serialization_helper.rb",
27
+ "lib/tasks/yaml_db_tasks.rake",
28
+ "lib/yaml_db.rb",
29
+ "spec/base.rb",
30
+ "spec/serialization_helper_base_spec.rb",
31
+ "spec/serialization_helper_dump_spec.rb",
32
+ "spec/serialization_helper_load_spec.rb",
33
+ "spec/serialization_utils_spec.rb",
34
+ "spec/yaml_dump_spec.rb",
35
+ "spec/yaml_load_spec.rb",
36
+ "spec/yaml_utils_spec.rb",
37
+ "ironmine_yaml_db.gemspec"
38
+ ]
39
+ s.homepage = "http://github.com/uudui/ironmine_yaml_db"
40
+ s.require_paths = ["lib"]
41
+ s.rubygems_version = "1.8.17"
42
+ s.summary = "ironmine_yaml_db allows export/import of database into/from yaml files"
43
+
44
+ if s.respond_to? :specification_version then
45
+ s.specification_version = 3
46
+
47
+ if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
48
+ else
49
+ end
50
+ else
51
+ end
52
+ end
53
+
data/lib/csv_db.rb ADDED
@@ -0,0 +1,78 @@
1
+ #require 'FasterCSV'
2
+ module CsvDb
3
+ module Helper
4
+ def self.loader
5
+ Load
6
+ end
7
+
8
+ def self.dumper
9
+ Dump
10
+ end
11
+
12
+ def self.extension
13
+ "csv"
14
+ end
15
+ end
16
+
17
+ class Load < SerializationHelper::Load
18
+ def self.load_documents(io, truncate = true)
19
+ tables = {}
20
+ curr_table = nil
21
+ io.each do |line|
22
+ if /BEGIN_CSV_TABLE_DECLARATION(.+)END_CSV_TABLE_DECLARATION/ =~ line
23
+ curr_table = $1
24
+ tables[curr_table] = {}
25
+ else
26
+ if tables[curr_table]["columns"]
27
+ tables[curr_table]["records"] << FasterCSV.parse(line)[0]
28
+ else
29
+ tables[curr_table]["columns"] = FasterCSV.parse(line)[0]
30
+ tables[curr_table]["records"] = []
31
+ end
32
+ end
33
+ end
34
+
35
+ tables.each_pair do |table_name, contents|
36
+ load_table(table_name, contents, truncate)
37
+ end
38
+ end
39
+ end
40
+
41
+ class Dump < SerializationHelper::Dump
42
+
43
+ def self.before_table(io,table)
44
+ io.write "BEGIN_CSV_TABLE_DECLARATION#{table}END_CSV_TABLE_DECLARATION\n"
45
+ end
46
+
47
+ def self.dump(io)
48
+ tables.each do |table|
49
+ before_table(io, table)
50
+ dump_table(io, table)
51
+ after_table(io, table)
52
+ end
53
+ end
54
+
55
+ def self.after_table(io,table)
56
+ io.write ""
57
+ end
58
+
59
+ def self.dump_table_columns(io, table)
60
+ io.write(table_column_names(table).to_csv)
61
+ end
62
+
63
+ def self.dump_table_records(io, table)
64
+
65
+ column_names = table_column_names(table)
66
+
67
+ each_table_page(table) do |records|
68
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
69
+ records.each do |record|
70
+ io.write(record.to_csv)
71
+ end
72
+ end
73
+ end
74
+
75
+ end
76
+
77
+
78
+ end
@@ -0,0 +1,216 @@
1
+ module SerializationHelper
2
+
3
+ class Base
4
+ attr_reader :extension
5
+
6
+ def initialize(helper)
7
+ @dumper = helper.dumper
8
+ @loader = helper.loader
9
+ @extension = helper.extension
10
+ end
11
+
12
+ def dump(filename)
13
+ disable_logger
14
+ @dumper.dump(File.new(filename, "w"))
15
+ reenable_logger
16
+ end
17
+
18
+ def dump_to_dir(dirname)
19
+ Dir.mkdir(dirname)
20
+ tables = @dumper.tables
21
+ tables.each do |table|
22
+ io = File.new "#{dirname}/#{table}.#{@extension}", "w"
23
+ @dumper.before_table(io, table)
24
+ @dumper.dump_table io, table
25
+ @dumper.after_table(io, table)
26
+ end
27
+ end
28
+
29
+ def load(filename, truncate = true)
30
+ disable_logger
31
+ @loader.load(File.new(filename, "r"), truncate)
32
+ reenable_logger
33
+ end
34
+
35
+ def load_from_dir(dirname, truncate = true)
36
+ Dir.entries(dirname).each do |filename|
37
+ if filename =~ /^[.]/
38
+ next
39
+ end
40
+ @loader.load(File.new("#{dirname}/#{filename}", "r"), truncate)
41
+ end
42
+ end
43
+
44
+ def disable_logger
45
+ @@old_logger = ActiveRecord::Base.logger
46
+ ActiveRecord::Base.logger = nil
47
+ end
48
+
49
+ def reenable_logger
50
+ ActiveRecord::Base.logger = @@old_logger
51
+ end
52
+ end
53
+
54
+ class Load
55
+ def self.load(io, truncate = true)
56
+ ActiveRecord::Base.connection.transaction do
57
+ load_documents(io, truncate)
58
+ end
59
+ end
60
+
61
+ def self.truncate_table(table)
62
+ begin
63
+ ActiveRecord::Base.connection.execute("DELETE FROM #{SerializationHelper::Utils.quote_table(table)}")
64
+ rescue Exception
65
+ ActiveRecord::Base.connection.execute("TRUNCATE #{SerializationHelper::Utils.quote_table(table)}")
66
+ end
67
+ end
68
+
69
+ def self.load_table(table, data, truncate = true)
70
+ column_names = data['columns']
71
+ if truncate
72
+ truncate_table(table)
73
+ end
74
+ load_records(table, column_names, data['records'])
75
+ reset_pk_sequence!(table)
76
+ end
77
+
78
+ def self.load_records(table, column_names, records)
79
+ if column_names.nil?
80
+ return
81
+ end
82
+ columns = column_names.map{|cn| ActiveRecord::Base.connection.columns(table).detect{|c| c.name == cn}}
83
+ quoted_column_names = column_names.map { |column| ActiveRecord::Base.connection.quote_column_name(column) }.join(',')
84
+ quoted_table_name = SerializationHelper::Utils.quote_table(table)
85
+ puts "Synch #{table}'s data......"
86
+ records.each do |record|
87
+ quoted_values = record.zip(columns).map{|c| ActiveRecord::Base.connection.quote(c.first, c.last)}.join(',')
88
+
89
+ ActiveRecord::Base.connection.execute("INSERT INTO #{quoted_table_name} (#{quoted_column_names}) VALUES (#{quoted_values})")
90
+ end
91
+ end
92
+
93
+ def self.reset_pk_sequence!(table_name)
94
+ if ActiveRecord::Base.connection.respond_to?(:reset_pk_sequence!)
95
+ ActiveRecord::Base.connection.reset_pk_sequence!(table_name)
96
+ end
97
+ end
98
+
99
+
100
+ end
101
+
102
+ module Utils
103
+
104
+ def self.unhash(hash, keys)
105
+ column_hash = keys.dup
106
+ keys = column_hash.keys
107
+
108
+ keys.map { |key|
109
+ if hash[key].blank? && !column_hash[key]
110
+ '0'
111
+ else
112
+ hash[key]
113
+ end
114
+ }
115
+ end
116
+
117
+ def self.unhash_records(records, keys)
118
+ records.each_with_index do |record, index|
119
+ records[index] = unhash(record, keys)
120
+ end
121
+
122
+ records
123
+ end
124
+
125
+ def self.convert_booleans(records, columns)
126
+ records.each do |record|
127
+ columns.each do |column|
128
+ next if is_boolean(record[column])
129
+ record[column] = convert_boolean(record[column])
130
+ end
131
+ end
132
+ records
133
+ end
134
+
135
+ def self.convert_boolean(value)
136
+ ['t', '1', true, 1].include?(value)
137
+ end
138
+
139
+ def self.boolean_columns(table)
140
+ columns = ActiveRecord::Base.connection.columns(table).reject { |c| silence_warnings { c.type != :boolean } }
141
+ columns.map { |c| c.name }
142
+ end
143
+
144
+ def self.is_boolean(value)
145
+ value.kind_of?(TrueClass) or value.kind_of?(FalseClass)
146
+ end
147
+
148
+ def self.quote_table(table)
149
+ ActiveRecord::Base.connection.quote_table_name(table)
150
+ end
151
+
152
+ end
153
+
154
+ class Dump
155
+ def self.before_table(io, table)
156
+
157
+ end
158
+
159
+ def self.dump(io)
160
+ base_tables.each do |table|
161
+ before_table(io, table)
162
+ dump_table(io, table)
163
+ after_table(io, table)
164
+ end
165
+ end
166
+
167
+ def self.after_table(io, table)
168
+
169
+ end
170
+
171
+ def self.base_tables
172
+ sql = "select table_name from information_schema.tables where table_schema='irm_dev' and table_type='base table';"
173
+ base_tables = ActiveRecord::Base.connection.execute(sql, 'SCHEMA').collect do |field|
174
+ field.first
175
+ end
176
+ base_tables.reject { |table| ['schema_info', 'schema_migrations','irm_data_accesses_top_org_t'].include?(table) }
177
+ end
178
+
179
+ def self.tables
180
+ ActiveRecord::Base.connection.tables.reject { |table| ['schema_info', 'schema_migrations'].include?(table) }
181
+ end
182
+
183
+ def self.dump_table(io, table)
184
+ return if table_record_count(table).zero?
185
+
186
+ dump_table_columns(io, table)
187
+ dump_table_records(io, table)
188
+ end
189
+
190
+ def self.table_column_names(table)
191
+ ActiveRecord::Base.connection.columns(table).map { |c| c.name }
192
+ end
193
+
194
+
195
+ def self.each_table_page(table, records_per_page=1000)
196
+ total_count = table_record_count(table)
197
+ pages = (total_count.to_f / records_per_page).ceil - 1
198
+ id = table_column_names(table).first
199
+ boolean_columns = SerializationHelper::Utils.boolean_columns(table)
200
+ quoted_table_name = SerializationHelper::Utils.quote_table(table)
201
+
202
+ (0..pages).to_a.each do |page|
203
+ query = Arel::Table.new(table).order(id).skip(records_per_page*page).take(records_per_page).project(Arel.sql('*'))
204
+ records = ActiveRecord::Base.connection.select_all(query)
205
+ records = SerializationHelper::Utils.convert_booleans(records, boolean_columns)
206
+ yield records
207
+ end
208
+ end
209
+
210
+ def self.table_record_count(table)
211
+ ActiveRecord::Base.connection.select_one("SELECT COUNT(*) FROM #{SerializationHelper::Utils.quote_table(table)}").values.first.to_i
212
+ end
213
+
214
+ end
215
+
216
+ end
@@ -0,0 +1,45 @@
1
+ namespace :db do
2
+ desc "Dump schema and data to db/schema.rb and db/data.yml"
3
+ task(:dump => [ "db:schema:dump", "db:data:dump" ])
4
+
5
+ desc "Load schema and data from db/schema.rb and db/data.yml"
6
+ task(:load => [ "db:schema:load", "db:data:load" ])
7
+
8
+ namespace :data do
9
+ def db_dump_data_file (extension = "yml")
10
+ "#{dump_dir}/data.#{extension}"
11
+ end
12
+
13
+ def dump_dir(dir = "")
14
+ "#{Rails.root}/db#{dir}"
15
+ end
16
+
17
+ desc "Dump contents of database to db/data.extension (defaults to yaml)"
18
+ task :dump => :environment do
19
+ format_class = ENV['class'] || "YamlDb::Helper"
20
+ helper = format_class.constantize
21
+ SerializationHelper::Base.new(helper).dump db_dump_data_file helper.extension
22
+ end
23
+
24
+ desc "Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)"
25
+ task :dump_dir => :environment do
26
+ format_class = ENV['class'] || "YamlDb::Helper"
27
+ dir = ENV['dir'] || "#{Time.now.to_s.gsub(/ /, '_')}"
28
+ SerializationHelper::Base.new(format_class.constantize).dump_to_dir dump_dir("/#{dir}")
29
+ end
30
+
31
+ desc "Load contents of db/data.extension (defaults to yaml) into database"
32
+ task :load => :environment do
33
+ format_class = ENV['class'] || "YamlDb::Helper"
34
+ helper = format_class.constantize
35
+ SerializationHelper::Base.new(helper).load(db_dump_data_file helper.extension)
36
+ end
37
+
38
+ desc "Load contents of db/data_dir into database"
39
+ task :load_dir => :environment do
40
+ dir = ENV['dir'] || "base"
41
+ format_class = ENV['class'] || "YamlDb::Helper"
42
+ SerializationHelper::Base.new(format_class.constantize).load_from_dir dump_dir("/#{dir}")
43
+ end
44
+ end
45
+ end
data/lib/yaml_db.rb ADDED
@@ -0,0 +1,80 @@
1
+ require 'rubygems'
2
+ require 'yaml'
3
+ require 'active_record'
4
+ require 'serialization_helper'
5
+ require 'active_support/core_ext/kernel/reporting'
6
+ require 'rails/railtie'
7
+
8
+ module YamlDb
9
+ module Helper
10
+ def self.loader
11
+ YamlDb::Load
12
+ end
13
+
14
+ def self.dumper
15
+ YamlDb::Dump
16
+ end
17
+
18
+ def self.extension
19
+ "yml"
20
+ end
21
+ end
22
+
23
+
24
+ module Utils
25
+ def self.chunk_records(records)
26
+ yaml = [ records ].to_yaml
27
+ yaml.sub!(/---\s\n|---\n/, '')
28
+ yaml.sub!('- - -', ' - -')
29
+ yaml
30
+ end
31
+
32
+ end
33
+
34
+ class Dump < SerializationHelper::Dump
35
+
36
+ def self.dump_table_columns(io, table)
37
+ io.write("\n")
38
+ io.write({ table => { 'columns' => table_column_names(table) } }.to_yaml)
39
+ end
40
+
41
+ def self.dump_table_records(io, table)
42
+ table_record_header(io)
43
+ column_names = {}
44
+ ActiveRecord::Base.connection.columns(table).each do |c|
45
+ column_names[c.name] = c.null
46
+ end
47
+ each_table_page(table) do |records|
48
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
49
+ io.write(YamlDb::Utils.chunk_records(records))
50
+ end
51
+ end
52
+
53
+ def self.table_record_header(io)
54
+ io.write(" records: \n")
55
+ end
56
+
57
+ end
58
+
59
+ class Load < SerializationHelper::Load
60
+ def self.load_documents(io, truncate = true)
61
+ YAML.load_documents(io) do |ydoc|
62
+ ydoc.keys.each do |table_name|
63
+ next if ydoc[table_name].nil?
64
+ if ActiveRecord::Base.connection.table_exists?(table_name)
65
+ load_table(table_name, ydoc[table_name], truncate)
66
+ else
67
+ puts "Table: #{table_name} is not exists."
68
+ end
69
+ end
70
+ end
71
+ end
72
+ end
73
+
74
+ class Railtie < Rails::Railtie
75
+ rake_tasks do
76
+ load File.expand_path('../tasks/yaml_db_tasks.rake', __FILE__)
77
+ end
78
+ end
79
+
80
+ end
data/spec/base.rb ADDED
@@ -0,0 +1,4 @@
1
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
2
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
3
+ require 'rspec'
4
+ require 'ironmine_yaml_db'
@@ -0,0 +1,56 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Base do
4
+ def prestub_active_record
5
+
6
+ end
7
+
8
+ before do
9
+ @io = StringIO.new
10
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
11
+ ActiveRecord::Base.stub(:connection).and_return(mock('connection'))
12
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
13
+ end
14
+
15
+ def stub_helper!
16
+ @helper = mock("MyHelper")
17
+ @dumper = mock("MyDumper");
18
+ @loader = mock("MyLoader");
19
+ @helper.stub!(:dumper).and_return(@dumper)
20
+ @helper.stub!(:loader).and_return(@loader)
21
+ @helper.stub!(:extension).and_return("yml")
22
+ @dumper.stub!(:tables).and_return([ActiveRecord::Base.connection.tables[0]])
23
+ @dumper.stub!(:before_table).and_return(nil)
24
+ @dumper.stub!(:after_table).and_return(nil)
25
+ end
26
+
27
+ context "for multi-file dumps" do
28
+ before do
29
+ File.should_receive(:new).once.with("dir_name/mytable.yml", "w").and_return(@io)
30
+ Dir.should_receive(:mkdir).once.with("dir_name")
31
+ stub_helper!
32
+ @dumper.should_receive(:dump_table).once.with(@io, "mytable")
33
+ end
34
+
35
+ it "should create the number of files that there are tables" do
36
+ SerializationHelper::Base.new(@helper).dump_to_dir "dir_name"
37
+ end
38
+
39
+ end
40
+
41
+ context "for multi-file loads" do
42
+
43
+ before do
44
+ stub_helper!
45
+ @loader.should_receive(:load).once.with(@io, true)
46
+ File.should_receive(:new).once.with("dir_name/mytable.yml", "r").and_return(@io)
47
+ Dir.stub!(:entries).and_return(["mytable.yml"])
48
+ end
49
+
50
+ it "should insert into then umber of tables that there are files" do
51
+ SerializationHelper::Base.new(@helper).load_from_dir "dir_name"
52
+ end
53
+
54
+ end
55
+
56
+ end
@@ -0,0 +1,65 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Dump do
4
+
5
+ before do
6
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a', :name => 'a', :type => :string), mock('b', :name => 'b', :type => :string) ])
10
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
11
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
12
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
13
+ end
14
+
15
+ before(:each) do
16
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
17
+ @io = StringIO.new
18
+ end
19
+
20
+ it "should return a list of column names" do
21
+ SerializationHelper::Dump.table_column_names('mytable').should == [ 'a', 'b' ]
22
+ end
23
+
24
+ it "should return a list of tables without the rails schema table" do
25
+ SerializationHelper::Dump.tables.should == ['mytable']
26
+ end
27
+
28
+ it "should return the total number of records in a table" do
29
+ SerializationHelper::Dump.table_record_count('mytable').should == 2
30
+ end
31
+
32
+ it "should return all records from the database and return them when there is only 1 page" do
33
+ SerializationHelper::Dump.each_table_page('mytable') do |records|
34
+ records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
35
+ end
36
+ end
37
+
38
+ it "should paginate records from the database and return them" do
39
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 } ], [ { 'a' => 3, 'b' => 4 } ])
40
+
41
+ records = [ ]
42
+ SerializationHelper::Dump.each_table_page('mytable', 1) do |page|
43
+ page.size.should == 1
44
+ records.concat(page)
45
+ end
46
+
47
+ records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
48
+ end
49
+
50
+ it "should dump a table's contents to yaml" do
51
+ SerializationHelper::Dump.should_receive(:dump_table_columns)
52
+ SerializationHelper::Dump.should_receive(:dump_table_records)
53
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
54
+ end
55
+
56
+ it "should not dump a table's contents when the record count is zero" do
57
+ SerializationHelper::Dump.stub!(:table_record_count).with('mytable').and_return(0)
58
+ SerializationHelper::Dump.should_not_receive(:dump_table_columns)
59
+ SerializationHelper::Dump.should_not_receive(:dump_table_records)
60
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
61
+ end
62
+
63
+
64
+
65
+ end
@@ -0,0 +1,77 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Load do
4
+ before do
5
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
6
+
7
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
8
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
9
+ ActiveRecord::Base.connection.stub!(:transaction).and_yield
10
+ @io = StringIO.new
11
+ end
12
+
13
+ it "should truncate the table" do
14
+ ActiveRecord::Base.connection.stub!(:execute).with("TRUNCATE mytable").and_return(true)
15
+ ActiveRecord::Base.connection.should_not_receive(:execute).with("DELETE FROM mytable")
16
+ SerializationHelper::Load.truncate_table('mytable')
17
+ end
18
+
19
+ it "should delete the table if truncate throws an exception" do
20
+ ActiveRecord::Base.connection.should_receive(:execute).with("TRUNCATE mytable").and_raise()
21
+ ActiveRecord::Base.connection.should_receive(:execute).with("DELETE FROM mytable").and_return(true)
22
+ SerializationHelper::Load.truncate_table('mytable')
23
+ end
24
+
25
+
26
+ it "should call reset pk sequence if the connection adapter is postgres" do
27
+ ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(true)
28
+ ActiveRecord::Base.connection.should_receive(:reset_pk_sequence!).with('mytable')
29
+ SerializationHelper::Load.reset_pk_sequence!('mytable')
30
+ end
31
+
32
+ it "should not call reset pk sequence for other adapters" do
33
+ ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(false)
34
+ ActiveRecord::Base.connection.should_not_receive(:reset_pk_sequence!)
35
+ SerializationHelper::Load.reset_pk_sequence!('mytable')
36
+ end
37
+
38
+ it "should insert records into a table" do
39
+ mca = mock('a',:name => 'a')
40
+ mcb = mock('b', :name => 'b')
41
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([mca , mcb ])
42
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
43
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('b').and_return('b')
44
+ ActiveRecord::Base.connection.stub!(:quote).with(1, mca).and_return("'1'")
45
+ ActiveRecord::Base.connection.stub!(:quote).with(2, mcb).and_return("'2'")
46
+ ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
47
+ ActiveRecord::Base.connection.stub!(:quote).with(4, mcb).and_return("'4'")
48
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('1','2')")
49
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('3','4')")
50
+
51
+ SerializationHelper::Load.load_records('mytable', ['a', 'b'], [[1, 2], [3, 4]])
52
+ end
53
+
54
+ it "should quote column names that correspond to sql keywords" do
55
+ mca = mock('a',:name => 'a')
56
+ mccount = mock('count', :name => 'count')
57
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([mca , mccount ])
58
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
59
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('count').and_return('"count"')
60
+ ActiveRecord::Base.connection.stub!(:quote).with(1, mca).and_return("'1'")
61
+ ActiveRecord::Base.connection.stub!(:quote).with(2, mccount).and_return("'2'")
62
+ ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
63
+ ActiveRecord::Base.connection.stub!(:quote).with(4, mccount).and_return("'4'")
64
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('1','2')")
65
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('3','4')")
66
+
67
+ SerializationHelper::Load.load_records('mytable', ['a', 'count'], [[1, 2], [3, 4]])
68
+ end
69
+
70
+ it "should truncate the table and then load the records into the table" do
71
+ SerializationHelper::Load.should_receive(:truncate_table).with('mytable')
72
+ SerializationHelper::Load.should_receive(:load_records).with('mytable', ['a', 'b'], [[1, 2], [3, 4]])
73
+ SerializationHelper::Load.should_receive(:reset_pk_sequence!).with('mytable')
74
+
75
+ SerializationHelper::Load.load_table('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] })
76
+ end
77
+ end
@@ -0,0 +1,51 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Utils, " convert records utility method" do
4
+ before do
5
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
6
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
7
+ end
8
+
9
+ it "returns an array of hash values using an array of ordered keys" do
10
+ SerializationHelper::Utils.unhash({ 'a' => 1, 'b' => 2 }, [ 'b', 'a' ]).should == [ 2, 1 ]
11
+ end
12
+
13
+ it "should unhash each hash an array using an array of ordered keys" do
14
+ SerializationHelper::Utils.unhash_records([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ], [ 'b', 'a' ]).should == [ [ 2, 1 ], [ 4, 3 ] ]
15
+ end
16
+
17
+ it "should return true if it is a boolean type" do
18
+ SerializationHelper::Utils.is_boolean(true).should == true
19
+ SerializationHelper::Utils.is_boolean('true').should_not == true
20
+ end
21
+
22
+ it "should return an array of boolean columns" do
23
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a',:type => :string), mock('b', :name => 'b',:type => :boolean) ])
24
+ SerializationHelper::Utils.boolean_columns('mytable').should == ['b']
25
+ end
26
+
27
+ it "should quote the table name" do
28
+ ActiveRecord::Base.connection.should_receive(:quote_table_name).with('values').and_return('`values`')
29
+ SerializationHelper::Utils.quote_table('values').should == '`values`'
30
+ end
31
+
32
+ it "should convert ruby booleans to true and false" do
33
+ SerializationHelper::Utils.convert_boolean(true).should == true
34
+ SerializationHelper::Utils.convert_boolean(false).should == false
35
+ end
36
+
37
+ it "should convert ruby string t and f to true and false" do
38
+ SerializationHelper::Utils.convert_boolean('t').should == true
39
+ SerializationHelper::Utils.convert_boolean('f').should == false
40
+ end
41
+
42
+ it "should convert ruby string 1 and 0 to true and false" do
43
+ SerializationHelper::Utils.convert_boolean('1').should == true
44
+ SerializationHelper::Utils.convert_boolean('0').should == false
45
+ end
46
+
47
+ it "should convert ruby integer 1 and 0 to true and false" do
48
+ SerializationHelper::Utils.convert_boolean(1).should == true
49
+ SerializationHelper::Utils.convert_boolean(0).should == false
50
+ end
51
+ end
@@ -0,0 +1,59 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe YamlDb::Dump do
4
+
5
+ before do
6
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a', :type => :string), mock('b', :name => 'b', :type => :string) ])
10
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
11
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
12
+ YamlDb::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
13
+ end
14
+
15
+ before(:each) do
16
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
17
+ @io = StringIO.new
18
+ end
19
+
20
+ it "should return a formatted string" do
21
+ YamlDb::Dump.table_record_header(@io)
22
+ @io.rewind
23
+ @io.read.should == " records: \n"
24
+ end
25
+
26
+
27
+ it "should return a yaml string that contains a table header and column names" do
28
+ if RUBY_VERSION.split(".")[1] == "9"
29
+
30
+ YAML::ENGINE.yamler = "syck"
31
+ end
32
+ YamlDb::Dump.stub!(:table_column_names).with('mytable').and_return([ 'a', 'b' ])
33
+ YamlDb::Dump.dump_table_columns(@io, 'mytable')
34
+ @io.rewind
35
+ @io.read.should == <<EOYAML
36
+
37
+ ---
38
+ mytable:
39
+ columns:
40
+ - a
41
+ - b
42
+ EOYAML
43
+ end
44
+
45
+ it "should return dump the records for a table in yaml to a given io stream" do
46
+ YamlDb::Dump.dump_table_records(@io, 'mytable')
47
+ @io.rewind
48
+ @io.read.should == <<EOYAML
49
+ records:
50
+ - - 1
51
+ - 2
52
+ - - 3
53
+ - 4
54
+ EOYAML
55
+ end
56
+
57
+
58
+
59
+ end
File without changes
@@ -0,0 +1,21 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe YamlDb::Utils, " convert records utility method" do
4
+
5
+ it "turns an array with one record into a yaml chunk" do
6
+ YamlDb::Utils.chunk_records([ %w(a b) ]).should == <<EOYAML
7
+ - - a
8
+ - b
9
+ EOYAML
10
+ end
11
+
12
+ it "turns an array with two records into a yaml chunk" do
13
+ YamlDb::Utils.chunk_records([ %w(a b), %w(x y) ]).should == <<EOYAML
14
+ - - a
15
+ - b
16
+ - - x
17
+ - y
18
+ EOYAML
19
+ end
20
+
21
+ end
metadata ADDED
@@ -0,0 +1,76 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: ironmin_yaml_db
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.2
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - uudui
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2013-07-15 00:00:00.000000000 Z
13
+ dependencies: []
14
+ description: ! '
15
+
16
+ IronmineYamlDb is a database-independent format for dumping and restoring data. It
17
+ complements the the database-independent schema format found in db/schema.rb. The
18
+ data is saved into db/data.yml.
19
+
20
+ This can be used as a replacement for mysqldump or pg_dump, but only for the databases
21
+ typically used by Rails apps. Users, permissions, schemas, triggers, and other
22
+ advanced database features are not supported - by design.
23
+
24
+ Any database that has an ActiveRecord adapter should work
25
+
26
+ '
27
+ email: 442453867@qq.com
28
+ executables: []
29
+ extensions: []
30
+ extra_rdoc_files:
31
+ - README.markdown
32
+ files:
33
+ - .travis.yml
34
+ - README.markdown
35
+ - Rakefile
36
+ - VERSION
37
+ - about.yml
38
+ - init.rb
39
+ - lib/csv_db.rb
40
+ - lib/serialization_helper.rb
41
+ - lib/tasks/yaml_db_tasks.rake
42
+ - lib/yaml_db.rb
43
+ - spec/base.rb
44
+ - spec/serialization_helper_base_spec.rb
45
+ - spec/serialization_helper_dump_spec.rb
46
+ - spec/serialization_helper_load_spec.rb
47
+ - spec/serialization_utils_spec.rb
48
+ - spec/yaml_dump_spec.rb
49
+ - spec/yaml_load_spec.rb
50
+ - spec/yaml_utils_spec.rb
51
+ - ironmine_yaml_db.gemspec
52
+ homepage: http://github.com/uudui/ironmine_yaml_db
53
+ licenses: []
54
+ post_install_message:
55
+ rdoc_options: []
56
+ require_paths:
57
+ - lib
58
+ required_ruby_version: !ruby/object:Gem::Requirement
59
+ none: false
60
+ requirements:
61
+ - - ! '>='
62
+ - !ruby/object:Gem::Version
63
+ version: '0'
64
+ required_rubygems_version: !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - ! '>='
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ requirements: []
71
+ rubyforge_project:
72
+ rubygems_version: 1.8.24
73
+ signing_key:
74
+ specification_version: 3
75
+ summary: ironmine_yaml_db allows export/import of database into/from yaml files
76
+ test_files: []