yaml_db_with_schema_tables 0.2.4

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: eeec6633e75b68276003927b191f5e4217c2e2b0
4
+ data.tar.gz: 577d9281469b47494ebc918566a75a1809a334dc
5
+ SHA512:
6
+ metadata.gz: 82c0a564358d3e814a966b5608d08e8b23541df4cc5015254517d43be96c96a0edbbc95a88393b0077649d0ba86f803b7acabab2ce64a332a41c9be398815ebd
7
+ data.tar.gz: 6e396e2a4884336894d9c894e78c0cf466e2023008a416b19ec86a8af70a9f5fa4b5eba0e10ec87c502c169457008765e129284db8c80c2408442547870205f3
data/.travis.yml ADDED
@@ -0,0 +1,5 @@
1
+ language: ruby
2
+ rvm:
3
+ - 1.8.7
4
+ - 1.9.2
5
+ - 1.9.3
data/README.markdown ADDED
@@ -0,0 +1,53 @@
1
+ # YamlDb
2
+
3
+
4
+ YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
5
+
6
+ This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
7
+
8
+ Any database that has an ActiveRecord adapter should work. This gem is now Rails 3 only. For Rails 2, clone and checkout the Rails2 branch.
9
+
10
+ This is a fork of the ludicast version which makes sure schema tables are being dumped, too.
11
+
12
+ [![Build Status](https://secure.travis-ci.org/ludicast/yaml_db.png)](http://travis-ci.org/ludicast/yaml_db)
13
+
14
+ ## Installation
15
+
16
+ Simply add to your Gemfile:
17
+
18
+ gem 'yaml_db_with_schema_tables'
19
+
20
+ All rake tasks will then be available to you.
21
+
22
+ ## Usage
23
+
24
+ rake db:data:dump -> Dump contents of Rails database to db/data.yml
25
+ rake db:data:load -> Load contents of db/data.yml into the database
26
+
27
+ Further, there are tasks db:dump and db:load which do the entire database (the equivalent of running db:schema:dump followed by db:data:load). Also, there are other tasks recently added that allow the export of the database contents to/from multiple files (each one named after the table being dumped or loaded).
28
+
29
+ rake db:data:dump_dir -> Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)
30
+ rake db:data:load_dir -> Load contents of db/data_dir into database
31
+
32
+ In addition, we have plugins whereby you can export your database to/from various formats. We only deal with yaml and csv right now, but you can easily write tools for your own formats (such as Excel or XML). To use another format, just load setting the "class" parameter to the class you are using. This defaults to "YamlDb::Helper" which is a refactoring of the old yaml_db code. We'll shorten this to use class nicknames in a little bit.
33
+
34
+ ## Examples
35
+
36
+ One common use would be to switch your data from one database backend to another. For example, let's say you wanted to switch from SQLite to MySQL. You might execute the following steps:
37
+
38
+ 1. rake db:dump
39
+
40
+ 2. Edit config/database.yml and change your adapter to mysql, set up database params
41
+
42
+ 3. mysqladmin create [database name]
43
+
44
+ 4. rake db:load
45
+
46
+ ## Credits
47
+
48
+ Created by Orion Henry and Adam Wiggins. Major updates by Ricardo Chimal, Jr.
49
+
50
+ Patches contributed by Michael Irwin, Tom Locke, and Tim Galeckas.
51
+
52
+ Send questions, feedback, or patches to the Heroku mailing list: http://groups.google.com/group/heroku
53
+
data/Rakefile ADDED
@@ -0,0 +1,30 @@
1
+ require 'rake'
2
+ require "rspec/core/rake_task"
3
+
4
+ begin
5
+ require 'jeweler'
6
+ Jeweler::Tasks.new do |gem|
7
+ gem.name = "yaml_db"
8
+ gem.summary = %Q{yaml_db allows export/import of database into/from yaml files}
9
+ gem.description = %Q{
10
+ YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
11
+ This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
12
+ Any database that has an ActiveRecord adapter should work
13
+ }
14
+ gem.email = "nate@ludicast.com"
15
+ gem.homepage = "http://github.com/ludicast/yaml_db"
16
+ gem.authors = ["Adam Wiggins","Orion Henry"]
17
+ end
18
+ Jeweler::GemcutterTasks.new
19
+ rescue LoadError
20
+ puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
21
+ end
22
+
23
+
24
+ RSpec::Core::RakeTask.new(:spec) do |spec|
25
+ spec.pattern = 'spec/*_spec.rb'
26
+ spec.rspec_opts = ['--backtrace']
27
+ end
28
+
29
+ task :default => :spec
30
+
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.2.4
data/about.yml ADDED
@@ -0,0 +1,5 @@
1
+ author: Orion Henry and Adam Wiggins of Heroku, Martin Honermeyer
2
+ summary: Dumps and loads a database-independent data dump format in db/data.yml.
3
+ homepage: http://opensource.heroku.com/
4
+ license: MIT
5
+ rails_version: 3.0+
data/init.rb ADDED
@@ -0,0 +1 @@
1
+ require 'yaml_db'
data/lib/csv_db.rb ADDED
@@ -0,0 +1,78 @@
1
+ #require 'FasterCSV'
2
+ module CsvDb
3
+ module Helper
4
+ def self.loader
5
+ Load
6
+ end
7
+
8
+ def self.dumper
9
+ Dump
10
+ end
11
+
12
+ def self.extension
13
+ "csv"
14
+ end
15
+ end
16
+
17
+ class Load < SerializationHelper::Load
18
+ def self.load_documents(io, truncate = true)
19
+ tables = {}
20
+ curr_table = nil
21
+ io.each do |line|
22
+ if /BEGIN_CSV_TABLE_DECLARATION(.+)END_CSV_TABLE_DECLARATION/ =~ line
23
+ curr_table = $1
24
+ tables[curr_table] = {}
25
+ else
26
+ if tables[curr_table]["columns"]
27
+ tables[curr_table]["records"] << FasterCSV.parse(line)[0]
28
+ else
29
+ tables[curr_table]["columns"] = FasterCSV.parse(line)[0]
30
+ tables[curr_table]["records"] = []
31
+ end
32
+ end
33
+ end
34
+
35
+ tables.each_pair do |table_name, contents|
36
+ load_table(table_name, contents, truncate)
37
+ end
38
+ end
39
+ end
40
+
41
+ class Dump < SerializationHelper::Dump
42
+
43
+ def self.before_table(io,table)
44
+ io.write "BEGIN_CSV_TABLE_DECLARATION#{table}END_CSV_TABLE_DECLARATION\n"
45
+ end
46
+
47
+ def self.dump(io)
48
+ tables.each do |table|
49
+ before_table(io, table)
50
+ dump_table(io, table)
51
+ after_table(io, table)
52
+ end
53
+ end
54
+
55
+ def self.after_table(io,table)
56
+ io.write ""
57
+ end
58
+
59
+ def self.dump_table_columns(io, table)
60
+ io.write(table_column_names(table).to_csv)
61
+ end
62
+
63
+ def self.dump_table_records(io, table)
64
+
65
+ column_names = table_column_names(table)
66
+
67
+ each_table_page(table) do |records|
68
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
69
+ records.each do |record|
70
+ io.write(record.to_csv)
71
+ end
72
+ end
73
+ end
74
+
75
+ end
76
+
77
+
78
+ end
@@ -0,0 +1,197 @@
1
+ module SerializationHelper
2
+
3
+ class Base
4
+ attr_reader :extension
5
+
6
+ def initialize(helper)
7
+ @dumper = helper.dumper
8
+ @loader = helper.loader
9
+ @extension = helper.extension
10
+ end
11
+
12
+ def dump(filename)
13
+ disable_logger
14
+ @dumper.dump(File.new(filename, "w"))
15
+ reenable_logger
16
+ end
17
+
18
+ def dump_to_dir(dirname)
19
+ Dir.mkdir(dirname)
20
+ tables = @dumper.tables
21
+ tables.each do |table|
22
+ io = File.new "#{dirname}/#{table}.#{@extension}", "w"
23
+ @dumper.before_table(io, table)
24
+ @dumper.dump_table io, table
25
+ @dumper.after_table(io, table)
26
+ end
27
+ end
28
+
29
+ def load(filename, truncate = true)
30
+ disable_logger
31
+ @loader.load(File.new(filename, "r"), truncate)
32
+ reenable_logger
33
+ end
34
+
35
+ def load_from_dir(dirname, truncate = true)
36
+ Dir.entries(dirname).each do |filename|
37
+ if filename =~ /^[.]/
38
+ next
39
+ end
40
+ @loader.load(File.new("#{dirname}/#{filename}", "r"), truncate)
41
+ end
42
+ end
43
+
44
+ def disable_logger
45
+ @@old_logger = ActiveRecord::Base.logger
46
+ ActiveRecord::Base.logger = nil
47
+ end
48
+
49
+ def reenable_logger
50
+ ActiveRecord::Base.logger = @@old_logger
51
+ end
52
+ end
53
+
54
+ class Load
55
+ def self.load(io, truncate = true)
56
+ ActiveRecord::Base.connection.transaction do
57
+ load_documents(io, truncate)
58
+ end
59
+ end
60
+
61
+ def self.truncate_table(table)
62
+ begin
63
+ ActiveRecord::Base.connection.execute("TRUNCATE #{SerializationHelper::Utils.quote_table(table)}")
64
+ rescue Exception
65
+ ActiveRecord::Base.connection.execute("DELETE FROM #{SerializationHelper::Utils.quote_table(table)}")
66
+ end
67
+ end
68
+
69
+ def self.load_table(table, data, truncate = true)
70
+ column_names = data['columns']
71
+ if truncate
72
+ truncate_table(table)
73
+ end
74
+ load_records(table, column_names, data['records'])
75
+ reset_pk_sequence!(table)
76
+ end
77
+
78
+ def self.load_records(table, column_names, records)
79
+ if column_names.nil?
80
+ return
81
+ end
82
+ columns = column_names.map{|cn| ActiveRecord::Base.connection.columns(table).detect{|c| c.name == cn}}
83
+ quoted_column_names = column_names.map { |column| ActiveRecord::Base.connection.quote_column_name(column) }.join(',')
84
+ quoted_table_name = SerializationHelper::Utils.quote_table(table)
85
+ records.each do |record|
86
+ quoted_values = record.zip(columns).map{|c| ActiveRecord::Base.connection.quote(c.first, c.last)}.join(',')
87
+ ActiveRecord::Base.connection.execute("INSERT INTO #{quoted_table_name} (#{quoted_column_names}) VALUES (#{quoted_values})")
88
+ end
89
+ end
90
+
91
+ def self.reset_pk_sequence!(table_name)
92
+ if ActiveRecord::Base.connection.respond_to?(:reset_pk_sequence!)
93
+ ActiveRecord::Base.connection.reset_pk_sequence!(table_name)
94
+ end
95
+ end
96
+
97
+
98
+ end
99
+
100
+ module Utils
101
+
102
+ def self.unhash(hash, keys)
103
+ keys.map { |key| hash[key] }
104
+ end
105
+
106
+ def self.unhash_records(records, keys)
107
+ records.each_with_index do |record, index|
108
+ records[index] = unhash(record, keys)
109
+ end
110
+
111
+ records
112
+ end
113
+
114
+ def self.convert_booleans(records, columns)
115
+ records.each do |record|
116
+ columns.each do |column|
117
+ next if is_boolean(record[column])
118
+ record[column] = convert_boolean(record[column])
119
+ end
120
+ end
121
+ records
122
+ end
123
+
124
+ def self.convert_boolean(value)
125
+ ['t', '1', true, 1].include?(value)
126
+ end
127
+
128
+ def self.boolean_columns(table)
129
+ columns = ActiveRecord::Base.connection.columns(table).reject { |c| silence_warnings { c.type != :boolean } }
130
+ columns.map { |c| c.name }
131
+ end
132
+
133
+ def self.is_boolean(value)
134
+ value.kind_of?(TrueClass) or value.kind_of?(FalseClass)
135
+ end
136
+
137
+ def self.quote_table(table)
138
+ ActiveRecord::Base.connection.quote_table_name(table)
139
+ end
140
+
141
+ end
142
+
143
+ class Dump
144
+ def self.before_table(io, table)
145
+
146
+ end
147
+
148
+ def self.dump(io)
149
+ tables.each do |table|
150
+ before_table(io, table)
151
+ dump_table(io, table)
152
+ after_table(io, table)
153
+ end
154
+ end
155
+
156
+ def self.after_table(io, table)
157
+
158
+ end
159
+
160
+ def self.tables
161
+ ActiveRecord::Base.connection.tables
162
+ end
163
+
164
+ def self.dump_table(io, table)
165
+ return if table_record_count(table).zero?
166
+
167
+ dump_table_columns(io, table)
168
+ dump_table_records(io, table)
169
+ end
170
+
171
+ def self.table_column_names(table)
172
+ ActiveRecord::Base.connection.columns(table).map { |c| c.name }
173
+ end
174
+
175
+
176
+ def self.each_table_page(table, records_per_page=1000)
177
+ total_count = table_record_count(table)
178
+ pages = (total_count.to_f / records_per_page).ceil - 1
179
+ id = table_column_names(table).first
180
+ boolean_columns = SerializationHelper::Utils.boolean_columns(table)
181
+ quoted_table_name = SerializationHelper::Utils.quote_table(table)
182
+
183
+ (0..pages).to_a.each do |page|
184
+ query = Arel::Table.new(table).order(id).skip(records_per_page*page).take(records_per_page).project(Arel.sql('*'))
185
+ records = ActiveRecord::Base.connection.select_all(query)
186
+ records = SerializationHelper::Utils.convert_booleans(records, boolean_columns)
187
+ yield records
188
+ end
189
+ end
190
+
191
+ def self.table_record_count(table)
192
+ ActiveRecord::Base.connection.select_one("SELECT COUNT(*) FROM #{SerializationHelper::Utils.quote_table(table)}").values.first.to_i
193
+ end
194
+
195
+ end
196
+
197
+ end
@@ -0,0 +1,45 @@
1
+ namespace :db do
2
+ desc "Dump schema and data to db/schema.rb and db/data.yml"
3
+ task(:dump => [ "db:schema:dump", "db:data:dump" ])
4
+
5
+ desc "Load schema and data from db/schema.rb and db/data.yml"
6
+ task(:load => [ "db:schema:load", "db:data:load" ])
7
+
8
+ namespace :data do
9
+ def db_dump_data_file (extension = "yml")
10
+ "#{dump_dir}/data.#{extension}"
11
+ end
12
+
13
+ def dump_dir(dir = "")
14
+ "#{Rails.root}/db#{dir}"
15
+ end
16
+
17
+ desc "Dump contents of database to db/data.extension (defaults to yaml)"
18
+ task :dump => :environment do
19
+ format_class = ENV['class'] || "YamlDb::Helper"
20
+ helper = format_class.constantize
21
+ SerializationHelper::Base.new(helper).dump db_dump_data_file helper.extension
22
+ end
23
+
24
+ desc "Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)"
25
+ task :dump_dir => :environment do
26
+ format_class = ENV['class'] || "YamlDb::Helper"
27
+ dir = ENV['dir'] || "#{Time.now.to_s.gsub(/ /, '_')}"
28
+ SerializationHelper::Base.new(format_class.constantize).dump_to_dir dump_dir("/#{dir}")
29
+ end
30
+
31
+ desc "Load contents of db/data.extension (defaults to yaml) into database"
32
+ task :load => :environment do
33
+ format_class = ENV['class'] || "YamlDb::Helper"
34
+ helper = format_class.constantize
35
+ SerializationHelper::Base.new(helper).load(db_dump_data_file helper.extension)
36
+ end
37
+
38
+ desc "Load contents of db/data_dir into database"
39
+ task :load_dir => :environment do
40
+ dir = ENV['dir'] || "base"
41
+ format_class = ENV['class'] || "YamlDb::Helper"
42
+ SerializationHelper::Base.new(format_class.constantize).load_from_dir dump_dir("/#{dir}")
43
+ end
44
+ end
45
+ end
data/lib/yaml_db.rb ADDED
@@ -0,0 +1,76 @@
1
+ require 'rubygems'
2
+ require 'yaml'
3
+ require 'active_record'
4
+ require 'serialization_helper'
5
+ require 'active_support/core_ext/kernel/reporting'
6
+ require 'rails/railtie'
7
+
8
+ module YamlDb
9
+ module Helper
10
+ def self.loader
11
+ YamlDb::Load
12
+ end
13
+
14
+ def self.dumper
15
+ YamlDb::Dump
16
+ end
17
+
18
+ def self.extension
19
+ "yml"
20
+ end
21
+ end
22
+
23
+
24
+ module Utils
25
+ def self.chunk_records(records)
26
+ yaml = [ records ].to_yaml
27
+ yaml.sub!(/---\s\n|---\n/, '')
28
+ yaml.sub!('- - -', ' - -')
29
+ yaml
30
+ end
31
+
32
+ end
33
+
34
+ class Dump < SerializationHelper::Dump
35
+
36
+ def self.dump_table_columns(io, table)
37
+ io.write("\n")
38
+ io.write({ table => { 'columns' => table_column_names(table) } }.to_yaml)
39
+ end
40
+
41
+ def self.dump_table_records(io, table)
42
+ table_record_header(io)
43
+
44
+ column_names = table_column_names(table)
45
+
46
+ each_table_page(table) do |records|
47
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
48
+ io.write(YamlDb::Utils.chunk_records(records))
49
+ end
50
+ end
51
+
52
+ def self.table_record_header(io)
53
+ io.write(" records: \n")
54
+ end
55
+
56
+ end
57
+
58
+ class Load < SerializationHelper::Load
59
+ def self.load_documents(io, truncate = true)
60
+ YAML.load_documents(io) do |ydoc|
61
+ ydoc.keys.each do |table_name|
62
+ next if ydoc[table_name].nil?
63
+ load_table(table_name, ydoc[table_name], truncate)
64
+ end
65
+ end
66
+ end
67
+ end
68
+
69
+ class Railtie < Rails::Railtie
70
+ rake_tasks do
71
+ load File.expand_path('../tasks/yaml_db_tasks.rake',
72
+ __FILE__)
73
+ end
74
+ end
75
+
76
+ end
@@ -0,0 +1 @@
1
+ require 'yaml_db'
data/spec/base.rb ADDED
@@ -0,0 +1,4 @@
1
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
2
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
3
+ require 'rspec'
4
+ require 'yaml_db'
@@ -0,0 +1,56 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Base do
4
+ def prestub_active_record
5
+
6
+ end
7
+
8
+ before do
9
+ @io = StringIO.new
10
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
11
+ ActiveRecord::Base.stub(:connection).and_return(mock('connection'))
12
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
13
+ end
14
+
15
+ def stub_helper!
16
+ @helper = mock("MyHelper")
17
+ @dumper = mock("MyDumper");
18
+ @loader = mock("MyLoader");
19
+ @helper.stub!(:dumper).and_return(@dumper)
20
+ @helper.stub!(:loader).and_return(@loader)
21
+ @helper.stub!(:extension).and_return("yml")
22
+ @dumper.stub!(:tables).and_return([ActiveRecord::Base.connection.tables[0]])
23
+ @dumper.stub!(:before_table).and_return(nil)
24
+ @dumper.stub!(:after_table).and_return(nil)
25
+ end
26
+
27
+ context "for multi-file dumps" do
28
+ before do
29
+ File.should_receive(:new).once.with("dir_name/mytable.yml", "w").and_return(@io)
30
+ Dir.should_receive(:mkdir).once.with("dir_name")
31
+ stub_helper!
32
+ @dumper.should_receive(:dump_table).once.with(@io, "mytable")
33
+ end
34
+
35
+ it "should create the number of files that there are tables" do
36
+ SerializationHelper::Base.new(@helper).dump_to_dir "dir_name"
37
+ end
38
+
39
+ end
40
+
41
+ context "for multi-file loads" do
42
+
43
+ before do
44
+ stub_helper!
45
+ @loader.should_receive(:load).once.with(@io, true)
46
+ File.should_receive(:new).once.with("dir_name/mytable.yml", "r").and_return(@io)
47
+ Dir.stub!(:entries).and_return(["mytable.yml"])
48
+ end
49
+
50
+ it "should insert into then umber of tables that there are files" do
51
+ SerializationHelper::Base.new(@helper).load_from_dir "dir_name"
52
+ end
53
+
54
+ end
55
+
56
+ end
@@ -0,0 +1,65 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Dump do
4
+
5
+ before do
6
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a', :name => 'a', :type => :string), mock('b', :name => 'b', :type => :string) ])
10
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
11
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
12
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
13
+ end
14
+
15
+ before(:each) do
16
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
17
+ @io = StringIO.new
18
+ end
19
+
20
+ it "should return a list of column names" do
21
+ SerializationHelper::Dump.table_column_names('mytable').should == [ 'a', 'b' ]
22
+ end
23
+
24
+ it "should return a list of tables including the rails schema table" do
25
+ SerializationHelper::Dump.tables.should == ['mytable', 'schema_info', 'schema_migrations']
26
+ end
27
+
28
+ it "should return the total number of records in a table" do
29
+ SerializationHelper::Dump.table_record_count('mytable').should == 2
30
+ end
31
+
32
+ it "should return all records from the database and return them when there is only 1 page" do
33
+ SerializationHelper::Dump.each_table_page('mytable') do |records|
34
+ records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
35
+ end
36
+ end
37
+
38
+ it "should paginate records from the database and return them" do
39
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 } ], [ { 'a' => 3, 'b' => 4 } ])
40
+
41
+ records = [ ]
42
+ SerializationHelper::Dump.each_table_page('mytable', 1) do |page|
43
+ page.size.should == 1
44
+ records.concat(page)
45
+ end
46
+
47
+ records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
48
+ end
49
+
50
+ it "should dump a table's contents to yaml" do
51
+ SerializationHelper::Dump.should_receive(:dump_table_columns)
52
+ SerializationHelper::Dump.should_receive(:dump_table_records)
53
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
54
+ end
55
+
56
+ it "should not dump a table's contents when the record count is zero" do
57
+ SerializationHelper::Dump.stub!(:table_record_count).with('mytable').and_return(0)
58
+ SerializationHelper::Dump.should_not_receive(:dump_table_columns)
59
+ SerializationHelper::Dump.should_not_receive(:dump_table_records)
60
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
61
+ end
62
+
63
+
64
+
65
+ end
@@ -0,0 +1,77 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Load do
4
+ before do
5
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
6
+
7
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
8
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
9
+ ActiveRecord::Base.connection.stub!(:transaction).and_yield
10
+ @io = StringIO.new
11
+ end
12
+
13
+ it "should truncate the table" do
14
+ ActiveRecord::Base.connection.stub!(:execute).with("TRUNCATE mytable").and_return(true)
15
+ ActiveRecord::Base.connection.should_not_receive(:execute).with("DELETE FROM mytable")
16
+ SerializationHelper::Load.truncate_table('mytable')
17
+ end
18
+
19
+ it "should delete the table if truncate throws an exception" do
20
+ ActiveRecord::Base.connection.should_receive(:execute).with("TRUNCATE mytable").and_raise()
21
+ ActiveRecord::Base.connection.should_receive(:execute).with("DELETE FROM mytable").and_return(true)
22
+ SerializationHelper::Load.truncate_table('mytable')
23
+ end
24
+
25
+
26
+ it "should call reset pk sequence if the connection adapter is postgres" do
27
+ ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(true)
28
+ ActiveRecord::Base.connection.should_receive(:reset_pk_sequence!).with('mytable')
29
+ SerializationHelper::Load.reset_pk_sequence!('mytable')
30
+ end
31
+
32
+ it "should not call reset pk sequence for other adapters" do
33
+ ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(false)
34
+ ActiveRecord::Base.connection.should_not_receive(:reset_pk_sequence!)
35
+ SerializationHelper::Load.reset_pk_sequence!('mytable')
36
+ end
37
+
38
+ it "should insert records into a table" do
39
+ mca = mock('a',:name => 'a')
40
+ mcb = mock('b', :name => 'b')
41
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([mca , mcb ])
42
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
43
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('b').and_return('b')
44
+ ActiveRecord::Base.connection.stub!(:quote).with(1, mca).and_return("'1'")
45
+ ActiveRecord::Base.connection.stub!(:quote).with(2, mcb).and_return("'2'")
46
+ ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
47
+ ActiveRecord::Base.connection.stub!(:quote).with(4, mcb).and_return("'4'")
48
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('1','2')")
49
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('3','4')")
50
+
51
+ SerializationHelper::Load.load_records('mytable', ['a', 'b'], [[1, 2], [3, 4]])
52
+ end
53
+
54
+ it "should quote column names that correspond to sql keywords" do
55
+ mca = mock('a',:name => 'a')
56
+ mccount = mock('count', :name => 'count')
57
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([mca , mccount ])
58
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
59
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('count').and_return('"count"')
60
+ ActiveRecord::Base.connection.stub!(:quote).with(1, mca).and_return("'1'")
61
+ ActiveRecord::Base.connection.stub!(:quote).with(2, mccount).and_return("'2'")
62
+ ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
63
+ ActiveRecord::Base.connection.stub!(:quote).with(4, mccount).and_return("'4'")
64
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('1','2')")
65
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('3','4')")
66
+
67
+ SerializationHelper::Load.load_records('mytable', ['a', 'count'], [[1, 2], [3, 4]])
68
+ end
69
+
70
+ it "should truncate the table and then load the records into the table" do
71
+ SerializationHelper::Load.should_receive(:truncate_table).with('mytable')
72
+ SerializationHelper::Load.should_receive(:load_records).with('mytable', ['a', 'b'], [[1, 2], [3, 4]])
73
+ SerializationHelper::Load.should_receive(:reset_pk_sequence!).with('mytable')
74
+
75
+ SerializationHelper::Load.load_table('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] })
76
+ end
77
+ end
@@ -0,0 +1,51 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe SerializationHelper::Utils, " convert records utility method" do
4
+ before do
5
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
6
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
7
+ end
8
+
9
+ it "returns an array of hash values using an array of ordered keys" do
10
+ SerializationHelper::Utils.unhash({ 'a' => 1, 'b' => 2 }, [ 'b', 'a' ]).should == [ 2, 1 ]
11
+ end
12
+
13
+ it "should unhash each hash an array using an array of ordered keys" do
14
+ SerializationHelper::Utils.unhash_records([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ], [ 'b', 'a' ]).should == [ [ 2, 1 ], [ 4, 3 ] ]
15
+ end
16
+
17
+ it "should return true if it is a boolean type" do
18
+ SerializationHelper::Utils.is_boolean(true).should == true
19
+ SerializationHelper::Utils.is_boolean('true').should_not == true
20
+ end
21
+
22
+ it "should return an array of boolean columns" do
23
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a',:type => :string), mock('b', :name => 'b',:type => :boolean) ])
24
+ SerializationHelper::Utils.boolean_columns('mytable').should == ['b']
25
+ end
26
+
27
+ it "should quote the table name" do
28
+ ActiveRecord::Base.connection.should_receive(:quote_table_name).with('values').and_return('`values`')
29
+ SerializationHelper::Utils.quote_table('values').should == '`values`'
30
+ end
31
+
32
+ it "should convert ruby booleans to true and false" do
33
+ SerializationHelper::Utils.convert_boolean(true).should == true
34
+ SerializationHelper::Utils.convert_boolean(false).should == false
35
+ end
36
+
37
+ it "should convert ruby string t and f to true and false" do
38
+ SerializationHelper::Utils.convert_boolean('t').should == true
39
+ SerializationHelper::Utils.convert_boolean('f').should == false
40
+ end
41
+
42
+ it "should convert ruby string 1 and 0 to true and false" do
43
+ SerializationHelper::Utils.convert_boolean('1').should == true
44
+ SerializationHelper::Utils.convert_boolean('0').should == false
45
+ end
46
+
47
+ it "should convert ruby integer 1 and 0 to true and false" do
48
+ SerializationHelper::Utils.convert_boolean(1).should == true
49
+ SerializationHelper::Utils.convert_boolean(0).should == false
50
+ end
51
+ end
@@ -0,0 +1,60 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe YamlDb::Dump do
4
+
5
+ before do
6
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a', :type => :string), mock('b', :name => 'b', :type => :string) ])
10
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
11
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
12
+ YamlDb::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
13
+ end
14
+
15
+ before(:each) do
16
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
17
+ @io = StringIO.new
18
+ end
19
+
20
+ it "should return a formatted string" do
21
+ YamlDb::Dump.table_record_header(@io)
22
+ @io.rewind
23
+ @io.read.should == " records: \n"
24
+ end
25
+
26
+
27
+ it "should return a yaml string that contains a table header and column names" do
28
+ if RUBY_VERSION.to_f >= 1.9 && RUBY_VERSION.to_f < 2.0
29
+ YAML::ENGINE.yamler = "syck"
30
+ end
31
+ YamlDb::Dump.stub!(:table_column_names).with('mytable').and_return([ 'a', 'b' ])
32
+ YamlDb::Dump.dump_table_columns(@io, 'mytable')
33
+ @io.rewind
34
+ expected_yaml = <<EOYAML
35
+
36
+ ---
37
+ mytable:
38
+ columns:
39
+ - a
40
+ - b
41
+ EOYAML
42
+ expected_yaml.gsub!(" \n", "\n") if RUBY_VERSION.to_f >= 2.0
43
+ @io.read.should == expected_yaml
44
+ end
45
+
46
+ it "should return dump the records for a table in yaml to a given io stream" do
47
+ YamlDb::Dump.dump_table_records(@io, 'mytable')
48
+ @io.rewind
49
+ @io.read.should == <<EOYAML
50
+ records:
51
+ - - 1
52
+ - 2
53
+ - - 3
54
+ - 4
55
+ EOYAML
56
+ end
57
+
58
+
59
+
60
+ end
@@ -0,0 +1,33 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+ require 'active_support/core_ext/kernel/debugger'
3
+
4
+ describe YamlDb::Load do
5
+ before do
6
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
7
+
8
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
9
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
10
+ ActiveRecord::Base.connection.stub!(:transaction).and_yield
11
+ end
12
+
13
+ before(:each) do
14
+ @io = StringIO.new
15
+ end
16
+
17
+
18
+ it "should call load structure for each document in the file" do
19
+ YAML.should_receive(:load_documents).with(@io).and_yield({ 'mytable' => {
20
+ 'columns' => [ 'a', 'b' ],
21
+ 'records' => [[1, 2], [3, 4]]
22
+ } } )
23
+ YamlDb::Load.should_receive(:load_table).with('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] },true)
24
+ YamlDb::Load.load(@io)
25
+ end
26
+
27
+ it "should not call load structure when the document in the file contains no records" do
28
+ YAML.should_receive(:load_documents).with(@io).and_yield({ 'mytable' => nil })
29
+ YamlDb::Load.should_not_receive(:load_table)
30
+ YamlDb::Load.load(@io)
31
+ end
32
+
33
+ end
@@ -0,0 +1,21 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+
3
+ describe YamlDb::Utils, " convert records utility method" do
4
+
5
+ it "turns an array with one record into a yaml chunk" do
6
+ YamlDb::Utils.chunk_records([ %w(a b) ]).should == <<EOYAML
7
+ - - a
8
+ - b
9
+ EOYAML
10
+ end
11
+
12
+ it "turns an array with two records into a yaml chunk" do
13
+ YamlDb::Utils.chunk_records([ %w(a b), %w(x y) ]).should == <<EOYAML
14
+ - - a
15
+ - b
16
+ - - x
17
+ - y
18
+ EOYAML
19
+ end
20
+
21
+ end
data/yaml_db.gemspec ADDED
@@ -0,0 +1,54 @@
1
+ # Generated by jeweler
2
+ # DO NOT EDIT THIS FILE DIRECTLY
3
+ # Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec'
4
+ # -*- encoding: utf-8 -*-
5
+
6
+ Gem::Specification.new do |s|
7
+ s.name = "yaml_db_with_schema_tables"
8
+ s.version = "0.2.4"
9
+
10
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11
+ s.authors = ["Adam Wiggins", "Orion Henry", "Martin Honermeyer"]
12
+ s.date = "2014-01-20"
13
+ s.description = "\nYamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.\nThis can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.\nAny database that has an ActiveRecord adapter should work\n"
14
+ s.email = "martin.honermeyer@zweitag.de"
15
+ s.extra_rdoc_files = [
16
+ "README.markdown"
17
+ ]
18
+ s.files = [
19
+ ".travis.yml",
20
+ "README.markdown",
21
+ "Rakefile",
22
+ "VERSION",
23
+ "about.yml",
24
+ "init.rb",
25
+ "lib/csv_db.rb",
26
+ "lib/serialization_helper.rb",
27
+ "lib/tasks/yaml_db_tasks.rake",
28
+ "lib/yaml_db.rb",
29
+ "lib/yaml_db_with_schema_tables.rb",
30
+ "spec/base.rb",
31
+ "spec/serialization_helper_base_spec.rb",
32
+ "spec/serialization_helper_dump_spec.rb",
33
+ "spec/serialization_helper_load_spec.rb",
34
+ "spec/serialization_utils_spec.rb",
35
+ "spec/yaml_dump_spec.rb",
36
+ "spec/yaml_load_spec.rb",
37
+ "spec/yaml_utils_spec.rb",
38
+ "yaml_db.gemspec"
39
+ ]
40
+ s.homepage = "http://github.com/zweitag/yaml_db"
41
+ s.require_paths = ["lib"]
42
+ s.rubygems_version = "1.8.17"
43
+ s.summary = "yaml_db allows export/import of database into/from yaml files"
44
+
45
+ if s.respond_to? :specification_version then
46
+ s.specification_version = 3
47
+
48
+ if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
49
+ else
50
+ end
51
+ else
52
+ end
53
+ end
54
+
metadata ADDED
@@ -0,0 +1,70 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: yaml_db_with_schema_tables
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.2.4
5
+ platform: ruby
6
+ authors:
7
+ - Adam Wiggins
8
+ - Orion Henry
9
+ - Martin Honermeyer
10
+ autorequire:
11
+ bindir: bin
12
+ cert_chain: []
13
+ date: 2014-01-20 00:00:00.000000000 Z
14
+ dependencies: []
15
+ description: |2
16
+
17
+ YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
18
+ This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
19
+ Any database that has an ActiveRecord adapter should work
20
+ email: martin.honermeyer@zweitag.de
21
+ executables: []
22
+ extensions: []
23
+ extra_rdoc_files:
24
+ - README.markdown
25
+ files:
26
+ - .travis.yml
27
+ - README.markdown
28
+ - Rakefile
29
+ - VERSION
30
+ - about.yml
31
+ - init.rb
32
+ - lib/csv_db.rb
33
+ - lib/serialization_helper.rb
34
+ - lib/tasks/yaml_db_tasks.rake
35
+ - lib/yaml_db.rb
36
+ - lib/yaml_db_with_schema_tables.rb
37
+ - spec/base.rb
38
+ - spec/serialization_helper_base_spec.rb
39
+ - spec/serialization_helper_dump_spec.rb
40
+ - spec/serialization_helper_load_spec.rb
41
+ - spec/serialization_utils_spec.rb
42
+ - spec/yaml_dump_spec.rb
43
+ - spec/yaml_load_spec.rb
44
+ - spec/yaml_utils_spec.rb
45
+ - yaml_db.gemspec
46
+ homepage: http://github.com/zweitag/yaml_db
47
+ licenses: []
48
+ metadata: {}
49
+ post_install_message:
50
+ rdoc_options: []
51
+ require_paths:
52
+ - lib
53
+ required_ruby_version: !ruby/object:Gem::Requirement
54
+ requirements:
55
+ - - '>='
56
+ - !ruby/object:Gem::Version
57
+ version: '0'
58
+ required_rubygems_version: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - '>='
61
+ - !ruby/object:Gem::Version
62
+ version: '0'
63
+ requirements: []
64
+ rubyforge_project:
65
+ rubygems_version: 2.1.11
66
+ signing_key:
67
+ specification_version: 3
68
+ summary: yaml_db allows export/import of database into/from yaml files
69
+ test_files: []
70
+ has_rdoc: