yaml_db_with_schema_tables 0.2.4 → 0.3.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: eeec6633e75b68276003927b191f5e4217c2e2b0
4
- data.tar.gz: 577d9281469b47494ebc918566a75a1809a334dc
3
+ metadata.gz: 56086fd52fca521d5a61ff835253a623ae9ee680
4
+ data.tar.gz: 577d01ac73bc69863cf41cf40051bf9d40d378be
5
5
  SHA512:
6
- metadata.gz: 82c0a564358d3e814a966b5608d08e8b23541df4cc5015254517d43be96c96a0edbbc95a88393b0077649d0ba86f803b7acabab2ce64a332a41c9be398815ebd
7
- data.tar.gz: 6e396e2a4884336894d9c894e78c0cf466e2023008a416b19ec86a8af70a9f5fa4b5eba0e10ec87c502c169457008765e129284db8c80c2408442547870205f3
6
+ metadata.gz: 80a7996018fb8202a8aa083e329482a202f6fb8e418db6ff7db957ee6bba9204cb7c79f5fb98ed2d4afcd3fa1c95743fc401648ff50702fc7c1657577ce01992
7
+ data.tar.gz: 7124cc5ade5ac68f92ee8e54bdc1e9005db4c0f81a9dd8087bdfedca36b77dceb9666db19f0694f2e7e98ec99996c3f8e23c53ddb13e1da41987d64b5e9bd4e3
data/README.markdown CHANGED
@@ -1,15 +1,14 @@
1
1
  # YamlDb
2
2
 
3
-
4
3
  YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
5
4
 
6
5
  This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
7
6
 
8
7
  Any database that has an ActiveRecord adapter should work. This gem is now Rails 3 only. For Rails 2, clone and checkout the Rails2 branch.
9
8
 
10
- This is a fork of the ludicast version which makes sure schema tables are being dumped, too.
9
+ _This is a fork of the ludicast version which makes sure schema tables are being dumped, too. Also, JSON is supported as an alternative serialization format._
11
10
 
12
- [![Build Status](https://secure.travis-ci.org/ludicast/yaml_db.png)](http://travis-ci.org/ludicast/yaml_db)
11
+ [![Build Status](https://secure.travis-ci.org/zweitag/yaml_db.png)](http://travis-ci.org/zweitag/yaml_db)
13
12
 
14
13
  ## Installation
15
14
 
@@ -29,7 +28,7 @@ Further, there are tasks db:dump and db:load which do the entire database (the e
29
28
  rake db:data:dump_dir -> Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)
30
29
  rake db:data:load_dir -> Load contents of db/data_dir into database
31
30
 
32
- In addition, we have plugins whereby you can export your database to/from various formats. We only deal with yaml and csv right now, but you can easily write tools for your own formats (such as Excel or XML). To use another format, just load setting the "class" parameter to the class you are using. This defaults to "YamlDb::Helper" which is a refactoring of the old yaml_db code. We'll shorten this to use class nicknames in a little bit.
31
+ In addition, we have plugins whereby you can export your database to/from various formats. We only deal with yaml and csv right now, but you can easily write tools for your own formats (such as Excel or XML). To use another format, just load setting the "class" parameter to the class you are using. This defaults to "YamlDb::Helper" which is a refactoring of the old yaml_db code. Set it to "JsonDb::Helper" in order to use JSON serialization (much faster!) instead.
33
32
 
34
33
  ## Examples
35
34
 
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.2.4
1
+ 0.3.2
data/lib/csv_db.rb CHANGED
@@ -1,4 +1,5 @@
1
- #require 'FasterCSV'
1
+ require 'csv'
2
+
2
3
  module CsvDb
3
4
  module Helper
4
5
  def self.loader
@@ -66,8 +67,8 @@ module CsvDb
66
67
 
67
68
  each_table_page(table) do |records|
68
69
  rows = SerializationHelper::Utils.unhash_records(records, column_names)
69
- records.each do |record|
70
- io.write(record.to_csv)
70
+ rows.each do |row|
71
+ io.write(row.to_csv)
71
72
  end
72
73
  end
73
74
  end
@@ -75,4 +76,4 @@ module CsvDb
75
76
  end
76
77
 
77
78
 
78
- end
79
+ end
data/lib/json_db.rb ADDED
@@ -0,0 +1,74 @@
1
+ require 'rubygems'
2
+ require 'json'
3
+ require 'json/add/core'
4
+ require 'active_record'
5
+ require 'serialization_helper'
6
+
7
+ module JsonDb
8
+ module Helper
9
+ def self.loader
10
+ Load
11
+ end
12
+
13
+ def self.dumper
14
+ Dump
15
+ end
16
+
17
+ def self.extension
18
+ "json"
19
+ end
20
+ end
21
+
22
+ class Dump < SerializationHelper::Dump
23
+ def self.dump(io)
24
+ io.write '{'
25
+ super io
26
+ io.write '}'
27
+ end
28
+
29
+ def self.before_table(io, table)
30
+ io.write '"' + table + '": {'
31
+ end
32
+
33
+ def self.after_table(io, table)
34
+ if table == tables.last
35
+ io.write "}\n"
36
+ else
37
+ io.write "},\n"
38
+ end
39
+ end
40
+
41
+ def self.dump_table_columns(io, table)
42
+ io.write '"columns": ' + JSON.dump(table_column_names(table)) + ', '
43
+ end
44
+
45
+ def self.table_record_header(io)
46
+ io.write('"records": [ ')
47
+ end
48
+
49
+ def self.dump_table_records(io, table)
50
+ table_record_header(io)
51
+
52
+ column_names = table_column_names(table)
53
+
54
+ each_table_page(table) do |records, page_types|
55
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
56
+ io.write JSON.dump(rows)[1..-2] # without opening and closing brackets
57
+ io.write ', ' unless :last.in? page_types
58
+ end
59
+
60
+ io.write ' ]'
61
+ end
62
+ end
63
+
64
+ class Load < SerializationHelper::Load
65
+ def self.load_documents(io, truncate = true)
66
+ JSON.load(io).tap do |json|
67
+ json.keys.each do |table_name|
68
+ next if json[table_name].nil?
69
+ load_table(table_name, json[table_name], truncate)
70
+ end
71
+ end
72
+ end
73
+ end
74
+ end
@@ -22,10 +22,16 @@ module SerializationHelper
22
22
  io = File.new "#{dirname}/#{table}.#{@extension}", "w"
23
23
  @dumper.before_table(io, table)
24
24
  @dumper.dump_table io, table
25
- @dumper.after_table(io, table)
25
+ @dumper.after_table(io, table)
26
26
  end
27
27
  end
28
28
 
29
+ def dump_to_io(io)
30
+ disable_logger
31
+ @dumper.dump(io)
32
+ reenable_logger
33
+ end
34
+
29
35
  def load(filename, truncate = true)
30
36
  disable_logger
31
37
  @loader.load(File.new(filename, "r"), truncate)
@@ -38,7 +44,13 @@ module SerializationHelper
38
44
  next
39
45
  end
40
46
  @loader.load(File.new("#{dirname}/#{filename}", "r"), truncate)
41
- end
47
+ end
48
+ end
49
+
50
+ def load_from_io(io, truncate = true)
51
+ disable_logger
52
+ @loader.load(io, truncate)
53
+ reenable_logger
42
54
  end
43
55
 
44
56
  def disable_logger
@@ -67,6 +79,7 @@ module SerializationHelper
67
79
  end
68
80
 
69
81
  def self.load_table(table, data, truncate = true)
82
+ return if table == 'ar_internal_metadata'
70
83
  column_names = data['columns']
71
84
  if truncate
72
85
  truncate_table(table)
@@ -75,16 +88,19 @@ module SerializationHelper
75
88
  reset_pk_sequence!(table)
76
89
  end
77
90
 
78
- def self.load_records(table, column_names, records)
91
+ def self.load_records(table, column_names, records, records_per_page=1000)
79
92
  if column_names.nil?
80
93
  return
81
94
  end
82
95
  columns = column_names.map{|cn| ActiveRecord::Base.connection.columns(table).detect{|c| c.name == cn}}
83
96
  quoted_column_names = column_names.map { |column| ActiveRecord::Base.connection.quote_column_name(column) }.join(',')
84
97
  quoted_table_name = SerializationHelper::Utils.quote_table(table)
85
- records.each do |record|
86
- quoted_values = record.zip(columns).map{|c| ActiveRecord::Base.connection.quote(c.first, c.last)}.join(',')
87
- ActiveRecord::Base.connection.execute("INSERT INTO #{quoted_table_name} (#{quoted_column_names}) VALUES (#{quoted_values})")
98
+
99
+ 0.step(records.count-1, records_per_page) do |offset|
100
+ all_quoted_values = records[offset, records_per_page].map do |record|
101
+ '(' + record.zip(columns).map{|c| ActiveRecord::Base.connection.quote(c.first, c.last)}.join(',') + ')'
102
+ end.join(', ')
103
+ ActiveRecord::Base.connection.execute("INSERT INTO #{quoted_table_name} (#{quoted_column_names}) VALUES #{all_quoted_values}")
88
104
  end
89
105
  end
90
106
 
@@ -99,41 +115,59 @@ module SerializationHelper
99
115
 
100
116
  module Utils
101
117
 
102
- def self.unhash(hash, keys)
103
- keys.map { |key| hash[key] }
104
- end
105
-
106
118
  def self.unhash_records(records, keys)
107
- records.each_with_index do |record, index|
108
- records[index] = unhash(record, keys)
119
+ records.map do |record|
120
+ keys.map { |key| record[key] }
109
121
  end
110
-
111
- records
112
122
  end
113
123
 
114
124
  def self.convert_booleans(records, columns)
115
- records.each do |record|
125
+ records.map do |record|
116
126
  columns.each do |column|
117
127
  next if is_boolean(record[column])
118
128
  record[column] = convert_boolean(record[column])
119
129
  end
130
+ record
131
+ end
132
+ end
133
+
134
+ def self.convert_jsons(records, columns)
135
+ records.map do |record|
136
+ columns.each do |column|
137
+ next if is_json(record[column])
138
+ record[column] = convert_json(record[column])
139
+ end
140
+ record
120
141
  end
121
- records
122
142
  end
123
143
 
124
144
  def self.convert_boolean(value)
125
145
  ['t', '1', true, 1].include?(value)
126
146
  end
127
147
 
148
+ def self.convert_json(value)
149
+ return nil if value.nil?
150
+ JSON.parse(value)
151
+ end
152
+
128
153
  def self.boolean_columns(table)
129
154
  columns = ActiveRecord::Base.connection.columns(table).reject { |c| silence_warnings { c.type != :boolean } }
130
155
  columns.map { |c| c.name }
131
156
  end
132
157
 
158
+ def self.json_columns(table)
159
+ columns = ActiveRecord::Base.connection.columns(table).select { |c| c.sql_type == 'json' }
160
+ columns.map { |c| c.name }
161
+ end
162
+
133
163
  def self.is_boolean(value)
134
164
  value.kind_of?(TrueClass) or value.kind_of?(FalseClass)
135
165
  end
136
166
 
167
+ def self.is_json(value)
168
+ value.kind_of?(Hash) or value.kind_of?(Array)
169
+ end
170
+
137
171
  def self.quote_table(table)
138
172
  ActiveRecord::Base.connection.quote_table_name(table)
139
173
  end
@@ -178,13 +212,20 @@ module SerializationHelper
178
212
  pages = (total_count.to_f / records_per_page).ceil - 1
179
213
  id = table_column_names(table).first
180
214
  boolean_columns = SerializationHelper::Utils.boolean_columns(table)
215
+ json_columns = SerializationHelper::Utils.json_columns(table)
181
216
  quoted_table_name = SerializationHelper::Utils.quote_table(table)
182
217
 
183
- (0..pages).to_a.each do |page|
218
+ (0..pages).to_a.each_with_index do |page, index|
184
219
  query = Arel::Table.new(table).order(id).skip(records_per_page*page).take(records_per_page).project(Arel.sql('*'))
185
220
  records = ActiveRecord::Base.connection.select_all(query)
186
221
  records = SerializationHelper::Utils.convert_booleans(records, boolean_columns)
187
- yield records
222
+ records = SerializationHelper::Utils.convert_jsons(records, json_columns)
223
+
224
+ page_types = []
225
+ page_types << :first if index == 0
226
+ page_types << :last if index == pages
227
+
228
+ yield records, page_types
188
229
  end
189
230
  end
190
231
 
@@ -1,3 +1,5 @@
1
+ require 'zlib'
2
+
1
3
  namespace :db do
2
4
  desc "Dump schema and data to db/schema.rb and db/data.yml"
3
5
  task(:dump => [ "db:schema:dump", "db:data:dump" ])
@@ -16,30 +18,45 @@ namespace :db do
16
18
 
17
19
  desc "Dump contents of database to db/data.extension (defaults to yaml)"
18
20
  task :dump => :environment do
19
- format_class = ENV['class'] || "YamlDb::Helper"
20
- helper = format_class.constantize
21
21
  SerializationHelper::Base.new(helper).dump db_dump_data_file helper.extension
22
22
  end
23
23
 
24
24
  desc "Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)"
25
25
  task :dump_dir => :environment do
26
- format_class = ENV['class'] || "YamlDb::Helper"
27
26
  dir = ENV['dir'] || "#{Time.now.to_s.gsub(/ /, '_')}"
28
- SerializationHelper::Base.new(format_class.constantize).dump_to_dir dump_dir("/#{dir}")
27
+ end
28
+
29
+ desc "Dump gzipped contents of database to stdout"
30
+ task :dump_gzipped_stdout => :environment do
31
+ gz = Zlib::GzipWriter.new($stdout.dup)
32
+ SerializationHelper::Base.new(helper).dump_to_io gz
33
+ gz.close
29
34
  end
30
35
 
31
36
  desc "Load contents of db/data.extension (defaults to yaml) into database"
32
37
  task :load => :environment do
33
- format_class = ENV['class'] || "YamlDb::Helper"
34
- helper = format_class.constantize
35
38
  SerializationHelper::Base.new(helper).load(db_dump_data_file helper.extension)
36
39
  end
37
40
 
38
41
  desc "Load contents of db/data_dir into database"
39
42
  task :load_dir => :environment do
40
43
  dir = ENV['dir'] || "base"
41
- format_class = ENV['class'] || "YamlDb::Helper"
42
- SerializationHelper::Base.new(format_class.constantize).load_from_dir dump_dir("/#{dir}")
44
+ SerializationHelper::Base.new(helper).load_from_dir dump_dir("/#{dir}")
45
+ end
46
+
47
+ desc "Load gzipped contents from stdin"
48
+ task :load_gzipped_stdin => :environment do
49
+ gz = Zlib::GzipReader.new($stdin.dup)
50
+ SerializationHelper::Base.new(helper).load_from_io gz
51
+ gz.close
52
+ end
53
+
54
+ def helper
55
+ @helper ||= begin
56
+ format_class = ENV['class'] || "YamlDb::Helper"
57
+ require format_class.split('::').first.underscore
58
+ format_class.constantize
59
+ end
43
60
  end
44
61
  end
45
62
  end
data/lib/yaml_db.rb CHANGED
@@ -45,7 +45,7 @@ module YamlDb
45
45
 
46
46
  each_table_page(table) do |records|
47
47
  rows = SerializationHelper::Utils.unhash_records(records, column_names)
48
- io.write(YamlDb::Utils.chunk_records(records))
48
+ io.write(YamlDb::Utils.chunk_records(rows))
49
49
  end
50
50
  end
51
51
 
data/spec/base.rb CHANGED
@@ -2,3 +2,5 @@ $LOAD_PATH.unshift(File.dirname(__FILE__))
2
2
  $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
3
3
  require 'rspec'
4
4
  require 'yaml_db'
5
+ require 'json_db'
6
+ require 'csv_db'
@@ -0,0 +1,81 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+ require 'date'
3
+
4
+ describe JsonDb::Dump do
5
+
6
+ before do
7
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
8
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
9
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
10
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a', name: 'a', type: :string, sql_type: 'text'), mock('b', name: 'b', type: :string, sql_type: 'text') ])
11
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
12
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
13
+ end
14
+
15
+ before(:each) do
16
+ File.stub!(:new).with('dump.json', 'w').and_return(StringIO.new)
17
+ @io = StringIO.new
18
+ end
19
+
20
+ it "should dump a valid json document with correct data" do
21
+ ActiveRecord::Base.connection.stub!(:columns).and_return([ mock('a', name: 'a', type: :string, sql_type: 'text'), mock('b', name: 'b', type: :string, sql_type: 'text') ])
22
+
23
+ JsonDb::Dump.dump(@io)
24
+ @io.rewind
25
+ expect { @json = JSON.load @io }.not_to raise_error
26
+
27
+ @json['mytable']['columns'].count.should == 2
28
+ @json['mytable']['records'].should match_array [[1, 2], [3, 4]]
29
+ end
30
+
31
+ it "should return a formatted string" do
32
+ JsonDb::Dump.table_record_header(@io)
33
+ @io.rewind
34
+ @io.read.should == '"records": [ '
35
+ end
36
+
37
+ it "should return a json string that contains column names" do
38
+ JsonDb::Dump.stub!(:table_column_names).with('mytable').and_return([ 'a', 'b' ])
39
+ JsonDb::Dump.dump_table_columns(@io, 'mytable')
40
+ @io.rewind
41
+ @io.read.should == '"columns": ["a","b"], '
42
+ end
43
+
44
+ it "should return dump the records for a table in json to a given io stream" do
45
+ JsonDb::Dump.dump_table_records(@io, 'mytable')
46
+ @io.rewind
47
+ @io.read.should == '"records": [ [1,2],[3,4] ]'
48
+ end
49
+
50
+ it 'should dump a valid json document for more than 1000 records' do
51
+ ActiveRecord::Base.connection.stub!(:tables).and_return(['mytable'])
52
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"1001"})
53
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([{'a'=>1, 'b'=>2}] * 1000, [{'a'=>1, 'b'=>2}])
54
+
55
+ JsonDb::Dump.dump(@io)
56
+ @io.rewind
57
+ expect { @json = JSON.load @io }.not_to raise_error
58
+
59
+ @json['mytable']['columns'].count.should == 2
60
+ @json['mytable']['records'].count.should == 1001
61
+ @json['mytable']['records'].should match_array([[1, 2]] * 1001)
62
+ end
63
+
64
+ it 'should dump datetime objects using custom Ruby serialization' do
65
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('datetime', name: 'datetime', type: :datetime, sql_type: 'datetime')])
66
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"1"})
67
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'datetime' => DateTime.new(2014, 1, 1, 12, 20, 00) } ])
68
+ JsonDb::Dump.dump_table_records(@io, 'mytable')
69
+ @io.rewind
70
+ @io.read.should == '"records": [ [{"json_class":"DateTime","y":2014,"m":1,"d":1,"H":12,"M":20,"S":0,"of":"0/1","sg":2299161.0}] ]'
71
+ end
72
+
73
+ it 'should correctly serialize json columns' do
74
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('json', name: 'json', type: :json, sql_type: 'json')])
75
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"1"})
76
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'json' => '[{"a":1},{"b":2}]' } ])
77
+ JsonDb::Dump.dump_table_records(@io, 'mytable')
78
+ @io.rewind
79
+ @io.read.should == '"records": [ [[{"a":1},{"b":2}]] ]'
80
+ end
81
+ end
@@ -0,0 +1,46 @@
1
+ require File.dirname(__FILE__) + '/base'
2
+ require 'active_support/core_ext/kernel/debugger'
3
+
4
+ describe JsonDb::Load do
5
+ before do
6
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
7
+
8
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
9
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
10
+ ActiveRecord::Base.connection.stub!(:transaction).and_yield
11
+ end
12
+
13
+ before(:each) do
14
+ @io = StringIO.new
15
+ end
16
+
17
+
18
+ it "should call load_table for each table in the file" do
19
+ JSON.should_receive(:load).with(@io).and_return({ 'mytable' => {
20
+ 'columns' => [ 'a', 'b' ],
21
+ 'records' => [[1, 2], [3, 4]]
22
+ } } )
23
+ JsonDb::Load.should_receive(:load_table).with('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] },true)
24
+ JsonDb::Load.load(@io)
25
+ end
26
+
27
+ it "should not call load_table when the table in the file contains no records" do
28
+ JSON.should_receive(:load).with(@io).and_return({ 'mytable' => nil })
29
+ JsonDb::Load.should_not_receive(:load_table)
30
+ JsonDb::Load.load(@io)
31
+ end
32
+
33
+ it "should load custom Ruby serialized datetime objects" do
34
+ @io = StringIO.new <<-EOYAML
35
+ {
36
+ "mytable": {
37
+ "columns": [ "datetime" ],
38
+ "records": [ [{"json_class":"DateTime","y":2014,"m":1,"d":1,"H":12,"M":20,"S":0,"of":"0/1","sg":2299161.0}] ]
39
+ }
40
+ }
41
+ EOYAML
42
+
43
+ JsonDb::Load.should_receive(:load_table).with('mytable', { 'columns' => [ 'datetime' ], 'records' => [ [DateTime.new(2014, 1, 1, 12, 20)] ] },true)
44
+ JsonDb::Load.load(@io)
45
+ end
46
+ end
@@ -2,64 +2,71 @@ require File.dirname(__FILE__) + '/base'
2
2
 
3
3
  describe SerializationHelper::Dump do
4
4
 
5
- before do
6
- silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
- ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
- ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
- ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a', :name => 'a', :type => :string), mock('b', :name => 'b', :type => :string) ])
10
- ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
11
- ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
12
- SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
13
- end
5
+ before do
6
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
+ ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([
10
+ mock('a', name: 'a', type: :string, sql_type: 'text'),
11
+ mock('b', name: 'b', type: :string, sql_type: 'text'),
12
+ mock('json', name: 'json', type: :json, sql_type: 'json')
13
+ ])
14
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
15
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([
16
+ { 'a' => 1, 'b' => 2, 'json' => '[{"c": 3}]' },
17
+ { 'a' => 3, 'b' => 4, 'json' => nil }
18
+ ])
19
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
20
+ end
14
21
 
15
- before(:each) do
16
- File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
17
- @io = StringIO.new
18
- end
22
+ before(:each) do
23
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
24
+ @io = StringIO.new
25
+ end
19
26
 
20
- it "should return a list of column names" do
21
- SerializationHelper::Dump.table_column_names('mytable').should == [ 'a', 'b' ]
22
- end
27
+ it "should return a list of column names" do
28
+ SerializationHelper::Dump.table_column_names('mytable').should == [ 'a', 'b', 'json' ]
29
+ end
23
30
 
24
- it "should return a list of tables including the rails schema table" do
25
- SerializationHelper::Dump.tables.should == ['mytable', 'schema_info', 'schema_migrations']
26
- end
31
+ it "should return a list of tables including the rails schema table" do
32
+ SerializationHelper::Dump.tables.should == ['mytable', 'schema_info', 'schema_migrations']
33
+ end
27
34
 
28
- it "should return the total number of records in a table" do
29
- SerializationHelper::Dump.table_record_count('mytable').should == 2
30
- end
35
+ it "should return the total number of records in a table" do
36
+ SerializationHelper::Dump.table_record_count('mytable').should == 2
37
+ end
31
38
 
32
- it "should return all records from the database and return them when there is only 1 page" do
33
- SerializationHelper::Dump.each_table_page('mytable') do |records|
34
- records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
35
- end
36
- end
37
-
38
- it "should paginate records from the database and return them" do
39
- ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 } ], [ { 'a' => 3, 'b' => 4 } ])
40
-
41
- records = [ ]
42
- SerializationHelper::Dump.each_table_page('mytable', 1) do |page|
43
- page.size.should == 1
44
- records.concat(page)
45
- end
46
-
47
- records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
48
- end
39
+ it "should return all records from the database and return them when there is only 1 page" do
40
+ SerializationHelper::Dump.each_table_page('mytable') do |records|
41
+ records.should == [ { 'a' => 1, 'b' => 2, 'json' => [{'c' => 3}] }, { 'a' => 3, 'b' => 4, 'json' => nil } ]
42
+ end
43
+ end
49
44
 
50
- it "should dump a table's contents to yaml" do
51
- SerializationHelper::Dump.should_receive(:dump_table_columns)
52
- SerializationHelper::Dump.should_receive(:dump_table_records)
53
- SerializationHelper::Dump.dump_table(@io, 'mytable')
54
- end
45
+ it "should paginate records from the database and return them" do
46
+ ActiveRecord::Base.connection.stub!(:select_all).and_return(
47
+ [ { 'a' => 1, 'b' => 2, 'json' => '[{"c": 3}]' } ],
48
+ [ { 'a' => 3, 'b' => 4, 'json' => nil } ]
49
+ )
55
50
 
56
- it "should not dump a table's contents when the record count is zero" do
57
- SerializationHelper::Dump.stub!(:table_record_count).with('mytable').and_return(0)
58
- SerializationHelper::Dump.should_not_receive(:dump_table_columns)
59
- SerializationHelper::Dump.should_not_receive(:dump_table_records)
60
- SerializationHelper::Dump.dump_table(@io, 'mytable')
51
+ records = [ ]
52
+ SerializationHelper::Dump.each_table_page('mytable', 1) do |page|
53
+ page.size.should == 1
54
+ records.concat(page)
61
55
  end
62
56
 
57
+ records.should == [ { 'a' => 1, 'b' => 2, 'json' => [{'c' => 3}] }, { 'a' => 3, 'b' => 4, 'json' => nil } ]
58
+ end
63
59
 
60
+ it "should dump a table's contents to yaml" do
61
+ SerializationHelper::Dump.should_receive(:dump_table_columns)
62
+ SerializationHelper::Dump.should_receive(:dump_table_records)
63
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
64
+ end
64
65
 
66
+ it "should not dump a table's contents when the record count is zero" do
67
+ SerializationHelper::Dump.stub!(:table_record_count).with('mytable').and_return(0)
68
+ SerializationHelper::Dump.should_not_receive(:dump_table_columns)
69
+ SerializationHelper::Dump.should_not_receive(:dump_table_records)
70
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
71
+ end
65
72
  end
@@ -45,8 +45,7 @@ describe SerializationHelper::Load do
45
45
  ActiveRecord::Base.connection.stub!(:quote).with(2, mcb).and_return("'2'")
46
46
  ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
47
47
  ActiveRecord::Base.connection.stub!(:quote).with(4, mcb).and_return("'4'")
48
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('1','2')")
49
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('3','4')")
48
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('1','2'), ('3','4')")
50
49
 
51
50
  SerializationHelper::Load.load_records('mytable', ['a', 'b'], [[1, 2], [3, 4]])
52
51
  end
@@ -61,8 +60,7 @@ describe SerializationHelper::Load do
61
60
  ActiveRecord::Base.connection.stub!(:quote).with(2, mccount).and_return("'2'")
62
61
  ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
63
62
  ActiveRecord::Base.connection.stub!(:quote).with(4, mccount).and_return("'4'")
64
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('1','2')")
65
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('3','4')")
63
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('1','2'), ('3','4')")
66
64
 
67
65
  SerializationHelper::Load.load_records('mytable', ['a', 'count'], [[1, 2], [3, 4]])
68
66
  end
@@ -6,10 +6,6 @@ describe SerializationHelper::Utils, " convert records utility method" do
6
6
  ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
7
7
  end
8
8
 
9
- it "returns an array of hash values using an array of ordered keys" do
10
- SerializationHelper::Utils.unhash({ 'a' => 1, 'b' => 2 }, [ 'b', 'a' ]).should == [ 2, 1 ]
11
- end
12
-
13
9
  it "should unhash each hash an array using an array of ordered keys" do
14
10
  SerializationHelper::Utils.unhash_records([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ], [ 'b', 'a' ]).should == [ [ 2, 1 ], [ 4, 3 ] ]
15
11
  end
@@ -6,7 +6,7 @@ describe YamlDb::Dump do
6
6
  silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
7
7
  ActiveRecord::Base.stub(:connection).and_return(stub('connection').as_null_object)
8
8
  ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
9
- ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a', :type => :string), mock('b', :name => 'b', :type => :string) ])
9
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a', name: 'a', type: :string, sql_type: 'text'), mock('b', name: 'b', type: :string, sql_type: 'text') ])
10
10
  ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
11
11
  ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
12
12
  YamlDb::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
data/yaml_db.gemspec CHANGED
@@ -5,11 +5,11 @@
5
5
 
6
6
  Gem::Specification.new do |s|
7
7
  s.name = "yaml_db_with_schema_tables"
8
- s.version = "0.2.4"
8
+ s.version = "0.3.2"
9
9
 
10
10
  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11
11
  s.authors = ["Adam Wiggins", "Orion Henry", "Martin Honermeyer"]
12
- s.date = "2014-01-20"
12
+ s.date = "2017-12-04"
13
13
  s.description = "\nYamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.\nThis can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.\nAny database that has an ActiveRecord adapter should work\n"
14
14
  s.email = "martin.honermeyer@zweitag.de"
15
15
  s.extra_rdoc_files = [
@@ -23,6 +23,7 @@ Gem::Specification.new do |s|
23
23
  "about.yml",
24
24
  "init.rb",
25
25
  "lib/csv_db.rb",
26
+ "lib/json_db.rb",
26
27
  "lib/serialization_helper.rb",
27
28
  "lib/tasks/yaml_db_tasks.rake",
28
29
  "lib/yaml_db.rb",
@@ -32,6 +33,8 @@ Gem::Specification.new do |s|
32
33
  "spec/serialization_helper_dump_spec.rb",
33
34
  "spec/serialization_helper_load_spec.rb",
34
35
  "spec/serialization_utils_spec.rb",
36
+ "spec/json_dump_spec.rb",
37
+ "spec/json_load_spec.rb",
35
38
  "spec/yaml_dump_spec.rb",
36
39
  "spec/yaml_load_spec.rb",
37
40
  "spec/yaml_utils_spec.rb",
@@ -40,7 +43,7 @@ Gem::Specification.new do |s|
40
43
  s.homepage = "http://github.com/zweitag/yaml_db"
41
44
  s.require_paths = ["lib"]
42
45
  s.rubygems_version = "1.8.17"
43
- s.summary = "yaml_db allows export/import of database into/from yaml files"
46
+ s.summary = "yaml_db allows export/import of database into/from yaml/json files"
44
47
 
45
48
  if s.respond_to? :specification_version then
46
49
  s.specification_version = 3
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: yaml_db_with_schema_tables
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.4
4
+ version: 0.3.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Adam Wiggins
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2014-01-20 00:00:00.000000000 Z
13
+ date: 2017-12-04 00:00:00.000000000 Z
14
14
  dependencies: []
15
15
  description: |2
16
16
 
@@ -23,18 +23,21 @@ extensions: []
23
23
  extra_rdoc_files:
24
24
  - README.markdown
25
25
  files:
26
- - .travis.yml
26
+ - ".travis.yml"
27
27
  - README.markdown
28
28
  - Rakefile
29
29
  - VERSION
30
30
  - about.yml
31
31
  - init.rb
32
32
  - lib/csv_db.rb
33
+ - lib/json_db.rb
33
34
  - lib/serialization_helper.rb
34
35
  - lib/tasks/yaml_db_tasks.rake
35
36
  - lib/yaml_db.rb
36
37
  - lib/yaml_db_with_schema_tables.rb
37
38
  - spec/base.rb
39
+ - spec/json_dump_spec.rb
40
+ - spec/json_load_spec.rb
38
41
  - spec/serialization_helper_base_spec.rb
39
42
  - spec/serialization_helper_dump_spec.rb
40
43
  - spec/serialization_helper_load_spec.rb
@@ -52,19 +55,18 @@ require_paths:
52
55
  - lib
53
56
  required_ruby_version: !ruby/object:Gem::Requirement
54
57
  requirements:
55
- - - '>='
58
+ - - ">="
56
59
  - !ruby/object:Gem::Version
57
60
  version: '0'
58
61
  required_rubygems_version: !ruby/object:Gem::Requirement
59
62
  requirements:
60
- - - '>='
63
+ - - ">="
61
64
  - !ruby/object:Gem::Version
62
65
  version: '0'
63
66
  requirements: []
64
67
  rubyforge_project:
65
- rubygems_version: 2.1.11
68
+ rubygems_version: 2.6.7
66
69
  signing_key:
67
70
  specification_version: 3
68
- summary: yaml_db allows export/import of database into/from yaml files
71
+ summary: yaml_db allows export/import of database into/from yaml/json files
69
72
  test_files: []
70
- has_rdoc: