ferry 0.0.6 → 0.0.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: e93fbb622f8e57b8062577a77cda6af5606aa9a6
-  data.tar.gz: 36925583a0ad7a5909126ea32ae6e9536de0079d
+  metadata.gz: 725496d9f272e4289334f35cd5f29dfb877fa77e
+  data.tar.gz: 62516d8f0982f15ebc85c52ea784b4e9e768b7d1
 SHA512:
-  metadata.gz: 832f010cce02e91778b93bfcfe622c90accd4928d04941e8c9c97db2cacb14ace02f3f842f7bbeb28a51d287d61af56491298dc3756f86f7a1b2353cdc61deae
-  data.tar.gz: 639f6a78c3ba9efdbcf90284dd4028bca2cadae6ab37d7d2bdad78301548ec7f357c906bca55bc0678e01d6914f7785d3ebc3fcf9c1cb699489252c123035445
+  metadata.gz: 2170274c90daecbb1874cb4a498aaf7cbe7beb9a9342f5efdb088527aac875bc77960cb71855a40bb94bbbe9ae7c4166d5f134cba1ee930ceaa68f0ca5d30248
+  data.tar.gz: d0b93db1a4a7f403ab6beb891f2b8bf53711b2c2672d1ff198a9d683326107dbb8e75438ca0a4aafa0b32e067fb3bf9d853ff07d478cec509493080fa99ce45f

data/Gemfile

@@ -1,4 +1,2 @@
 source 'https://rubygems.org'
-
-# Specify your gem's dependencies in ferry.gemspec
 gemspec

data/README.md

@@ -1,23 +1,35 @@
 # Ferry
 
+## What is Ferry?
 Ferry is a data migration and data manipulation tool that seeks to quickly and easily reduce overhead when dealing with big data problems.
 
-## TO-DO
+## What can I use Ferry for? (Use Cases)
+See the [ferry_demo](http://github.com/cmu-is-projects/ferry_demo.com) ROR/Sqlite app for guidance on using Ferry!
 
+Manipulation Use Cases
+  - RESTful column/ row interaction
+
+Migration
+  - Exporting data to various file formats (.csv, .sql, .yml)
+  - Importing data from various file formats
+  - Migrating data to third party hosts (Amazon S3, Oracle)
+  - Migrating data to a different database
+
+## TO-DOs
 - [ ] Refactoring before public release
-  - [x] Define action-items for refactor
-  - [x] Provide working example(s) of using ferry
-  - [ ] Public release fine-tuning
+- [x] Define action-items for refactor
+- [x] Provide working example(s) of using ferry (ferry_demo app)
+- [ ] CLI tool
+- [ ] Simple CSV export
+  - [ ] using sqlite3
+  - [ ] using psql
+- [ ] RESTful column interaction
+  - [ ] Understanding relationships between generating migrations and migration files in place
 - [ ] Tests
-  - [ ] Testing input for migrate method (max_workers, batch_size)
-  - [ ] Testing that there is an ActiveRecord::Relation object being passed to find_in_batches
-  - [ ] Migration Scenarios - dummy class migration
-- [ ] Refactor logging logic into Logger class
-  - [x] Initial revision
-  - [ ] Review
+- [ ] Rolling back on errors / mishaps during migrations and manipulations
+  - [ ] Host documentation site via GitHub pages
 
 ## Installation
-
 Add this line to your application's Gemfile:
 
     gem 'ferry'
@@ -30,197 +42,6 @@ Or install it yourself as:
 
     $ gem install ferry
 
-## Usage
-
-Usage pending. See examples / submit PR's for your ideas.
-
-## Example(s)
-###### 3 September 2014
-Use Case Ideas
-
-Note: Demo app can initially function with RoR and Postgres.
-
-Manipulation Use Cases
-  - CRUD for Columns
-  - Copy & Paste Columns
-  - CRUD for Rows
-  - Understanding relationships between generating migrations and migration files in place
-
-Migration
-  - Exporting data to various file formats (.csv, .sql, .yml)
-  - Importing data from various file formats
-  - Migrating data to third party hosts (Amazon S3, Oracle)
-  - Migrating data to a different database
-
-Important things to consider and remember
-  - Rolling back on errors / mishaps during migrations and manipulations
-  - Host documentation site via GitHub pages
-
-
-###### 30 August 2014
-Below is an initial implementation of how ferry will work
-
-```
-# encoding: UTF-8
-require 'consortium'
-
-task :load_wm_design do
-  class WmDesign < Design
-    self.table_name = :wm_design
-  end
-end
-
-namespace :consortium_example do
-  desc "writes design cigs to individual xml files using consortium"
-  task :write_local => [:load_wm_design] do
-    hostname = Socket.gethostname
-    FileUtils.mkdir "consortium_migration_#{hostname}" unless Dir["consortium_migration_#{hostname}"].present?
-    homedir = "consortium_migration_#{hostname}"
-
-    range = Design.where("savedate > ?", 15.hours.ago.strftime("%d.%m.%Y %H").to_datetime)
-
-    consortium_runtime = Benchmark.measure do
-      range.migrate({max_workers: 4, batch_size: 500}) do |collection|
-        collection.each do |design|
-          cons_place_design_content_in_batch(design, homedir, design.composite_id)
-        end
-      end
-    end
-    puts "#{consortium_runtime}"
-  end
-
-  private
-
-  def cons_place_design_content_in_batch(design, homedir, composite_id)
-    begin
-    create_xml_file(homedir, composite_id, design)
-    rescue Exception => e
-      File.rename("#{homedir}/#{composite_id}.xml", "#{homedir}/#{composite_id}.xml.failed")
-      raise e
-    end
-  end
-
-  def create_xml_file(homedir, composite_id, design)
-    design.updated_at ? updated_at = design.updated_at.to_time : updated_at = design.created_at.to_time
-    FileUtils.touch "#{homedir}/#{composite_id}.xml"
-    file = File.open("#{homedir}/#{composite_id}.xml", 'w')
-    file.puts design.content
-    file.close
-    FileUtils.touch "#{homedir}/#{composite_id}.xml", :mtime => updated_at
-  end
-end
-```
-
-###### 29 July 2014
-Version 0.0.1 is functional with the rake task defined here :: https://github.com/customink/design_content_migration/blob/master/lib/tasks/ferry_example.rake#L10
-
-Please manually install ferry from your locally cloned repo ...
-```
-git clone git@github.com:customink/ferry.git
-cd ferry
-gem build ferry.gemspec
-gem install ferry
-```
-add it to your app's Gemfile
-```
-gem 'ferry'
-```
-and then
-```
-bundle install
-```
-as it has not been pushed to rubygems.com yet.
-
-Tests - Coming soon to an editor near me!
-
-###### 28 July 2014
-Ferry should not support Oracle.
-
-###### 25 July 2014
-After a few more reviews with @metaskills, @gilr00y, @jdlehman, and @danielwheeler1987, Ferry will extend ActiveRecord with a "migrate" (more legit name search still in naming progress) method. From there we are going to pass the same relation to find in batches to a worker which will plow through the batch passed to it via a yield call from the task.
-
-Tests will include; validate the data passed into the worker (log) and testing that there is an ActiveRecord::Relation being passed to find_in_batches.
-
-###### 23 July 2014
-After a few chats with @gilr00y and @jdlehman Ferry may extend ActiveRecord with a "migrate" method we could call on an ActiveRecord object. From there that object would call an Engine instance with appropriate fields to kickoff the actual data migration.
-
-There is some logic duplication and layer duplication between the Engine class and the "migrate" method that extends ActiveRecord. Still working out how to concisely write logic that handles the management of forking connection and engine init calls.
-
-```
-require "ferry/version"
-require 'models/engine'
-require 'models/logger'
-
-module Ferry
-  class ActiveRecord
-    def self.migrate(&block)
-      yield
-    end
-  end
-end
-```
-
-This implementation should be able to run something like this ...
-
-```
-engine = Engine.new(
-  Design.where("savedate > ?", 6.months.ago.strftime("%d.%m.%Y %H").to_datetime).id,
-  Design.where("savedate > ?", 3.months.ago.strftime("%d.%m.%Y %H").to_datetime).id,
-  100_000,
-  1_000,
-  "log/ferry"
-)
-
-Design.where("savedate > ?", 130.hours.ago.strftime("%d.%m.%Y %H").to_datetime).migrate(
-  engine.run do | start_id, end_id, chunk_size, batch_size, log |
-    worker.run do | start_id, chunk_size, batch_size, log |
-      worker_end_id = start_id + chunk_size - 1
-      Design.where("id >= ? && id <= ?", start_id, worker_end_id).find_in_batches(batch_size: batch_size) do |batch|
-        # move and manipulate data as you please
-      end
-      start_id += batch_size
-    end
-  end
-)
-
-```
-
-###### 22 July 2014
-After installing ferry to your local machine or bundling from your gemfile - in your migration task make sure to define your chunker as such ...
-
-```
-require 'ferry'
-
-namespace :example do
-  task "my_migration_task" do
-
-    ferry = Engine.new(
-      :max_workers  => number_of_workers ex:8,
-      :start_id     => where_are_we_starting ex:2910, Model.first.id,
-      :end_id       => where_are_we_ending ex:8190, Model.last.id,
-      :chunk_size   => size_of_chunks_that_workers_will_process ex:42,
-      :working_dir  => ex:"path/to/working_dir"
-    )
-
-    ferry.run do |start_id, chunk_size, log|
-      begin
-        work = Model.select(":id").where("? <= id and id < ?", start_id, start_id + chunk_size)
-        rows_to_process = rel.count
-        log.puts("rows_to_process: #{rows_to_process}")
-        work.find_in_batches(:batch_size => 1_000) do
-          # doing things and logging stuff as you please ...
-        end
-      rescue Exception => e
-        log.puts "Broken on id #{id}"
-        raise e
-      end
-    end
-
-  end
-end
-```
-
-
 ## Contributing
 
 1. Fork it ( https://github.com/[my-github-username]/ferry/fork )

data/Rakefile

@@ -1,4 +1,4 @@
-require "bundler/gem_tasks"
-Bundler::GemHelper.install_tasks
 
+require "bundler/gem_tasks"
+# Bundler::GemHelper.install_tasks
 

data/bin/ferry

@@ -1,12 +1,5 @@
 #!/usr/bin/env ruby
-
 require 'ferry'
-
-export = Ferry::Export.new
-
-#ARGV[0] should be the ferry function they want to call
-#subsequent params are the options to that function
-
-if(ARGV[0] == "to_csv" )
-	export.to_csv
-end
+exporter = Ferry::Exporter.new
+exporter.to_csv if ARGV[0] == 'to_csv'
+exporter.to_new_db_type if ARGV[0] == 'to_new_db_type'

data/ferry.gemspec

@@ -15,20 +15,17 @@ Gem::Specification.new do |spec|
 
   spec.files         = `git ls-files -z`.split("\x0")
   spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
-  # spec.executables   = ["ferry"]
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ["lib"]
 
   spec.add_development_dependency "activerecord"
   spec.add_development_dependency "bundler", "~> 1.6"
-  spec.add_development_dependency "progressbar"
+  # spec.add_development_dependency "progressbar"
   spec.add_development_dependency "rake"
   spec.add_development_dependency "minitest"
   spec.add_development_dependency "rspec"
-  # spec.add_development_dependency "yaml"
-  #to test db access
   spec.add_development_dependency "pg"
   spec.add_development_dependency "sqlite3"
-
-  #spec.add_dependency "activerecord"  #whats difference between dev and not dev?
+  # need to look over installation problems with mysql :(
+  # spec.add_development_dependency "mysql"
 end

data/lib/ferry.rb

@@ -1,118 +1,259 @@
-require "ferry/version"
-require "ferry/engine"
-require "ferry/logger"
-require "csv"
 require 'active_record'
-require 'fileutils'
+require 'csv'
+require 'ferry/version'
+# require 'progressbar'
 require 'yaml'
 
 module Ferry
+  class Exporter
 
+    def which_db_env
+      ARGV[1]
+    end
 
-  class Export
-    def to_csv()
-
-      # ActiveRecord::Base.connection
-      #ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: 'db/development.sqlite3') #need to automatically get db name
-      #puts ActiveRecord::Base.configurations[Rails.env]['adapter']
-
-      info = YAML::load(IO.read("config/database.yml"))     #this holds all the db config information. pretty much a rosetta stone for dbs
-      db_type = info["production"]["adapter"]               #this tells us the db rails is using
+    def switch_to_db_type
+      ARGV[2]
+    end
 
+    def to_csv
+      info = YAML::load(IO.read("config/database.yml"))
+      db_type = info[which_db_env||"production"]["adapter"]
 
-      # puts Rails.configuration#.database_configuration[Rails.env]
-      # puts ActiveRecord::Base.configurations[Rails.env]
 
-      # type = db_type.downcase
 
+      #issues:
+      #   csv placement in directory:  do we want it in the lib?
+      #   the case where dev/test dbs are different from production db (sqlite for dev/test, pg for prod??)
+      #   empty ARGs
+      #   ARGs with invalid values
 
 
       case db_type
-        when "sqlite3"
-          puts "its sqlite3"
-
+      when "sqlite3"
+        puts "operating with sqlite3"
 
+        if(which_db_env)
+          homedir = "lib/ferry_to_csv_#{which_db_env}"
+          ActiveRecord::Base.establish_connection(adapter: db_type, database: info[which_db_env]['database'])
+          puts "connected to #{which_db_env} env db"
+          FileUtils.mkdir homedir unless Dir[homedir].present?
+          puts "exporting tables to #{homedir}"
+          # sqlite_pbar = ProgressBar.new("sqlite_to_csv", 100)
+          ActiveRecord::Base.connection.tables.each do |model|
+            full_table = ActiveRecord::Base.connection.execute("SELECT * FROM #{model};")
+            # do not create a csv for an empty table
+            if !full_table[0].nil?
+              CSV.open("#{homedir}/#{model}.csv", "w") do |csv|
+                size = full_table[0].length / 2
+                keys = full_table[0].keys.first(size)
+                #first row contains column names
+                csv << keys
+                full_table.each do |row|
+                  csv << row.values_at(*keys)
+                  # sqlite_pbar.inc
+                end
+              end
+            end
+          end
+        else #no db_env is specified
           info.keys.each do |environment|
 
-              if(environment == 'default')  #in Rails 4.1+ environments inherit from default, which does not have database so we will not include it
-                next
+            if(environment == 'default')  #in Rails 4.1+ environments inherit from default, which does not have database so we will not include it
+              next
+            end
+
+            homedir = "lib/ferry_to_csv_#{environment}"
+            ActiveRecord::Base.establish_connection(adapter: db_type, database: info[environment]['database'])  #connect to sqlite3 file
+            puts "connected to #{environment} env db"
+            FileUtils.mkdir homedir unless Dir[homedir].present?
+            puts "exporting tables to #{homedir}"
+            # sqlite_pbar = ProgressBar.new("sqlite_to_csv", 100)
+            ActiveRecord::Base.connection.tables.each do |model|                                #for each model in the db
+              full_table = ActiveRecord::Base.connection.execute("SELECT * FROM #{model};")    #get all the records
+              if !full_table[0].nil?
+                CSV.open("#{homedir}/#{model}.csv", "w") do |csv|
+                  size = full_table[0].length / 2
+                  keys = full_table[0].keys.first(size)
+                  #first row contains column names
+                  csv << keys
+                  full_table.each do |row|
+                    csv << row.values_at(*keys)
+                    # sqlite_pbar.inc
+                  end
+                end
               end
+            end
+          end
+        end
 
-              ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: info[environment]['database'])  #connect to sqlite3 file
+      when "postgresql"
+        puts "operating with postgres"
 
-              unless(Dir.exists?('db/csv'))   #creating a 'csv' folder in the 'db' folder
-                Dir.mkdir('db/csv')
-                puts 'db/csv created'
+        if(which_db_env)
+          homedir = "lib/ferry_to_csv_#{which_db_env}"
+          ActiveRecord::Base.establish_connection(
+            adapter:  'postgresql', 
+            host:     info[which_db_env]['host'] || 'localhost', 
+            username: info[which_db_env]['username'], 
+            password: info[which_db_env]['password'], 
+            database: info[which_db_env]['database'], 
+            encoding: info[which_db_env]['encoding']
+          )
+          puts "connected to #{which_db_env} env db"
+          FileUtils.mkdir homedir unless Dir[homedir].present?
+          puts "exporting tables to #{homedir}"
+          # psql_pbar = ProgressBar.new("psql_to_csv", 100)
+          ActiveRecord::Base.connection.tables.each do |model|
+            full_table = ActiveRecord::Base.connection.execute("SELECT * FROM #{model};")
+            # do not create a csv for an empty table
+            if full_table.num_tuples > 0
+              CSV.open("#{homedir}/#{model}.csv", "w") do |csv|
+                size = full_table[0].length / 2
+                keys = full_table[0].keys.first(size)
+                #first row contains column names
+                csv << keys
+                full_table.each do |row|
+                  csv << row.values_at(*keys)
+                  # psql_pbar.inc
+                end
               end
+            end
+          end
+        else
+          info.keys.each do |environment|
 
-                unless(Dir.exists?('db/csv/'+environment))    #creating folders for each file (dev, test, prod, etc)
-                  Dir.mkdir('db/csv/'+environment)
-                  puts 'db/csv/'+environment+' created'
-                end
+            if(environment == 'default')  #in Rails 4.1+ environments inherit from default, which does not have database so we will not include it
+              next
+            end
+            homedir = "lib/ferry_to_csv_#{environment}"
 
-                      ActiveRecord::Base.connection.tables.each do |model|                                #for each model in the db
-                        everything = ActiveRecord::Base.connection.execute('SELECT * FROM '+model+';')    #get all the records
+            ActiveRecord::Base.establish_connection(
+              adapter:  'postgresql', 
+              host:     info[environment]['host'] || 'localhost', 
+              username: info[environment]['username'], 
+              password: info[environment]['password'], 
+              database: info[environment]['database'], 
+              encoding: info[environment]['encoding']
+            )
+            puts "connected to #{environment} env db"
+            FileUtils.mkdir homedir unless Dir[homedir].present?
+            puts "exporting tables to #{homedir}"
+            # psql_pbar = ProgressBar.new("psql_to_csv", 100)
+            ActiveRecord::Base.connection.tables.each do |model|
+              full_table = ActiveRecord::Base.connection.execute("SELECT * FROM #{model};")
+              # do not create a csv for an empty table
+              if full_table.num_tuples > 0
+                CSV.open("#{homedir}/#{model}.csv", "w") do |csv|
+                  size = full_table[0].length / 2
+                  keys = full_table[0].keys.first(size)
+                  #first row contains column names
+                  csv << keys
+                  full_table.each do |row|
+                    csv << row.values_at(*keys)
+                    # psql_pbar.inc
+                  end
+                end
+              end
+            end
+          end
+        end
+      when "mysql2"
+        puts "operating with mysql2"
 
-                        if everything[0].nil?   #do not create a csv for an empty table
-                          next
-                        end
+        if(which_db_env)
+          homedir = "lib/ferry_to_csv_#{which_db_env}"
 
-                        CSV.open("db/csv/"+environment+"/"+model+".csv", "w") do |csv|    #create a csv for each table, titled 'model.csv'
+          ActiveRecord::Base.establish_connection(    #this may not work on default rails if production db is not created (must run rake db:create:all)
+            adapter:  'mysql2',
+            host:     info[which_db_env]['host'] || 'localhost', 
+            username: info[which_db_env]['username'], 
+            password: info[which_db_env]['password'], 
+            database: info[which_db_env]['database']
+          )
+          puts "connected to #{which_db_env} env db"
+          FileUtils.mkdir homedir unless Dir[homedir].present?
+          puts "exporting tables to #{homedir}"
+          # psql_pbar = ProgressBar.new("psql_to_csv", 100)
 
-                          size = everything[0].length / 2
-                          keys = everything[0].keys.first(size)
+          ActiveRecord::Base.connection.tables.each do |model|                              #for each model in the db
+            columns = ActiveRecord::Base.connection.execute("SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA`= '#{info[which_db_env]['database']}' AND `TABLE_NAME`='#{model}';")
+            CSV.open("#{homedir}/#{model}.csv", "w") do |csv|
 
-                          csv << keys     #first row contains column names
-                          
-                          everything.each do |row|
-                            csv << row.values_at(*keys)     #subsequent rows hold record values
-                          end
-                        end
-                      end
+              col_names=[]
+              columns.each do |col|
+                col_names.append(col[0])  #append the column names to an array, makes for good formatting
+              end
+              csv << col_names  #first csv row is of column names
 
+              full_table = ActiveRecord::Base.connection.execute("SELECT * FROM #{model};")
+              full_table.each do |row|
+                csv << row  #not sure if this will hold, but no 'values_at' method exists
+                # psql_pbar.inc
+              end
+            end
           end
+        else
+          info.keys.each do |environment|
 
+            if(environment == 'default')  #in Rails 4.1+ environments inherit from default, which does not have database so we will not include it
+              next
+            end
 
-        when "mysql"
-          puts "its mysql"
-        when "postgres"
-          puts "its postgres"
-        else
-          puts "unknown db type"
+            homedir = "lib/ferry_to_csv_#{environment}"
+
+              ActiveRecord::Base.establish_connection(    #this may not work on default rails if production db is not created (must run rake db:create:all)
+                adapter:  'mysql2',
+                host:     info[environment]['host'] || 'localhost', 
+                username: info[environment]['username'], 
+                password: info[environment]['password'], 
+                database: info[environment]['database']
+              )
+            puts "connected to #{environment} env db"
+            FileUtils.mkdir homedir unless Dir[homedir].present?
+            puts "exporting tables to #{homedir}"
+            # psql_pbar = ProgressBar.new("psql_to_csv", 100)
+
+            ActiveRecord::Base.connection.tables.each do |model|                              #for each model in the db
+              columns = ActiveRecord::Base.connection.execute("SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA`= '#{info[environment]['database']}' AND `TABLE_NAME`='#{model}';")
+              CSV.open("#{homedir}/#{model}.csv", "w") do |csv|
+
+                col_names=[]
+                columns.each do |col|
+                  col_names.append(col[0])  #append the column names to an array, makes for good formatting
+                end
+                csv << col_names  #first csv row is of column names
+
+                full_table = ActiveRecord::Base.connection.execute("SELECT * FROM #{model};")
+                full_table.each do |row|
+                  csv << row  #not sure if this will hold, but no 'values_at' method exists
+                  # psql_pbar.inc
+                end
+              end
+            end
+          end
+        end  
+      when "mongo"
+        puts "mongo is currently not supported"
+      else
+        puts "Unknown db type or no database associated with this application."
       end
-          
-            
-      puts Time.now()
+    end
+
+    def to_new_db_type
+      info = YAML::load(IO.read("config/database.yml"))
+      current_db_type = info[which_db_env]["adapter"]
+      puts "switching the #{which_db_env} database's adapter"
+      puts "current_db_type: #{current_db_type}"
+      puts "to_new_db_type: #{switch_to_db_type}"
 
+      # check for dependencies
+      # if dependencies exist - install them
+      # create new connection
+      # transfer old db into new connection
+      # drop old connection
+      # update the config file
+      # profit
     end
-  end
 
-  # class ActiveRecord::Relation
-    # def migrate(options, &block)
-    #   options[:max_workers] ||= 4
-    #   options[:batch_size]  ||= 10_000
-
-    #   log = Logger.new()
-
-    #   active_workers = []
-    #   collection = self
-    #   collection.find_in_batches(batch_size: options[:batch_size]) do |batch|
-    #     if active_workers.length >= options[:max_workers]
-    #       log.write "active_workers oversized at capacity of #{active_workers.length}/#{options[:max_workers]}"
-    #       finished_process = Process.wait
-    #       log.write "finished_process: #{finished_process}"
-    #       active_workers.delete finished_process
-    #       log.write "active_workers capacity now at: #{active_workers.length}/#{options[:max_workers]}"
-    #     else
-    #       active_workers << fork do
-    #         ActiveRecord::Base.connection.reconnect!
-    #         log.write "kicking off engine on batch(#{batch.first}-#{batch.last})"
-    #         engine = Engine.new()
-    #         engine.run({log: log, batch: batch}, &block)
-    #       end
-    #     end
-    #     ActiveRecord::Base.connection.reconnect!
-    #   end
-    # end
-  # end
+  end
 end

data/lib/ferry/version.rb

@@ -1,3 +1,3 @@
 module Ferry
-  VERSION = "0.0.6"
+  VERSION = "0.0.8"
 end

data/spec/{ferry_spec.rb → lib/ferry_spec.rb}

@@ -6,5 +6,4 @@ describe "connect" do
       Post.by_year.map(&:text).should include("First post!")
     end
   end
-
-end
+end

data/spec/spec_helper.rb

@@ -1,89 +1,11 @@
-# This file was generated by the `rspec --init` command. Conventionally, all
-# specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
-# The generated `.rspec` file contains `--require spec_helper` which will cause this
-# file to always be loaded, without a need to explicitly require it in any files.
-#
-# Given that it is always loaded, you are encouraged to keep this file as
-# light-weight as possible. Requiring heavyweight dependencies from this file
-# will add to the boot time of your test suite on EVERY test run, even for an
-# individual file that may not need all of that loaded. Instead, make a
-# separate helper file that requires this one and then use it only in the specs
-# that actually need it.
-#
-# The `.rspec` file also contains a few flags that are not defaults but that
-# users commonly want.
-#
-# See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
 require 'ferry'
 
-ActiveRecord::Base.establish_connection(:adapter => "sqlite3", 
-                                       :database => File.dirname(__FILE__) + "/ferry.sqlite3")
+ActiveRecord::Base.establish_connection(:adapter => "sqlite3",
+                                        :database => File.dirname(__FILE__) + "/ferry.sqlite3")
 
 load File.dirname(__FILE__) + '/support/schema.rb'
 load File.dirname(__FILE__) + '/support/models.rb'
 load File.dirname(__FILE__) + '/support/data.rb'
 
-
-
 RSpec.configure do |config|
-# The settings below are suggested to provide a good initial experience
-# with RSpec, but feel free to customize to your heart's content.
-=begin
-  # These two settings work together to allow you to limit a spec run
-  # to individual examples or groups you care about by tagging them with
-  # `:focus` metadata. When nothing is tagged with `:focus`, all examples
-  # get run.
-  config.filter_run :focus
-  config.run_all_when_everything_filtered = true
-
-  # Many RSpec users commonly either run the entire suite or an individual
-  # file, and it's useful to allow more verbose output when running an
-  # individual spec file.
-  if config.files_to_run.one?
-    # Use the documentation formatter for detailed output,
-    # unless a formatter has already been configured
-    # (e.g. via a command-line flag).
-    config.default_formatter = 'doc'
-  end
-
-  # Print the 10 slowest examples and example groups at the
-  # end of the spec run, to help surface which specs are running
-  # particularly slow.
-  config.profile_examples = 10
-
-  # Run specs in random order to surface order dependencies. If you find an
-  # order dependency and want to debug it, you can fix the order by providing
-  # the seed, which is printed after each run.
-  #     --seed 1234
-  config.order = :random
-
-  # Seed global randomization in this process using the `--seed` CLI option.
-  # Setting this allows you to use `--seed` to deterministically reproduce
-  # test failures related to randomization by passing the same `--seed` value
-  # as the one that triggered the failure.
-  Kernel.srand config.seed
-
-  # rspec-expectations config goes here. You can use an alternate
-  # assertion/expectation library such as wrong or the stdlib/minitest
-  # assertions if you prefer.
-  config.expect_with :rspec do |expectations|
-    # Enable only the newer, non-monkey-patching expect syntax.
-    # For more details, see:
-    #   - http://myronmars.to/n/dev-blog/2012/06/rspecs-new-expectation-syntax
-    expectations.syntax = :expect
-  end
-
-  # rspec-mocks config goes here. You can use an alternate test double
-  # library (such as bogus or mocha) by changing the `mock_with` option here.
-  config.mock_with :rspec do |mocks|
-    # Enable only the newer, non-monkey-patching expect syntax.
-    # For more details, see:
-    #   - http://teaisaweso.me/blog/2013/05/27/rspecs-new-message-expectation-syntax/
-    mocks.syntax = :expect
-
-    # Prevents you from mocking or stubbing a method that does not exist on
-    # a real object. This is generally recommended.
-    mocks.verify_partial_doubles = true
-  end
-=end
 end

data/spec/support/data.rb

@@ -1,11 +1,11 @@
 Design.create(
-      :design_id => 1,
-      :product_id => 1,
-      :account_id => 1,
-      :account_file => "Reunion 2014",
-      :save_method => "WWW",
-      :total_units => 25,
-      :has_upload => true,
-      :created_at => DateTime.now,
-      :updated_at => DateTime.now,
-      :postal_code => 96822)
+  :design_id => 1,
+  :product_id => 1,
+  :account_id => 1,
+  :account_file => "Reunion 2014",
+  :save_method => "WWW",
+  :total_units => 25,
+  :has_upload => true,
+  :created_at => DateTime.now,
+  :updated_at => DateTime.now,
+  :postal_code => 96822)

data/spec/support/models.rb

@@ -1,3 +1,2 @@
 class Design < ActiveRecord::Base
-
-end
+end

data/spec/support/schema.rb

@@ -1,18 +1,17 @@
 ActiveRecord::Schema.define do
   self.verbose = false
+  create_table :designs do |t|
+    t.integer :design_id
+    t.integer :product_id
+    t.integer :account_id
+    t.string :account_file
+    t.string :save_method
+    t.integer :total_units
+    t.boolean :has_upload
+    t.date :created_at
+    t.date :updated_at
+    t.integer :postal_code
 
-    create_table :designs do |t|
-      t.integer :design_id
-      t.integer :product_id
-      t.integer :account_id
-      t.string :account_file
-      t.string :save_method
-      t.integer :total_units
-      t.boolean :has_upload
-      t.date :created_at
-      t.date :updated_at
-      t.integer :postal_code
-
-      t.timestamps
-    end
-end
+    t.timestamps
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ferry
 version: !ruby/object:Gem::Version
-  version: 0.0.6
+  version: 0.0.8
 platform: ruby
 authors:
 - Anthony Corletti
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-09-24 00:00:00.000000000 Z
+date: 2014-09-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -40,20 +40,6 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.6'
-- !ruby/object:Gem::Dependency
-  name: progressbar
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -142,15 +128,11 @@ files:
 - README.md
 - Rakefile
 - bin/ferry
-- dbfile.sqlite3
 - ferry.gemspec
 - lib/ferry.rb
-- lib/ferry/engine.rb
-- lib/ferry/logger.rb
 - lib/ferry/version.rb
-- lib/tasks/dump_task.rake
 - spec/ferry.sqlite3
-- spec/ferry_spec.rb
+- spec/lib/ferry_spec.rb
 - spec/spec_helper.rb
 - spec/support/data.rb
 - spec/support/models.rb
@@ -181,7 +163,7 @@ specification_version: 4
 summary: Ferry is a data migration and data manipulation tool
 test_files:
 - spec/ferry.sqlite3
-- spec/ferry_spec.rb
+- spec/lib/ferry_spec.rb
 - spec/spec_helper.rb
 - spec/support/data.rb
 - spec/support/models.rb

data/lib/ferry/engine.rb

@@ -1,16 +0,0 @@
-class Engine
-  def initialize(options={})
-  end
-
-  def run(options, &block)
-    log = options[:log]
-    collection = options[:batch]
-    log.write "collection length: #{collection.length}"
-    begin
-      instance_exec(collection, &block)
-    rescue Exception => e
-      log.write "Error: #{e}"
-    end
-    log.write "worker finished"
-  end
-end

data/lib/ferry/logger.rb

@@ -1,12 +0,0 @@
-class Logger
-  # def initialize(options={})
-  #   @homedir = options[:homedir] ||= "log"
-  #   FileUtils.mkdir @homedir unless Dir[@homedir].present?
-  #   FileUtils.touch "#{@homedir}/ferry.log"
-  # end
-
-  # def write(msg)
-  #   log = File.open("#{@homedir}/ferry.log", 'w')
-  #   log.puts msg
-  # end
-end

data/lib/tasks/dump_task.rake

@@ -1,18 +0,0 @@
-namespace :db do
-  desc "Dump schema and data to db/schema.rb and db/data.yml"
-  task(:dump => [ "db:schema:dump", "db:data:dump" ])
-
-  namespace :data do
-	    def db_dump_data_file (extension = "yml")
-	      "#{dump_dir}/data.#{extension}"
-	    end
-
-	    desc "Dump contents of database to db/data.extension (defaults to yaml)"
-	    task :dump => :environment do
-	      # format_class = ENV['class'] || "YamlDb::Helper"
-	      # helper = format_class.constantize
-	      # SerializationHelper::Base.new(helper).dump db_dump_data_file helper.extension
-	      puts "yolo"
-	    end
-	end
-end