dbsync 0.2.0 → 0.3.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 4d8a3ddf36966a8c9aec5c43f62b383f6a01dac4
+  data.tar.gz: 04dea7b42bca9c6492be1153947ca10975bca739
+SHA512:
+  metadata.gz: 2a0119d2c7b4f67f5e7ec95ebcf29000a6692a7f8c0fd1523703da10efcb71b827c4d5debc347858a086e52a1261b5d5bdb372aae27aadd59a8bee9c9d9de7dc
+  data.tar.gz: 7481b11c17c29b718c6b4f7addc1ae1345e7e468a012a10b23c0ff693e8a53b815f788fef3eb83312bb93e684b5557ae8ccf86fce23215e6624ddb53f5e974d7

data/.gitignore

@@ -2,3 +2,4 @@
 .bundle
 Gemfile.lock
 pkg/*
+*.dump

data/CHANGELOG.md

@@ -1,3 +1,9 @@
-## 0.2.0
+## 0.3.0 (unreleased)
+* Removed all Rails dependencies. Rails is now optional.
+* Removed scp usage. Everything uses rsync now.
+* Added Dbsync.file_config and Dbsync.db_config to manually set configuration.
+* [DEPRECATED] local_dir, remote_dir, remote_host, filename options. Instead, combine them into just `local` and `remote` configurations (see README for more).
+
+## 0.2.0 (2014-02-13)
 * Internal refactoring.
 * Updated gem dependencies.

data/README.md

@@ -0,0 +1,92 @@
+# dbsync
+
+A set of rake tasks to help you sync your production data with your local database for development. **Currently only supports MySQL.**
+
+## Usage
+
+Add to your gemfile for the groups that will need it:
+
+```ruby
+group :development, :staging do
+  gem 'dbsync'
+end
+```
+
+Or just install it as a gem:
+
+```
+gem install dbsync
+```
+
+### For Rails
+
+Add the following to your `config/environments/development.rb` file. Depending on your staging setup, it may also be useful to you to add some `dbsync` config to your `staging.rb` environment file. **`dbsync` will not run in production.**
+
+```ruby
+config.dbsync = {
+  :remote => '66.123.4.567:~/dbsync/mydb.dump',
+  :local  => '../dbsync/mydb.dump'
+}
+```
+
+Dbsync will automatically use your Rails environment's database configuration.
+
+### For non-Rails
+
+You can also specify the dbsync configuration with `Dbsync.file_config` and `Dbsync.db_config`:
+
+```ruby
+Dbsync.file_config = {
+  :local => "../dbsync/dbsync.dump",
+  :remote => "dbuser@100.0.100.100:~dbuser/dbsync.dump"
+}
+
+Dbsync.db_config = {
+  :adapter  => "mysql2", # Not actually used yet
+  :database => "yourdb",
+  :username => "youruser",
+  :password => "yourcoolpassword"
+}
+```
+
+### The server
+
+Now just make sure you have something on the remote server updating that dumpfile. I recommend a cronjob:
+
+```
+0 */12 * * * /usr/bin/mysqldump yourapp_production > /home/dbsync/yourapp_production_data.dump
+```
+
+If you are using the SSH form of rsync, you will need proper SSH access into the remote server.
+
+### Rake
+
+Run `rake -T dbsync` for all of the available tasks. The tasks are named after `git` commands mostly, so they should be pretty straight-forward for those who use `git`:
+
+```
+rake dbsync             # Alias for dbsync:pull
+rake dbsync:clone       # Copy the remote dump file, reset the local database, and load in the dump file
+rake dbsync:clone_dump  # Copy the remote dump file to a local destination
+rake dbsync:config      # Show the dbsync configuration
+rake dbsync:fetch       # Update the local dump file from the remote source
+rake dbsync:merge       # Update the local database with the local dump file
+rake dbsync:pull        # Update the local dump file, and merge it into the local database
+rake dbsync:reset       # Drop and Create the database, then load the dump file
+```
+
+
+### Caveats
+
+* The `merge` process doesn't clear out your database first. This is to improve performance. Therefore, any tables which you removed on the remote host won't be removed locally. To do a complete reset of your database, run `rake dbsync:reset`. This resets your database (`db:drop` and `db:create`), and then merges in the local file.
+* Rails: the test database isn't automatically updated when syncing to your development database. After a `dbsync` and before you run tests, you'll need to run `rake db:test:prepare` to setup your database.
+* Rails: your schema.rb isn't involed in `dbsync` at all. You need to manage it yourself.
+
+
+### TODO
+
+- Support postgres, sqlite, and anything else.
+
+
+### Feedback/Support
+
+Open an issue or send a pull request.

data/dbsync.gemspec

@@ -17,9 +17,7 @@ Gem::Specification.new do |s|
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ["lib", "lib/tasks"]
 
-  s.add_dependency "activerecord", [">= 3.2.8", "< 5"]
-  s.add_dependency "railties", [">= 3.2.8", "< 5"]
-  s.add_dependency "cocaine", "~> 0.5.3"
+  s.add_dependency "cocaine", [">= 0.5.0", "< 0.6"]
 
-  s.add_development_dependency "rspec"
+  s.add_development_dependency "rspec", '~> 0'
 end

data/lib/dbsync.rb

@@ -2,9 +2,30 @@ require "dbsync/version"
 require 'dbsync/sync'
 
 module Dbsync
-  class Railtie < Rails::Railtie
-    rake_tasks do
-      Dir[File.join(File.dirname(__FILE__),'tasks/*.rake')].each { |f| load f }
+  if defined?(Rails)
+    class Railtie < Rails::Railtie
+      rake_tasks do
+        Dir[File.join(File.dirname(__FILE__),'tasks/*.rake')].each { |f| load f }
+      end
+    end
+  end
+
+
+  class << self
+    def file_config
+      @file_config
+    end
+
+    def file_config=(config)
+      @file_config = config
+    end
+
+    def db_config
+      @db_config
+    end
+
+    def db_config=(config)
+      @db_config = config
     end
   end
 end

data/lib/dbsync/sync.rb

@@ -1,4 +1,5 @@
 require 'cocaine'
+require 'fileutils'
 
 module Dbsync
   class Sync
@@ -10,46 +11,68 @@ module Dbsync
 
 
     def initialize(ssh_config, db_config, options={})
-      ssh_config  = ssh_config.with_indifferent_access
-      db_config   = db_config.with_indifferent_access
+      ssh_config  = symbolize_keys(ssh_config)
+      db_config   = symbolize_keys(db_config)
 
       @verbose = !!options[:verbose]
 
-      @remote_host  = ssh_config[:remote_host] || raise_missing("remote_host")
-      @remote_dir   = ssh_config[:remote_dir]  || raise_missing("remote_dir")
-      @local_dir    = ssh_config[:local_dir]   || raise_missing("local_dir")
-      @filename     = ssh_config[:filename]    || raise_missing("filename")
-
       @db_username  = db_config[:username]
       @db_password  = db_config[:password]
       @db_host      = db_config[:host]
       @db_database  = db_config[:database]
 
-      @remote_file = "#{@remote_host}:" + File.join(@remote_dir, @filename)
-      @local_file  = File.expand_path(File.join(@local_dir, @filename))
+      @remote   = ssh_config[:remote]
+      @local    = File.expand_path(ssh_config[:local]) if ssh_config[:local]
+
+      if !@remote
+        $stdout.puts "DEPRECATED: The remote_host, remote_dir, and filename " \
+          "options will be removed. " \
+          "Instead, combine remote_host, remote_dir, and filename into a " \
+          "single 'remote' configuration. Example: " \
+          "'{ remote: \"dbuser@100.0.1.100:~/dbuser/yourdb.dump\" }'"
+
+        remote_host = ssh_config[:remote_host]
+        remote_dir  = ssh_config[:remote_dir]
+        filename    = ssh_config[:filename]
+        @remote = "#{remote_host}:#{File.join(remote_dir, filename)}"
+      end
+
+      if !@local
+        $stdout.puts "DEPRECATED: The local_dir and filename " \
+          "options will be removed. " \
+          "Instead, combine local_dir and filename into a " \
+          "single 'local' configuration. Example: " \
+          "'{ local: \"../dbsync/yourdb.dump\" }'"
+
+        local_dir = ssh_config[:local_dir]
+        filename  = ssh_config[:filename]
+        @local  = File.expand_path(File.join(local_dir, filename))
+      end
     end
 
 
     # Update the local dump file from the remote source (using rsync).
     def fetch
-      notify "Updating '#{@local_file}' from '#{@remote_file}' via rsync..."
+      notify "Updating '#{@local}' from '#{@remote}' via rsync..."
+
+      FileUtils.mkdir_p(File.dirname(@local))
 
       line = Cocaine::CommandLine.new('rsync', '-v :remote :local')
       line.run({
-        :remote => @remote_file,
-        :local  => @local_file
+        :remote => @remote,
+        :local  => @local
       })
     end
 
 
     # Update the local database with the local dump file.
     def merge
-      notify "Dumping data from '#{@local_file}' into '#{@db_database}'..."
+      notify "Dumping data from '#{@local}' into '#{@db_database}'..."
 
       options = ""
-      options += "-u :username " if @db_username.present?
-      options += "-p:password "  if @db_password.present?
-      options += "-h :host "     if @db_host.present?
+      options += "-u :username " if @db_username
+      options += "-p:password "  if @db_password
+      options += "-h :host "     if @db_host
 
       line = Cocaine::CommandLine.new('mysql', "#{options} :database < :local")
       line.run({
@@ -57,7 +80,7 @@ module Dbsync
         :password   => @db_password,
         :host       => @db_host,
         :database   => @db_database,
-        :local      => @local_file
+        :local      => @local
       })
     end
 
@@ -70,21 +93,27 @@ module Dbsync
 
 
     # Copy the remote dump file to a local destination.
-    # This does a full copy (using scp), so it will take longer than
-    # fetch (which uses rsync).
+    # Instead of requiring two different tools (rsync and scp) for this
+    # library, instead we'll just remove the local file if it exists
+    # then run rsync, which is essentially a full copy.
     def clone_dump
-      notify "Copying '#{@remote_file}' into '#{@local_dir}' via scp..."
-
-      line = Cocaine::CommandLine.new('scp', ':remote :local_dir')
-      line.run({
-        :remote    => @remote_file,
-        :local_dir => @local_dir
-      })
+      notify "Copying '#{@remote}' into '#{@local}' via rsync..."
+      FileUtils.rm_f(@local)
+      fetch
     end
 
 
     private
 
+    def symbolize_keys(hash)
+      return hash unless hash.keys.any? { |k| k.is_a?(String) }
+
+      result = {}
+      hash.each_key { |k| result[k.to_sym] = hash[k] }
+      result
+    end
+
+
     def raise_missing(config="")
       raise "Missing Configuration: '#{config}'. " \
             "See README for required config."

data/lib/dbsync/version.rb

@@ -1,3 +1,3 @@
 module Dbsync
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/tasks/dbsync.rake

@@ -5,20 +5,23 @@ task :dbsync do
   Rake::Task['dbsync:pull'].invoke
 end
 
+if !defined?(Rails)
+  # Dummy task
+  task :environment
+end
+
 namespace :dbsync do
   task :setup => :environment do
-    Dbsync::Sync.notify "Environment: #{Rails.env}"
+    if defined?(Rails)
+      Dbsync::Sync.notify "Rails Environment: #{Rails.env}"
 
-    if Rails.env == 'production'
-      raise "These tasks are destructive and shouldn't " \
-            "be used in the production environment."
+      if Rails.env == 'production'
+        raise "These tasks are destructive and shouldn't " \
+              "be used in the production environment."
+      end
     end
 
-    @dbsync = Dbsync::Sync.new(
-      Rails.application.config.dbsync,
-      ActiveRecord::Base.configurations[Rails.env],
-      verbose: true
-    )
+    @dbsync = Dbsync::Sync.new(file_config, db_config, verbose: true)
   end
 
 
@@ -26,7 +29,7 @@ namespace :dbsync do
   task :config => :setup do
     # We don't use Sync.notify here because we don't want or need
     # the extra output that comes with it.
-    $stdout.puts Rails.application.config.dbsync.to_yaml
+    $stdout.puts file_config.to_yaml
   end
 
 
@@ -69,3 +72,19 @@ namespace :dbsync do
     @dbsync.merge
   end
 end
+
+
+# If Dbsync.config was explicitly set, use it.
+# If Rails is defined, use the dbsync configuration there.
+# Otherwise, raise an error.
+def db_config
+  return Dbsync.db_config if Dbsync.db_config
+  return ActiveRecord::Base.configurations[Rails.env] if defined?(Rails)
+  raise "No database configuration found."
+end
+
+def file_config
+  return Dbsync.file_config if Dbsync.file_config
+  return Rails.application.config.dbsync if defined?(Rails)
+  raise "No remote configuration found."
+end

data/spec/spec_helper.rb

@@ -1,5 +1,9 @@
 require 'bundler/setup'
+require 'fileutils'
 Bundler.require
 
 RSpec.configure do |config|
+  config.after do
+    FileUtils.rm(Dir[File.expand_path("../local/*", __FILE__)])
+  end
 end

data/spec/sync_spec.rb

@@ -1,10 +1,23 @@
 require 'spec_helper'
 
 describe Dbsync::Sync do
+  let(:local_path) { File.expand_path("../local/dbsync.dump", __FILE__) }
+  let(:remote_path) { File.expand_path("../remote/dbsync.dump", __FILE__) }
+
   let(:ssh_config) do
+    {
+      :local  => File.expand_path("../local/dbsync.dump", __FILE__),
+      :remote => File.expand_path("../remote/dbsync.dump", __FILE__)
+    }
   end
 
   let(:db_config) do
+    {
+      :adapter    => "mysql2",
+      :database   => "cool_db",
+      :username   => "root",
+      :password   => nil
+    }
   end
 
 
@@ -12,12 +25,38 @@ describe Dbsync::Sync do
   end
 
   describe '#fetch' do
+    it "creates the directory if it's missing" do
+      ssh_config[:local] = File.expand_path("../local_missing/dbsync.dump", __FILE__)
+      sync = Dbsync::Sync.new(ssh_config, db_config)
+
+      sync.fetch
+
+      File.directory?(File.expand_path("../local_missing", __FILE__)).should be_true
+      FileUtils.rm_rf(File.expand_path("../local_missing", __FILE__))
+    end
+
+    it "copies the remote file the the local file" do
+      sync = Dbsync::Sync.new(ssh_config, db_config)
+      sync.fetch
+
+      File.read(local_path).should eq File.read(remote_path)
+    end
   end
 
   describe '#merge' do
   end
 
   describe '#clone_dump' do
+    it "removes the local file and rsyncs a fresh one" do
+      # Put a dummy local file in place to make sure it gets removed
+      File.open(local_path, "w") { |f| f.write "Hello." }
+      File.read(local_path).should eq "Hello."
+
+      sync = Dbsync::Sync.new(ssh_config, db_config)
+      sync.clone_dump
+
+      File.read(local_path).should eq File.read(remote_path)
+    end
   end
 
   describe '#pull' do

metadata

@@ -1,66 +1,49 @@
 --- !ruby/object:Gem::Specification
 name: dbsync
 version: !ruby/object:Gem::Version
-  version: 0.2.0
-  prerelease: 
+  version: 0.3.0
 platform: ruby
 authors:
 - Bryan Ricker
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-02-13 00:00:00.000000000 Z
+date: 2014-04-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: activerecord
-  requirement: &70160586951120 !ruby/object:Gem::Requirement
-    none: false
+  name: cocaine
+  requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ! '>='
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 3.2.8
-    - - <
+        version: 0.5.0
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '5'
+        version: '0.6'
   type: :runtime
   prerelease: false
-  version_requirements: *70160586951120
-- !ruby/object:Gem::Dependency
-  name: railties
-  requirement: &70160586945980 !ruby/object:Gem::Requirement
-    none: false
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ! '>='
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 3.2.8
-    - - <
+        version: 0.5.0
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '5'
-  type: :runtime
-  prerelease: false
-  version_requirements: *70160586945980
-- !ruby/object:Gem::Dependency
-  name: cocaine
-  requirement: &70160586961540 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: 0.5.3
-  type: :runtime
-  prerelease: false
-  version_requirements: *70160586961540
+        version: '0.6'
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70160586961120 !ruby/object:Gem::Requirement
-    none: false
+  requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ! '>='
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70160586961120
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0'
 description: A set of rake tasks to help you sync your remote production data with
   your local database for development.
 email:
@@ -69,45 +52,46 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- .gitignore
+- ".gitignore"
 - CHANGELOG.md
 - Gemfile
 - MIT-LICENSE
-- README.markdown
+- README.md
 - Rakefile
 - dbsync.gemspec
 - lib/dbsync.rb
 - lib/dbsync/sync.rb
 - lib/dbsync/version.rb
 - lib/tasks/dbsync.rake
+- spec/local/.gitkeep
 - spec/spec_helper.rb
 - spec/sync_spec.rb
 homepage: https://github.com/bricker/dbsync
 licenses:
 - MIT
+metadata: {}
 post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
 - lib/tasks
 required_ruby_version: !ruby/object:Gem::Requirement
-  none: false
   requirements:
-  - - ! '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
-  none: false
   requirements:
-  - - ! '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.11
+rubygems_version: 2.2.2
 signing_key: 
-specification_version: 3
+specification_version: 4
 summary: Easy syncing from remote to development database in Rails.
 test_files:
+- spec/local/.gitkeep
 - spec/spec_helper.rb
 - spec/sync_spec.rb

data/README.markdown

@@ -1,72 +0,0 @@
-# dbsync
-
-A set of rake tasks to help you sync your production 
-data with your local database for development.
-
-Currently only supports MySQL. The Rake tasks are written for Rails-only,
-but the Sync class can be used with ruby framework, as long as you pass in the
-correct database configuration.
-
-Support for more things will happen if anybody needs it.
-
-
-## Usage
-
-Add to your gemfile for the groups that will need it:
-
-```ruby
-group :development, :staging do
-  gem 'dbsync'
-end
-```
-
-Add the following to your `config/environments/development.rb` 
-file. Depending on your staging setup, it may also be useful 
-to you to add some `dbsync` config to your `staging.rb` 
-environment file. **Note** `dbsync` will not run in production.
-
-```ruby
-config.dbsync = {
-  :filename    => "yourapp_production_data.dump", # The name of the remote dumpfile.
-  :local_dir   => "#{Rails.root}/../dbsync",   # The local directory to store the dump file.
-  :remote_host => "66.123.4.567",              # Remote server where the dumpfile is located.
-  :remote_dir  => "~dbsync"                    # The directory on the remote server where the dumpfile is.
-}
-```
-
-Now just make sure you have something on the remote 
-server updating that dumpfile. I recommend a cronjob:
-
-    0 */12 * * * /usr/bin/mysqldump yourapp_production > /home/dbsync/yourapp_production_data.dump
-
-
-You will need proper SSH access into the remote server, 
-as the tasks use `rsync` and `scp` directly.
-
-Run `rake -T dbsync` for all of the available tasks. The 
-tasks are named after `git` commands mostly, so they
-should be pretty straight-forward for those who use `git`:
-
-```
-rake dbsync             # Alias for dbsync:pull
-rake dbsync:clone       # Copy the remote dump file, reset the local database, and load in the dump file
-rake dbsync:clone_dump  # Copy the remote dump file to a local destination
-rake dbsync:config      # Show the dbsync configuration
-rake dbsync:fetch       # Update the local dump file from the remote source
-rake dbsync:merge       # Update the local database with the local dump file
-rake dbsync:pull        # Update the local dump file, and merge it into the local database
-rake dbsync:reset       # Drop and Create the database, then load the dump file
-```
-
-### TODO
-
-- Support postgres, sqlite, and anything else.
-
-
-### Copyright
-
-Copyright (c) 2012 Bryan Ricker/SCPR.
-
-### Licence
-
-See MIT-LICENSE for more.