db_sucker 3.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 9cd1460131f4ecf6834b962773035a2d56999ed1627967302564ce9ee325a6c8
+  data.tar.gz: 8f7d8c1d1e60c4ab6c34e1fa30912a7b77e29b5a9683d2ec61bceb91f900fad2
+SHA512:
+  metadata.gz: e85e270e579c2c51e4929f54325044066ff2f49a7fa0c75e0cf3c469f1b8f783e7def3a2d59bc04f2b7f948225350d8123ecf22ef2642813efd777ffd0217d43
+  data.tar.gz: e5066416e3ca76571897f76d20f9c3943506f9a5645919d8ba4fc298aec50db4215e6e86a5d534b2f593465c9caf0f3bd5b54d7679503d9742bd9dc1db45078b

data/.gitignore

@@ -0,0 +1,16 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/CHANGELOG.md

@@ -0,0 +1,45 @@
+## 3.0.0
+
+### Updates
+
+* **Complete rewrite** using curses for status drawing, way better code structure and more features
+
+  * DbSucker is now structured to be DBMS agnostic but each DBMS will require it's API implementation.<br>
+    Mysql is the only supported adapter for now but feel free to add support for other DBMS.
+  * Note that the SequelImporter has been temporarily removed since I need to work on it some more.
+  * Added integrity checking (checksum checking of transmitted files)
+  * Added a lot of status displays
+  * Added a vim-like command interface (press : and then a command, e.g. ":?"+enter but actually you could just press ?)
+  * Configurations haven't changed *except*
+    * an "adapter" option is now mandatory on both source and variations.
+    * gzip option has been removed, if your "file" option ends with ".gz" we assume gzip
+    * a lot of new options have been added, view example_config.yml to see them all
+
+### Fixes
+
+* lots of stuff I guess :)
+
+-------------------
+
+## 2.0.0
+
+* Added option to skip deferred import
+* Added DBS_CFGDIR environment variable to change configuration directory
+* Added Sequel importer
+* Added variation option "file" and "gzip" (will copy the downloaded file in addition to importing)
+* Added variation option "ignore_always" (always excludes given tables)
+* Added mysql dirty importer
+* Make some attempts to catch mysql errors that occur while dumping tables
+* lots of other small fixes/changes
+
+-------------------
+
+## 1.0.1
+
+* Initial release
+
+-------------------
+
+## 1.0.0
+
+* Unreleased

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in db_sucker.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2017 Sven Pachnit, bmonkeys.net
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,193 @@
+# DbSucker
+
+**DbSucker – Sucks DBs as sucking DBs sucks!**
+
+`db_sucker` is an executable which allows you to "suck"/pull remote MySQL (others may follow) databases to your local server.
+You configure your hosts via an YAML configuration in which you can define multiple variations to add constraints on what to dump (and pull).
+
+This tool is meant for pulling live data into your development environment. **It is not designed for backups!** but you might get away with it.
+
+![screenshot](https://i.imgur.com/EAjWrEd.png)
+
+---
+## Alpha product (v3 is a rewrite), use at your own risk, always have a backup!
+---
+
+## Features
+
+* independent parallel dump / download / import cycle for each table
+* verifies file integrity via SHA
+* flashy and colorful curses based interface with keyboard shortcuts
+* more status indications than you would ever want (even more if the remote has a somewhat recent `pv` (pipeviewer) installed)
+* limit concurrency of certain type of tasks (e.g. limit downloads, imports, etc.)
+* uses more threads than any application should ever use (seriously it's a nightmare)
+
+
+## Requirements
+
+Currently `db_sucker` only handles the following data-flow constellation:
+
+  - Remote MySQL -> [SSH] -> local MySQL
+
+On the local side you will need:
+  - unixoid OS
+  - Ruby (>= 2.0, != 2.3.1 see gotchas)
+  - mysql2 gem
+  - MySQL client (`mysql` command will be used for importing)
+
+On the remote side you will need:
+  - unixoid OS
+  - Probably SSH access + sftp subsystem (password and/or keyfile)
+  - any folder with write permissions (for the temporary dumps)
+  - mysqldump executable
+  - MySQL credentials :)
+
+
+## Installation
+
+Simple as:
+
+    $ gem install db_sucker
+
+At the moment you are advised to adjust the MaxSessions limit on your remote SSH server if you run into issues, see Caveats.
+
+You will also need at least one configuration, see Configuration.
+
+
+## Usage
+
+To get a list of available options invoke `db_sucker` with the `--help` or `-h` option:
+
+    Usage: db_sucker [options] [identifier [variation]]
+
+    # Application options
+            --new NAME                   Generates new container config in /Users/chaos/.db_sucker
+        -a, --action ACTION              Dispatch given action
+        -m, --mode MODE                  Dispatch action with given mode
+        -n, --no-deffer                  Don't use deferred import for files > 50 MB SQL data size.
+        -l, --list-databases             List databases for given identifier.
+        -t, --list-tables [DATABASE]     List tables for given identifier and database.
+                                         If used with --list-databases the DATABASE parameter is optional.
+        -o, --only table,table2          Only suck given tables. Identifier is required, variation is optional (defaults to default).
+                                         WARNING: ignores ignore_always option
+        -e, --except table,table2        Don't suck given tables. Identifier is required, variation is optional (defaults to default).
+        -c, --consumers NUM=10           Maximal amount of tasks to run simultaneously
+            --stat-tmp                   Show information about the remote temporary directory.
+                                         If no identifier is given check local temp directory instead.
+            --cleanup-tmp                Remove all temporary files from db_sucker in target directory.
+            --simulate                   To use with --cleanup-tmp to not actually remove anything.
+
+    # General options
+        -d, --debug [lvl=1]              Enable debug output
+            --monochrome                 Don't colorize output (does not apply to curses)
+            --no-window                  Disables curses window alltogether (no progress)
+        -h, --help                       Shows this help
+        -v, --version                    Shows version and other info
+        -z                               Do not check for updates on GitHub (with -v/--version)
+
+    The current config directory is /Users/chaos/.db_sucker
+
+To get a list of available interface options and shortcuts press `?` or type `:help` while the curses interface is running (if you just want to see the help without running a task use `db_sucker -a cloop`).
+
+    Key Bindings (case sensitive):
+
+        ?  shows this help
+        ^  eval prompt (app context, synchronized)
+        L  show latest spooled log entries (no scrolling)
+        P  kill SSH polling (if it stucks)
+        T  create core dump and open in editor
+        q  quit prompt
+        Q  same as ctrl-c
+        :  main prompt
+
+    Main prompt commands:
+
+        :? :h(elp)                      shows this help
+        :q(uit)                         quit prompt
+        :q! :quit!                      same as ctrl-c
+        :kill                           (dirty) interrupts all workers
+        :kill!                          (dirty) essentially SIGKILL (no cleanup)
+        :dump                           create and open coredump
+        :eval       [code]              executes code or opens eval prompt (app context, synchronized)
+        :c(ancel)   <table_name|--all>  cancels given or all workers
+        :p(ause)    <table_name|--all>  pauses given or all workers
+        :r(esume)   <table_name|--all>  resumes given or all workers
+
+## Configuration (for sucking) - YAML format
+
+* Note: The name is just for the filename, how you address it later is defined within the file.
+* Create a new configuration with `db_sucker --new <name>`, the name should optimally consist of `a-z_-`.
+* If `ENV["EDITOR"]` is set, the newly generated config file will be opened with that, i.e. `EDITOR=vim db_sucker --new <name>`.
+* Change the file to your liking and be aware that YAML is indendation sensitive (don't mix spaces with tabs).
+* If you want to DbSucker to ignore a certain configuration, rename it to start with two underscores, e.g. `__foo.yml`.
+* The default destination for configuration files is `~/.db_sucker` (indicated in --help) but can be changed with the `DBS_CFGDIR` enviromental variable.
+
+## Configuration (application) - Ruby format
+
+DbSucker has a lot of settings and other mechanisms which you can tweak and utilize by creating a `~/.db_sucker/config.rb` file. You can change settings, add hooks or define own actions. For more information please take a look at the [documented example config](https://github.com/2called-chaos/db_sucker/blob/master/doc/config_example.rb) and/or [complete list of all settings](https://github.com/2called-chaos/db_sucker/blob/master/lib/db_sucker/application.rb#L58-L129).
+
+
+## Deferred import
+
+Tables with an uncompressed filesize of over 50MB will be queued up for import. Files smaller than 50MB will be imported concurrently with other tables. When all those have finished the large ones will import one after another. You can skip this behaviour with the `-n` resp. `--no-deffer` option. The threshold is changeable in your `config.rb`, see Configuration.
+
+
+## Importer
+
+Currently there is only the "binary" importer which will use the mysql client binary. A [sequel](https://github.com/jeremyevans/sequel) importer has yet to be ported from v2.
+
+* **void10** Used for development/testing. Sleeps for 10 seconds and then exits.
+* **binary** Default import using `mysql` executable
+  * **+dirty** Same as default but the dump will get wrapped:
+    ```
+      (
+        echo "SET AUTOCOMMIT=0;"
+        echo "SET UNIQUE_CHECKS=0;"
+        echo "SET FOREIGN_KEY_CHECKS=0;"
+        cat dumpfile.sql
+        echo "SET FOREIGN_KEY_CHECKS=1;"
+        echo "SET UNIQUE_CHECKS=1;"
+        echo "SET AUTOCOMMIT=1;"
+        echo "COMMIT;"
+      ) | mysql -u... -p... target_database
+    ```
+    **The wrapper will only be used on deferred imports (since it alters global MySQL sever variables)!**
+* **sequel** Not yet implemented
+
+## Caveats  / Bugs
+
+### General
+
+* Ruby 2.3.0 has a bug that might segfault your ruby if some exceptions occur, this is fixed since 2.3.1 and later
+* Consumers that are waiting (e.g. deferred or slot pool) won't release their tasks, if you have to few consumers you might softlock
+
+### SSH errors / MaxSessions
+
+Under certain conditions the program might softlock when the remote unexpectedly closes the SSH connection or stops responding to it (bad packet error). The same might happen when the remote denies a new connection (e.g. to many connections/sessions). If you think it stalled, try `:kill` (semi-clean) or `:kill!` (basically SIGKILL). If you did kill it make sure to run the cleanup task to get rid of potentially big dump files.
+
+**DbSucker typically needs 2 sessions + 1 for each download and you should have some spare for canceling remote processes**
+
+If you get warnings that SSH errors occured (and most likely tasks fail), please do any of the following to prevent the issue:
+
+  * Raise the MaxSession setting on the remote SSHd server if you can (recommended)
+  * Lower the amount of slots for concurrent downloads (see Configuration)
+  * Lower the amount of consumers (not recommended, use slots instead)
+
+You can run basic SSH diagnosis tests with `db_sucker <config_identifier> -a sshdiag`.
+
+## Todo
+
+* Migrate sequel importer from v2
+* Add dirty features again (partial dumps, dumps with SQL constraints)
+* Figure out a way for consumers to release waiting tasks to prevent logical softlocks
+* Optional encrypted HTTP(s) gateway for faster download of files (I can't figure out why Ruby SFTP is soooo slow)
+
+
+## Contributing
+
+1. Fork it ( http://github.com/2called-chaos/db_sucker/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+6. Get a psych, if you understand what I did here you deserve a medal!

data/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

data/VERSION

@@ -0,0 +1 @@
+3.0.0

data/bin/db_sucker

@@ -0,0 +1,12 @@
+#!/usr/bin/env ruby
+require "db_sucker"
+begin
+  STDIN.sync = true
+  STDOUT.sync = true
+
+  Process.setproctitle("db_sucker")
+  DbSucker::Application.dispatch(ENV, ARGV)
+rescue Interrupt
+  puts("\n\nInterrupted")
+  exit 1
+end

data/bin/db_sucker.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# That's __FILE__ in BASH :)
+# From: http://stackoverflow.com/questions/59895/can-a-bash-script-tell-what-directory-its-stored-in
+SOURCE="${BASH_SOURCE[0]}"
+while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
+  MCLDIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
+  SOURCE="$(readlink "$SOURCE")"
+  [[ $SOURCE != /* ]] && SOURCE="$MCLDIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
+done
+PROJECT_ROOT="$( cd -P "$( dirname "$SOURCE" )"/.. && pwd )"
+
+# Actually run script
+cd $PROJECT_ROOT && DBS_DEVELOPER=true bundle exec ruby bin/db_sucker "$@"

data/db_sucker.gemspec

@@ -0,0 +1,29 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'db_sucker/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "db_sucker"
+  spec.version       = DbSucker::VERSION
+  spec.authors       = ["Sven Pachnit"]
+  spec.email         = ["sven@bmonkeys.net"]
+  spec.summary       = %q{Sucks your remote databases via SSH for local tampering.}
+  spec.description   = %q{Suck whole databases, tables and even incremental updates and save your presets for easy reuse.}
+  spec.homepage      = "https://github.com/2called-chaos/db_sucker"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "curses", "~> 1.2"
+  spec.add_dependency "activesupport", ">= 4.1"
+  spec.add_dependency "net-ssh", "~> 4.2"
+  spec.add_dependency "net-sftp", "~> 2.1"
+  spec.add_development_dependency "bundler"
+  spec.add_development_dependency "rake"
+  spec.add_development_dependency "pry"
+  spec.add_development_dependency "pry-remote"
+end

data/doc/config_example.rb

@@ -0,0 +1,53 @@
+# Create this file as ~/.db_sucker/config.rb
+# This file is eval'd in the application object's context after it's initialized!
+
+# Change option defaults (arguments will still override these settings)
+# For all options refer to application.rb#initialize
+#     https://github.com/2called-chaos/db_sucker/blob/master/lib/db_sucker/application.rb
+
+# These are default options, you can clear the whole file if you want.
+
+opts[:debug] = false                    # --debug flag
+opts[:colorize] = true                  # --monochrome flag
+opts[:consumers] = 10                   # amount of workers to run at the same time
+opts[:deferred_threshold] = 50_000_000  # 50 MB
+opts[:status_format] = :full            # used for IO operations, can be one of: none, minimal, full
+opts[:pv_enabled] = true                # disable pv utility autodiscovery (force non-usage)
+
+# used to open core dumps (should be a blocking call, e.g. `subl -w' or `mate -w')
+# MUST be windowed! vim, nano, etc. will not work!
+opts[:core_dump_editor] = "subl -w"
+
+# amount of workers that can use a slot (false = infinite)
+opts[:slot_pools][:all] = false
+opts[:slot_pools][:remote] = false
+opts[:slot_pools][:download] = false
+opts[:slot_pools][:local] = false
+opts[:slot_pools][:import] = 3
+opts[:slot_pools][:deferred] = 1
+
+
+# Add event listeners, there are currently these events with their arguments:
+#   - core_exception(app, exception)
+#   - core_shutdown(app)
+#   - dispatch_before(app, [action || false if not found])
+#   - dispatch_after(app, [action || false if not found], [exception if raised])
+#   - worker_routine_before_all(app, worker)
+#   - worker_routine_before(app, worker, current_routine)
+#   - worker_routine_after(app, worker, current_routine)
+#   - worker_routine_after_all(app, worker)
+#   - prompt_start(app, prompt_label, prompt_options)
+#   - prompt_stop(app, prompt_label)
+
+hook :core_shutdown do |app|
+  puts "We're done! (event listener example in config.rb)"
+end
+
+# Define additional actions that can be invoked using `-a/--action foo`
+# Must start with `dispatch_` to be available.
+def dispatch_foo
+  configful_dispatch(ARGV.shift, ARGV.shift) do |identifier, ctn, variation, var|
+    # execute command on remote and print results
+    puts ctn.blocking_channel_result("lsb_release -a").to_a
+  end
+end

data/doc/container_example.yml

@@ -0,0 +1,150 @@
+# This is the example configuration and contains all settings available and serves also
+# as documentation. All settings that are commented out are default values, all other
+# values should be changed according to you environment but you can still remove some
+# of those (e.g. SSH/MySQL have a few defaults like username, keyfile, etc.).
+
+# The name of the container, this is what you have to type later when invoking `db_sucker`.
+# It must not match the filename (the filename is irrelevant actually).
+my_identifier:
+  # Source contains all information regarding the remote host and database
+  source:
+    ssh:
+      hostname: my.server.net
+
+      ## If left blank use username of current logged in user (SSH default behaviour)
+      username: my_dump_user
+
+      ## If you login via password you must declare it here (prompt will break).
+      ## Consider using key authentication and leave the password blank/remove it.
+      password: my_secret
+
+      ## Can be a string or an array of strings.
+      ## Can be absolute or relative to the _location of this file_ (or start with ~)
+      ## If left blank Net::SSH will attempt to use your ~/.ssh/id_rsa automatically.
+      #keyfile: ~/.ssh/id_rsa
+
+      ## The remote temp directory to place dumpfiles in. Must be writable by the
+      ## SSH user and should be exclusively used for this tool though cleanup only
+      ## removes .dbsc files. The directory MUST EXIST!
+      ## If you want to use a directory relative to your home directory use single
+      ## dot since tilde won't work (e.g. ./my_tmp vs ~/my_tmp)
+      tmp_location: ./db_sucker_tmp
+
+    ## Remote database settings. DON'T limit tables via arguments!
+    adapter: mysql2 # only mysql is supported at the moment
+    hostname: 127.0.0.1
+    username: my_dump_user
+    password: my_secret
+    database: my_database
+    args: --single-transaction # for innoDB
+
+    ## Binaries to be used (normally there is no need to change these)
+    #client_binary: mysql # used to query remote server
+    #dump_binary: mysqldump # used to dump on remote server
+    #gzip_binary: gzip # used to compress file on remote server
+
+    ## SHA type to use for file integrity checking, can be set to "off" to disable feature.
+    ## Should obviously be the same (algorithm) as your variation.
+    #integrity_sha: 512
+
+    ## Binary to generate integrity hashes.
+    ## Note: The sha type (e.g. 1, 128, 512) will be appended!
+    #integrity_binary: shasum -ba
+
+  # Define as many variations here as you want. It is recommended that you always have a "default" variation.
+  variations:
+    # Define your local database settings, args will be passed to the `mysql` command for import.
+    default:
+      adapter: mysql2 # only mysql is supported at the moment
+      database: tomatic
+      hostname: localhost
+      username: root
+      password:
+      args:
+      #client_binary: mysql # used to query/import locally
+
+    # You can inherit all settings from another variation with the `base` setting.
+    # Warning/Note: If you base from a variation that also has a `base` setting it will be resolved,
+    #               be aware of infinite loops, the app won't handle it. If you have a loop the app
+    #               will die with "stack level too deep (SystemStackError)"
+    quick:
+      base: default # <-- copy all settings from $base variation
+      label: "This goes quick, I promise"
+      only: [this_table, and_that_table]
+
+    # You can use `only` or `except` to limit the tables you want to suck but not both at the same time.
+    # There is also an option `ignore_always` for tables which you never want to pull (intended for your default variation)
+    # so that you don't need to repeat them in your except statements (which overwrites your base, no merge)
+    unheavy:
+      base: default
+      except: [orders, order_items, activities]
+      ignore_always: [schema_migrations]
+
+    # You can also copy the downloaded files to a separate directory (this is not a proper backup!?)
+    # - If database is also given perform both operations.
+    # - If file suffix is ".gz" gzip file will be copied, otherwise we copy raw SQL file
+    # - Path will be created in case it doesn't exist (mkdir -p).
+    # - Path must be absolute (or start with ~) any may use the following placeholders:
+    #    :date       2015-08-26
+    #    :time       14-05-54
+    #    :datetime   2015-08-26_14-05-54
+    #    :table      my_table
+    #    :id         981f8e2f278fa7f029399996b02e869ed8fd7709
+    #    :combined   :datetime_-_:table
+    with_copy:
+      base: default
+      file: ~/Desktop/:date_:id_:table.sql.gz # if you would want to store raw SQL omit the ".gz"
+
+    # Only save files, don't import
+    only_copy:
+      base: with_copy
+      database: false
+
+    # You may use a different importer. Currently there are the following:
+    #   binary     Use `dump_binary' executable
+    #   void10     Sleep 10 seconds and do nothing (for testing/development)
+    #
+    # Mysql adapter supports the following flags for binary importer
+    #   +dirty     Use `mysql' executable with dirty speedups (only deferred)
+    dirty:
+      base: default
+      importer: binary
+      importer_flags: +dirty
+
+    # limit data by passing SQL constraints to mysqldump
+    # WARNING:
+    #   If a constraint is in effect...
+    #     - mysqldump options get added `--compact --skip-extended-insert --no-create-info --complete-insert'
+    #     - your local table will not get removed! Only data will get pulled.
+    #     - you can have different columns as long as all remote columns exist locally
+    #     - import will be performed by a ruby implementation (using Sequel)
+    #     - import will ignore records that violate indices (or duplicate IDs)
+    #     - *implied* this will only ADD records, it won't update nor delete existing records!
+    recent_orders:
+      base: default
+      only: [orders, order_items]
+      constraints:
+        # this is some YAML magic (basically a variable)
+        last_week: &last_week "created_at > date_sub(now(), INTERVAL 7 DAY)"
+        orders: *last_week
+        order_items: *last_week
+        # you can use the following to apply a query to all tables which get dumped:
+        __default: *last_week
+
+    # NOT YET IMPLEMENTED!
+    # pseudo-incremental update function:
+    # If you want to pull latest data you can either use constraints like above or use this dirty feature
+    # to make it even more efficient. It will lookup the highest value of a given column per table and only
+    # pulls data with a value greater than that. Typically that column is your ID column.
+    #
+    # WARNING:
+    #   - you will get weird data constellations if you create some data locally and then apply a "delta"
+    #   - records removed on the remote will still be there locally and existing records won't get updated
+    #   - this is just for the lazy and impatient of us!
+    eeekkksss:
+      base: default
+      except: [table_without_id_column]
+      incremental: true
+      incremental_columns: # in case you want to override the `id' column per table (default is id)
+        some_table: custom_id
+        another_table: created_at # don't use dates, use unique IDs!

data/lib/db_sucker/adapters/mysql2.rb

@@ -0,0 +1,103 @@
+module DbSucker
+  module Adapters
+    module Mysql2
+      module Api
+        def self.require_dependencies
+          begin; require "mysql2"; rescue LoadError; end
+        end
+
+        def client_binary
+          source["client_binary"] || "mysql"
+        end
+
+        def local_client_binary
+          data["client_binary"] || "mysql"
+        end
+
+        def dump_binary
+          source["dump_binary"] || "mysqldump"
+        end
+
+        def client_call
+          [].tap do |r|
+            r << "#{client_binary}"
+            r << "-u#{source["username"]}" if source["username"]
+            r << "-p#{source["password"]}" if source["password"]
+            r << "-h#{source["hostname"]}" if source["hostname"]
+          end * " "
+        end
+
+        def local_client_call
+          [].tap do |r|
+            r << "#{local_client_binary}"
+            r << "-u#{data["username"]}" if data["username"]
+            r << "-p#{data["password"]}" if data["password"]
+            r << "-h#{data["hostname"]}" if data["hostname"]
+          end * " "
+        end
+
+        def dump_call
+          [].tap do |r|
+            r << "#{dump_binary}"
+            r << "-u#{source["username"]}" if source["username"]
+            r << "-p#{source["password"]}" if source["password"]
+            r << "-h#{source["hostname"]}" if source["hostname"]
+          end * " "
+        end
+
+        def dump_command_for table
+          [].tap do |r|
+            r << dump_call
+            if c = constraint(table)
+              r << "--compact --skip-extended-insert --no-create-info --complete-insert"
+              r << Shellwords.escape("-w#{c}")
+            end
+            r << source["database"]
+            r << table
+            r << "#{source["args"]}"
+          end * " "
+        end
+
+        def import_instruction_for file, flags = {}
+          {}.tap do |instruction|
+            instruction[:bin] = [local_client_call, data["database"], data["args"]].join(" ")
+            instruction[:file] = file
+            if flags[:dirty] && flags[:deferred]
+              instruction[:file_prepend] = %{
+                  echo "SET AUTOCOMMIT=0;"
+                  echo "SET UNIQUE_CHECKS=0;"
+                  echo "SET FOREIGN_KEY_CHECKS=0;"
+              }
+              instruction[:file_append] = %{
+                  echo "SET FOREIGN_KEY_CHECKS=1;"
+                  echo "SET UNIQUE_CHECKS=1;"
+                  echo "SET AUTOCOMMIT=1;"
+                  echo "COMMIT;"
+              }
+            end
+          end
+        end
+
+        def database_list include_tables = false
+          dbs = blocking_channel_result(%{#{client_call} -N -e 'SHOW DATABASES;'}).for_group(:stdout).join("").split("\n")
+
+          if include_tables
+            dbs.map do |db|
+              [db, table_list(db)]
+            end
+          else
+            dbs
+          end
+        end
+
+        def table_list database
+          blocking_channel_result(%{#{client_call} -N -e 'SHOW FULL TABLES IN #{database};'}).for_group(:stdout).join("").split("\n").map{|r| r.split("\t") }
+        end
+
+        def hostname
+          blocking_channel_result(%{#{client_call} -N -e 'select @@hostname;'}).for_group(:stdout).join("").strip
+        end
+      end
+    end
+  end
+end