snapback 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+--- 
+SHA1: 
+  data.tar.gz: 1a664b28c0d9ab5bc744ee05541062e603d3e637
+  metadata.gz: de56065c29b02b45a8ab80b1f06922c2ad5349b3
+SHA512: 
+  data.tar.gz: d82b6e0c7f95d7f83251da16087b9ca15c8ba0deb3a9762eca97d85944802074b4f4e8a32cff1f7f9e6b246f137a41642dfd6a5cb4d91b94278278a0572006ab
+  metadata.gz: f264844f630a2f7973c99ec5dd7c912af64bae519523110f287888ca260b00407cb56021c7677527a88ade801b5751fc4d97f40a9cafa6821e833aa7e3a151fd

data/README.md

@@ -0,0 +1,173 @@
+# Snapback
+
+Version 0.1 Alpha
+
+Create MySQL snapshots for easy development rollback.
+
+## Project Details
+
+### Use case
+
+Large databases can be cumbersome to reload particularly when testing database migrations. 
+This gem utilises logical volume management (LVM) to create database snapshots, allowing you to rollback to specified point in time.
+
+### Problem domain
+
+This scripts automates the concepts that are discussed in the following articles:
+
+* lullabot.com -- [MySQL Backups Using LVM Snapshots](http://www.lullabot.com/articles/mysql-backups-using-lvm-snapshots)
+* tldp.org -- [Taking a Backup Using Snapshots](http://tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html)
+* mysqlperformanceblog.com -- [Faster Point In Time Recovery with LVM2 Snaphots and Binary Logs](http://www.mysqlperformanceblog.com/2012/02/23/faster-point-in-time-recovery-with-lvm2-snaphots-and-binary-logs/)
+
+If you'd like more information on how to setup LVM on a Linux installation, see:
+
+* howtogeek.com -- [What is Logical Volume Management and How Do You Enable It in Ubuntu?](http://www.howtogeek.com/howto/36568/what-is-logical-volume-management-and-how-do-you-enable-it-in-ubuntu/)
+
+
+### Dependencies
+
+* Linux (tested on Ubuntu 12.04.1 LTS)
+* Sudo shell access
+* Ruby (tested on version 1.8.7)
+* Logical volume management (tested on version 2.02.66(2))
+* MySQL (tested on version 5.5.29-0ubuntu0.12.04.1)
+
+### Installation
+
+    # If the MySQL client libraries aren't already installed, you may need to run this
+    sudo apt-get update
+    sudo apt-get install build-essential
+    sudo apt-get install libmysql-ruby
+    sudo apt-get install libmysqlclient-dev
+
+    # Install the gem
+    sudo gem install snapback
+
+The gem should also automatically install the following gems:
+
+* mysql
+* colorize
+* open4
+* ruby-lvm
+
+You must then enable multiple tablespaces.
+Add a line to the [mysqld] section of your MySQL my.cnf:
+
+[mysqld]
+innodb_file_per_table=ON
+
+### Configuration
+
+If you're on Ubuntu (or other Linux system that uses AppArmor), you may need to allow MySQL to use the mounted directories.
+
+    sudo nano /etc/apparmor.d/usr.sbin/mysqld
+
+Add the following lines to the file:
+
+    /mnt/mysql/ rwk,
+    /mnt/mysql/** rwk,
+
+Restart AppArmor and MySQL
+
+    service apparmor restart
+    service mysql restart
+
+### Usage
+
+Only recommended for use on virtual machines.
+Not recommended for production environments.
+
+## Setup
+
+To start using this application, you must run this once:
+
+    sudo snapback install
+
+This creates the appropriate directories and checks that you have the required programs installed.
+It will also ask you questions about your environment and save your configuration in your home directory.
+
+## Creating a new database
+
+To create a new database, you must specify the following values: 
+
+* The name of the new database
+* The size you expect the database to be (+10% for good measure)
+
+E.g.: Create a database called "camera" which should hold 1G
+
+    sudo snapback create camera --size 1G
+
+## Snapshot an existing database
+
+To make a snapshot, you must specify the following values:
+
+* The name of the database being snapshot
+* The amount you expect the database to grow by (+10% for good measure). Remember that deleting items in the database will cause the database grow (as you're recording changes, not size of files).
+
+E.g.: Snapshot a database called "camera", where the changes will amount to an extra 100MB.
+
+    sudo snapback snapshot camera --size 100M
+
+## Rollback a database to the snapshot
+
+Once you've finished trying to make changes in your database, you can create a snapshot 
+
+E.g.: Rollback the database "camera" back to the date of the snapshot.
+
+    sudo snapback rollback camera
+
+## Drop a database
+
+Once you've finished with the database, you can drop it from MySQL and remove the logical volumes.
+
+E.g.: Drop the database "camera".
+
+    sudo snapback drop camera
+
+## Mount existing devices
+
+If you want to mount a logical volume (e.g.: after a reboot), you can use the mount command.
+
+E.g.: Continue using the database "camera"
+
+    sudo snapback mount camera
+
+## Unmount existing devices
+
+If (for any reason) you want to unmount a logical volume (e.g.: disable a database temporarily), you can use the unmount command.
+
+E.g.: Stop using the database "camera"
+
+    sudo snapback unmount camera
+
+## LVM Issues
+
+If you're getting an error message similar to following:
+
+    File descriptor ? (/home/?/.snapback.yml) leaked on lvcreate invocation.
+
+You can circumvent this error by adding the following environmental variable before your command:
+
+    LVM_SUPPRESS_FD_WARNINGS=1
+
+For example:
+
+    sudo LVM_SUPPRESS_FD_WARNINGS=1 sudo snapback snapshot camera --size 100M
+
+## Licence
+
+Copyright (C) 2013, [Bashkim Isai](http://www.bashkim.com.au)
+
+This script is distributed under the MIT licence.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## Contributors
+
+* @bashaus -- [Bashkim Isai](http://www.bashkim.com.au/)
+
+If you fork this project and create a pull request add your GitHub username, your full name and website to the end of list above.

data/bin/snapback

@@ -0,0 +1,82 @@
+#!/usr/bin/env ruby
+
+require 'rubygems'
+require 'yaml'
+require 'optparse'
+require 'pp'
+
+require 'snapback/options'
+require 'snapback/transaction'
+require 'snapback/database'
+require 'snapback/dsl'
+
+begin
+  # Ensure user is root
+  if Process.uid != 0 then
+    raise "Must run as root"
+  end
+rescue
+  $stderr.puts "#{$!.to_s.red}"
+  exit
+end
+
+begin
+  $options = Snapback::Options.parse(ARGV)
+
+  case $options[:command]
+  when "install"
+    require "snapback/app/install"
+    Snapback::App::Install.instance.go
+
+    puts ""
+    exit
+  end
+
+  begin
+    # Read confirmation options
+    $config = YAML.load_file($options[:config])
+  rescue
+    raise "Could not load configuration file: #{$options[:config]}"
+  end
+
+  # Connect to MySQL
+  $database = Snapback::Database.instance
+  $database.hostname = $config['mysql']['hostname']
+  $database.username = $config['mysql']['username']
+  $database.password = $config['mysql']['password']
+  $database.connect
+
+  case $options[:command]
+  when "create"
+    require "snapback/app/create"
+    Snapback::App::Create.instance.go
+  when "snapshot"
+    require "snapback/app/snapshot"
+    Snapback::App::Snapshot.instance.go
+  when "commit"
+    require "snapback/app/commit"
+    Snapback::App::Commit.instance.go
+  when "rollback"
+    require "snapback/app/rollback"
+    Snapback::App::Rollback.instance.go
+  when "drop"
+    require "snapback/app/drop"
+    Snapback::App::Drop.instance.go
+  when "mount"
+    require "snapback/app/mount"
+    Snapback::App::Mount.instance.go
+  when "unmount"
+    require "snapback/app/unmount"
+    Snapback::App::Unmount.instance.go
+  end
+rescue
+  puts ""
+  $stderr.puts "#{$!.to_s.red}"
+  puts "Use -v (--verbose) to view entire process".red if !$options[:verbose]
+  puts "Rolling back".red
+  puts ""
+
+  Snapback::Transaction.instance.do_rollback
+end
+
+puts ""

data/lib/snapback/app/commit.rb

@@ -0,0 +1,35 @@
+require "singleton"
+
+module Snapback
+  module App
+	  class Commit
+	    include Singleton
+
+	    def go
+        volume_group_name = "#{$config['lvm']['volume_group']}"
+        logical_volume_name = "#{$config['lvm']['prefix_backup']}-#{$options[:database]}"
+
+        # Flush
+        exec_flush
+
+        # Stop the MySQL Server
+        $database.server_stop
+
+        on_rollback lambda {
+          $database.server_start
+        }
+
+        # Remove logical volume
+        run "Committing logical volume", 
+          "lvremove -f /dev/#{volume_group_name}/#{logical_volume_name}"
+
+        # Start the MySQL Server
+        $database.server_start
+
+        on_rollback lambda {
+          $database.server_stop
+        }
+	    end
+	  end
+	end
+end

data/lib/snapback/app/create.rb

@@ -0,0 +1,117 @@
+require "singleton"
+
+module Snapback
+  module App
+    class Create
+      include Singleton
+
+      def go
+        # Ensure we have a size parameter
+        if $options[:size].nil? then
+          raise "You must specify a size attribute. E.g.: -s 500M"
+        end
+
+        volume_group_name = "#{$config['lvm']['volume_group']}"
+        logical_volume_name = "#{$config['lvm']['prefix_database']}-#{$options[:database]}"
+        mount_dir = get_mount_dir $options[:database]
+        mysql_data_dir = $database.get_data_dir
+
+        if !check "Checking database exists #{$options[:database]}", lambda {
+          $database.db_exists?($options[:database])
+        } then
+          check "Creating database '#{$options[:database]}'", lambda {
+            $database.db_create($options[:database])
+          }
+          
+          on_rollback lambda {
+            check "Database '#{$options[:database]}' is being dropped", lambda {
+              $database.db_drop($options[:database])
+            }
+          }
+        end
+
+        exec_flush
+
+        # Stop the MySQL Server
+        $database.server_stop
+
+        on_rollback lambda {
+          $database.server_start
+        }
+
+        # Create a new logical volume (500MB)
+        # lvcreate –L 500MB –n mysql-{dbName} {vgName};
+        run "Create logical volume", 
+          "lvcreate -L #{$options[:size]} -n #{logical_volume_name} #{volume_group_name}"
+
+        on_rollback lambda {
+          run "Removing logical volume",
+            "lvremove -f /dev/#{volume_group_name}/#{logical_volume_name}"
+        }
+
+        # Format the logical volume in ext4 format
+        # mkfs.ext4 /dev/{vgName}/mysql-{dbName};
+        run "Format logical volume filesystem",
+          "mkfs.ext4 /dev/#{volume_group_name}/#{logical_volume_name}"
+
+        # Create a new directory for where the MySQL database can be mounted
+        # mkdir /mnt/mysql/{dbName};
+
+        if !File.directory? mount_dir then
+          run "Make mount directory", 
+            "mkdir #{mount_dir}"
+
+          on_rollback lambda {
+            run "Removing mount directory",
+              "rmdir #{$mount_dir}"
+          }
+
+          run "Changing permissions of mount directory",
+            "chmod 0777 #{mount_dir}"
+        end
+        
+        # # Mount the new logical volume
+        # mount /dev/{vgName}/mysql-{dbName} /mnt/mysql/{dbName};
+        exec_mount "/dev/#{volume_group_name}/#{logical_volume_name}", mount_dir
+
+        # # Move the contents of the database to the new logical volume
+        # mv {mysql-data-dir}/{dbName}/* /mnt/mysql/{dbName};
+
+        move_mysql_files "#{mysql_data_dir}/#{$options[:database]}", mount_dir
+
+        on_rollback lambda {
+          move_mysql_files mount_dir, "#{mysql_data_dir}/#{$options[:database]}"
+        }
+
+        # # Remove the folder in the MySQL data directory
+        # rmdir {mysql-data-dir}/{dbName}/
+        run "Remove mysql database directory",
+          "rm -rf #{mysql_data_dir}/#{$options[:database]}"
+
+        on_rollback lambda {
+          run "Re-creating MySQL database directory",
+            "mkdir #{mysql_data_dir}/#{$options[:database]}"
+
+          exec_chown "#{mysql_data_dir}/#{$options[:database]}"
+        }
+
+        # # Symbolic-link the MySQL data directory to the new logical volume
+        # ln -s /mnt/mysql/{dbName} {mysql-data-dir}/{dbName}/
+        exec_link "#{mysql_data_dir}/#{$options[:database]}", mount_dir
+
+        # # Change the permissions & ownership to MySQL 
+        # chown -R mysql:mysql {mysql-data-dir}/{dbName}/
+        # chown -R mysql:mysql /mnt/mysql/{dbName}/
+        exec_chown "#{mysql_data_dir}/#{$options[:database]}"
+        exec_chown mount_dir
+
+        # Stop the MySQL Server
+        $database.server_start
+
+        on_rollback lambda {
+          $database.server_stop
+        }
+      end
+    end
+  end
+end

data/lib/snapback/app/drop.rb

@@ -0,0 +1,51 @@
+require "singleton"
+require "src/app/commit"
+
+module Snapback
+  module App
+    class Drop
+      include Singleton
+
+      def go
+        volume_group_name = "#{$config['lvm']['volume_group']}"
+        logical_volume_name = "#{$config['lvm']['prefix_database']}-#{$options[:database]}"
+        mount_dir = get_mount_dir $options[:database]
+        mysql_data_dir = $database.get_data_dir
+
+        # Remove the backup
+        Snapback::App::Commit.instance.go
+
+        if !$database.db_exists?($options[:database]) then
+          raise "Database '#{$options[:database]}' does not exist"
+        end
+
+        # Flush
+        exec_flush
+
+        # Stop the MySQL Server
+        $database.server_stop
+
+        on_rollback lambda {
+          $database.server_start
+        }
+
+        # Unlink
+        exec_unlink "#{mysql_data_dir}/#{$options[:database]}", mount_dir
+
+        # Unmount
+        exec_unmount "/dev/#{volume_group_name}/#{logical_volume_name}", mount_dir
+
+        # Remove logical volume
+        run "Remove logical volume", 
+          "lvremove -f /dev/#{volume_group_name}/#{logical_volume_name}"
+
+        # Start the MySQL Server
+        $database.server_start
+
+        on_rollback lambda {
+          $database.server_stop
+        }
+      end
+    end
+  end
+end