cfbackup 0.7.1
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +6 -0
- data/CHANGELOG.markdown +45 -0
- data/LICENSE +14 -0
- data/README.markdown +63 -0
- data/Rakefile +64 -0
- data/VERSION.yml +4 -0
- data/bin/cfbackup +6 -0
- data/cfbackup.gemspec +66 -0
- data/cfconfig.yml +2 -0
- data/conf/cfconfig.yml +2 -0
- data/example_scripts/piped.sh +17 -0
- data/example_scripts/temp_directory.sh +24 -0
- data/lib/cfbackup.rb +306 -0
- data/lib/optcfbackup.rb +86 -0
- data/temp/README +5 -0
- data/test/cfbackup_test.rb +107 -0
- data/test/cfconfig.yml +2 -0
- data/test/data/data.txt +11 -0
- data/test/data/folder_1/file1.txt +2 -0
- data/test/data/folder_1/file2.txt +2 -0
- data/test/data/folder_1/folder_3/file1.txt +2 -0
- data/test/data/folder_2/file1.txt +2 -0
- data/test/data/folder_2/file2.txt +2 -0
- data/test/test_helper.rb +10 -0
- metadata +89 -0
data/.gitignore
ADDED
data/CHANGELOG.markdown
ADDED
@@ -0,0 +1,45 @@
|
|
1
|
+
CFBackup ChangeLog
|
2
|
+
==================
|
3
|
+
|
4
|
+
0.7.1 2009-05-19
|
5
|
+
-----------------
|
6
|
+
* Version bump and re-release of 0.6.1 because the published gem by GitHub was marked as 0.7.0.
|
7
|
+
|
8
|
+
0.6.1 2009-05-19
|
9
|
+
-----------------
|
10
|
+
* Use official Rackspace CloudFiles API from GitHub
|
11
|
+
* Corrected case issue on case-sensitive filesystems
|
12
|
+
|
13
|
+
0.6.0 2009-05-03
|
14
|
+
-----------------
|
15
|
+
* Added example_scripts to RDoc.
|
16
|
+
* gem creates cfconfig.yml in /etc for reference
|
17
|
+
* CFBackup looks in multiple places for config file
|
18
|
+
* Hidden directory in $HOME ($HOME/.cfconfig.yml)
|
19
|
+
* Non-hidden in current directory (./cfconfig.yml)
|
20
|
+
* In /etc/cfconfig.yml
|
21
|
+
* Refactoring and new usage with --action push|pull|delete
|
22
|
+
* Pulling files implemented
|
23
|
+
* Deleting files implemented
|
24
|
+
* Initial unit tests completed
|
25
|
+
|
26
|
+
0.5.0 2009-04-18
|
27
|
+
-----------------
|
28
|
+
|
29
|
+
* Conversion to a Ruby Gem
|
30
|
+
* cfbackup put in bin so that it can be called from anywhere
|
31
|
+
* Looks in a few standard locations for the config file
|
32
|
+
* Unit test framework prepped
|
33
|
+
* Cloud Files API has been moved to a separate repository and made available as a gem
|
34
|
+
* Most doc files converted to markdown
|
35
|
+
|
36
|
+
0.0.4 2009-04-11
|
37
|
+
-----------------
|
38
|
+
|
39
|
+
* Pipe data straight to container
|
40
|
+
* Specify remote path or file name
|
41
|
+
|
42
|
+
0.0.3 2009-04-08
|
43
|
+
-------------------
|
44
|
+
|
45
|
+
* Initial release to public
|
data/LICENSE
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
Copyright (C) 2009 Jon Stacey
|
2
|
+
|
3
|
+
This program is free software: you can redistribute it and/or modify
|
4
|
+
it under the terms of the GNU General Public License as published by
|
5
|
+
the Free Software Foundation, either version 3 of the License, or
|
6
|
+
(at your option) any later version.
|
7
|
+
|
8
|
+
This program is distributed in the hope that it will be useful,
|
9
|
+
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
10
|
+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
11
|
+
GNU General Public License for more details.
|
12
|
+
|
13
|
+
You should have received a copy of the GNU General Public License
|
14
|
+
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
data/README.markdown
ADDED
@@ -0,0 +1,63 @@
|
|
1
|
+
CFBackup
|
2
|
+
=========
|
3
|
+
|
4
|
+
CFBackup is a small ruby program that transfers files or directories from the local machine to a Cloud Files container. It is meant to serve as a useful tool for automated backups.
|
5
|
+
|
6
|
+
Features
|
7
|
+
-----------
|
8
|
+
|
9
|
+
* Push (backup) a single file or directory (uses pseudo directories)
|
10
|
+
* Pull (restore) a single file or directory
|
11
|
+
* Delete (rotate) remote objects
|
12
|
+
* Pipe data straight to container
|
13
|
+
* Free transfers over local Rackspace network for Slicehost/Cloud Server
|
14
|
+
customers in DFW1 datacenter
|
15
|
+
|
16
|
+
Requirements
|
17
|
+
--------------
|
18
|
+
|
19
|
+
* ruby-cloudfiles
|
20
|
+
|
21
|
+
Notes:
|
22
|
+
* If you install CFBackup as a gem, all of the dependencies _should_ automatically be installed for you.
|
23
|
+
* Ubuntu Users: The Ubuntu rubygems package will installs executables outside your normal PATH. You will
|
24
|
+
need to update it or create a symlink to access cfbackup from anywhere. See the wiki for more information.reating a symlink.
|
25
|
+
|
26
|
+
Install
|
27
|
+
-----------
|
28
|
+
|
29
|
+
* gem sources -a http://gems.github.com
|
30
|
+
* sudo gem install jmstacey-cfbackup
|
31
|
+
|
32
|
+
Configuration
|
33
|
+
-----------
|
34
|
+
|
35
|
+
A sample configuration file named cfconfig.yml should have be placed in /etc if installed as a gem.
|
36
|
+
CFBackup will look in the following places (in order) for the configuration file named cfconfig.yml:
|
37
|
+
|
38
|
+
* Hidden in home directory (~/.cfbackup.yml)
|
39
|
+
* Non-hidden in present working directory (./cfbackup.yml)
|
40
|
+
* In etc (/etc/cfbackup.yml)
|
41
|
+
|
42
|
+
The configuration file can be overridden at any time with the --config_file option
|
43
|
+
|
44
|
+
Configuration
|
45
|
+
-----------
|
46
|
+
|
47
|
+
Usage: cfbackup.rb --action push|pull|delete options --container CONTAINER
|
48
|
+
--action ACTION Action to perform: push, pull, or delete.
|
49
|
+
--pipe_data Pipe data from another application and stream it to Cloud Files
|
50
|
+
-r, --recursive Traverse local path recursivley
|
51
|
+
-v, --verbose Run verbosely
|
52
|
+
--local_path LOCAL_PATH Local path or file
|
53
|
+
--container CONTAINER Cloud Files container name
|
54
|
+
--version Show current version
|
55
|
+
--config_file PATH Use specified config file, rather than the default
|
56
|
+
--local_net Use unmetered connection in DFW1 (only applicable to Slicehost or Mosso Cloud Server customers)
|
57
|
+
|
58
|
+
The wiki has usage examples and some sample automation scripts can be found in the example_scripts directory.
|
59
|
+
|
60
|
+
Copyright
|
61
|
+
------------
|
62
|
+
|
63
|
+
Copyright (c) 2009 Jon Stacey. See LICENSE for details.
|
data/Rakefile
ADDED
@@ -0,0 +1,64 @@
|
|
1
|
+
require 'rake'
|
2
|
+
|
3
|
+
$LOAD_PATH.unshift('lib')
|
4
|
+
|
5
|
+
begin
|
6
|
+
require 'jeweler'
|
7
|
+
Jeweler::Tasks.new do |gem|
|
8
|
+
gem.name = "cfbackup"
|
9
|
+
gem.summary = "A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files."
|
10
|
+
gem.description = "A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files."
|
11
|
+
gem.email = "jon@jonsview.com"
|
12
|
+
gem.homepage = "http://github.com/jmstacey/cfbackup"
|
13
|
+
gem.authors = ["Jon Stacey"]
|
14
|
+
|
15
|
+
# Dependencies
|
16
|
+
gem.add_dependency('rackspace-cloudfiles', '>=1.3.0.3')
|
17
|
+
|
18
|
+
# Include Files
|
19
|
+
gem.files.include %w(lib/optcfbackup.rb)
|
20
|
+
|
21
|
+
# gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
|
22
|
+
end
|
23
|
+
rescue LoadError
|
24
|
+
puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
|
25
|
+
end
|
26
|
+
|
27
|
+
require 'rake/testtask'
|
28
|
+
Rake::TestTask.new(:test) do |test|
|
29
|
+
test.libs << 'lib' << 'test'
|
30
|
+
test.pattern = 'test/**/*_test.rb'
|
31
|
+
test.verbose = true
|
32
|
+
end
|
33
|
+
|
34
|
+
begin
|
35
|
+
require 'rcov/rcovtask'
|
36
|
+
Rcov::RcovTask.new do |test|
|
37
|
+
test.libs << 'test'
|
38
|
+
test.pattern = 'test/**/*_test.rb'
|
39
|
+
test.verbose = true
|
40
|
+
end
|
41
|
+
rescue LoadError
|
42
|
+
task :rcov do
|
43
|
+
abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
task :default => :test
|
48
|
+
|
49
|
+
require 'rake/rdoctask'
|
50
|
+
Rake::RDocTask.new do |rdoc|
|
51
|
+
if File.exist?('VERSION.yml')
|
52
|
+
config = YAML.load(File.read('VERSION.yml'))
|
53
|
+
version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
|
54
|
+
else
|
55
|
+
version = ""
|
56
|
+
end
|
57
|
+
|
58
|
+
rdoc.rdoc_dir = 'rdoc'
|
59
|
+
rdoc.title = "cfbackup #{version}"
|
60
|
+
rdoc.rdoc_files.include('README*')
|
61
|
+
rdoc.rdoc_files.include('example_scripts/**')
|
62
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
63
|
+
end
|
64
|
+
|
data/VERSION.yml
ADDED
data/bin/cfbackup
ADDED
data/cfbackup.gemspec
ADDED
@@ -0,0 +1,66 @@
|
|
1
|
+
Gem::Specification.new do |s|
|
2
|
+
s.name = %q{cfbackup}
|
3
|
+
s.version = "0.7.1"
|
4
|
+
|
5
|
+
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
6
|
+
s.authors = ["Jon Stacey"]
|
7
|
+
s.date = %q{2009-05-19}
|
8
|
+
s.default_executable = %q{cfbackup}
|
9
|
+
s.description = %q{A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.}
|
10
|
+
s.email = %q{jon@jonsview.com}
|
11
|
+
s.executables = ["cfbackup"]
|
12
|
+
s.extra_rdoc_files = [
|
13
|
+
"LICENSE",
|
14
|
+
"README.markdown"
|
15
|
+
]
|
16
|
+
s.files = [
|
17
|
+
".gitignore",
|
18
|
+
"CHANGELOG.markdown",
|
19
|
+
"LICENSE",
|
20
|
+
"README.markdown",
|
21
|
+
"Rakefile",
|
22
|
+
"VERSION.yml",
|
23
|
+
"bin/cfbackup",
|
24
|
+
"cfbackup.gemspec",
|
25
|
+
"cfconfig.yml",
|
26
|
+
"conf/cfconfig.yml",
|
27
|
+
"example_scripts/piped.sh",
|
28
|
+
"example_scripts/temp_directory.sh",
|
29
|
+
"lib/cfbackup.rb",
|
30
|
+
"lib/optcfbackup.rb",
|
31
|
+
"lib/optcfbackup.rb",
|
32
|
+
"temp/README",
|
33
|
+
"test/cfbackup_test.rb",
|
34
|
+
"test/cfconfig.yml",
|
35
|
+
"test/data/data.txt",
|
36
|
+
"test/data/folder_1/file1.txt",
|
37
|
+
"test/data/folder_1/file2.txt",
|
38
|
+
"test/data/folder_1/folder_3/file1.txt",
|
39
|
+
"test/data/folder_2/file1.txt",
|
40
|
+
"test/data/folder_2/file2.txt",
|
41
|
+
"test/test_helper.rb"
|
42
|
+
]
|
43
|
+
s.has_rdoc = true
|
44
|
+
s.homepage = %q{http://github.com/jmstacey/cfbackup}
|
45
|
+
s.rdoc_options = ["--charset=UTF-8"]
|
46
|
+
s.require_paths = ["lib"]
|
47
|
+
s.rubygems_version = %q{1.2.0}
|
48
|
+
s.summary = %q{A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.}
|
49
|
+
s.test_files = [
|
50
|
+
"test/cfbackup_test.rb",
|
51
|
+
"test/test_helper.rb"
|
52
|
+
]
|
53
|
+
|
54
|
+
if s.respond_to? :specification_version then
|
55
|
+
current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
|
56
|
+
s.specification_version = 2
|
57
|
+
|
58
|
+
if current_version >= 3 then
|
59
|
+
s.add_runtime_dependency(%q<rackspace-cloudfiles>, [">= 1.3.0.3"])
|
60
|
+
else
|
61
|
+
s.add_dependency(%q<rackspace-cloudfiles>, [">= 1.3.0.3"])
|
62
|
+
end
|
63
|
+
else
|
64
|
+
s.add_dependency(%q<rackspace-cloudfiles>, [">= 1.3.0.3"])
|
65
|
+
end
|
66
|
+
end
|
data/cfconfig.yml
ADDED
data/conf/cfconfig.yml
ADDED
@@ -0,0 +1,17 @@
|
|
1
|
+
#!/bin/sh
|
2
|
+
|
3
|
+
# In this example, data is piped directly from the backup to a Cloud Files
|
4
|
+
# container, bypassing the intermediate temp directory step
|
5
|
+
#
|
6
|
+
# This is an example script that you could use in combination with CFBackup.
|
7
|
+
# Modify it to suit your needs. Rename to file without an extension if you
|
8
|
+
# are going to place in /etc/cron.daily.
|
9
|
+
|
10
|
+
CONTAINER=backups
|
11
|
+
NOW=$(date +_%b_%d_%y)
|
12
|
+
|
13
|
+
# Backup all MySQL databases
|
14
|
+
mysqldump -u root -pPASSWORD --all-databases --flush-logs --lock-all-tables | gzip -9 | cfbackup --action push --pipe_data --container $CONTAINER:mysql_all_backup$NOW.sql.gz
|
15
|
+
|
16
|
+
# Create main backup archive
|
17
|
+
tar -cvf - /home/user /etc /usr/local/nginx | gzip -9 | cfbackup --action push --pipe_data --container $CONTAINER:main_backup$NOW.tar.gz
|
@@ -0,0 +1,24 @@
|
|
1
|
+
#!/bin/sh
|
2
|
+
|
3
|
+
# In this example, backups are first created and placed in a temporary
|
4
|
+
# holding directory. Then, the entire directory is uploaded to
|
5
|
+
# Cloud Files. Finally, the temp directory is cleared in preparation
|
6
|
+
# for the next backup.
|
7
|
+
#
|
8
|
+
# This is an example script that you could use in combination with CFBackup.
|
9
|
+
# Modify it to suit your needs. Rename to file without an extension if you
|
10
|
+
# are going to place in /etc/cron.daily.
|
11
|
+
|
12
|
+
cd ~/cfbackup/temp
|
13
|
+
CONTAINER=backups
|
14
|
+
NOW=$(date +_%b_%d_%y)
|
15
|
+
|
16
|
+
# Dump all MySQL databases
|
17
|
+
mysqldump -u root -pPASSWORD --all-databases --flush-logs --lock-all-tables | gzip -9 > mysql_all_backup$NOW.sql.gz
|
18
|
+
|
19
|
+
# Create main backup archive
|
20
|
+
tar -czvf main_backup$NOW.tar.gz /home/user /etc /usr/local/nginx
|
21
|
+
|
22
|
+
cd ~/cfbackup
|
23
|
+
cfbackup --action push --local_path temp/ --container $CONTAINER
|
24
|
+
rm -f temp/*
|
data/lib/cfbackup.rb
ADDED
@@ -0,0 +1,306 @@
|
|
1
|
+
# CFBackup, a small utility script to backup files to Mosso Cloud Files
|
2
|
+
# Copyright (C) 2009 Jon Stacey
|
3
|
+
#
|
4
|
+
# This program is free software: you can redistribute it and/or modify
|
5
|
+
# it under the terms of the GNU General Public License as published by
|
6
|
+
# the Free Software Foundation, either version 3 of the License, or
|
7
|
+
# (at your option) any later version.
|
8
|
+
#
|
9
|
+
# This program is distributed in the hope that it will be useful,
|
10
|
+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
11
|
+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
12
|
+
# GNU General Public License for more details.
|
13
|
+
#
|
14
|
+
# You should have received a copy of the GNU General Public License
|
15
|
+
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
16
|
+
|
17
|
+
require 'rubygems'
|
18
|
+
require 'ftools'
|
19
|
+
require 'cloudfiles'
|
20
|
+
require 'optcfbackup'
|
21
|
+
require 'yaml'
|
22
|
+
|
23
|
+
class CFBackup
|
24
|
+
|
25
|
+
def initialize(args)
|
26
|
+
@opts = OptCFBackup.new(args)
|
27
|
+
|
28
|
+
# Special case if the version is requested
|
29
|
+
if @opts.options.show_ver
|
30
|
+
version_file = File.join(File.dirname(__FILE__), '..', 'VERSION.yml')
|
31
|
+
if File.exist?(version_file)
|
32
|
+
config = YAML.load(File.read(version_file))
|
33
|
+
version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
|
34
|
+
else
|
35
|
+
version = "unkown version"
|
36
|
+
end
|
37
|
+
show_error("CFBackup #{version}")
|
38
|
+
end
|
39
|
+
|
40
|
+
# Locate and load config file
|
41
|
+
@opts.options.config.each do |path|
|
42
|
+
if (File.exist?(path))
|
43
|
+
@conf = YAML::load(File.open(path))
|
44
|
+
break
|
45
|
+
end
|
46
|
+
end
|
47
|
+
show_error('Error: Unable to locate config file.') unless (@conf != nil)
|
48
|
+
|
49
|
+
prep_connection
|
50
|
+
|
51
|
+
end # initialize()
|
52
|
+
|
53
|
+
def run
|
54
|
+
|
55
|
+
show_error() unless (@opts.options.container != "")
|
56
|
+
|
57
|
+
# Run appropriate action
|
58
|
+
case @opts.options.action
|
59
|
+
when 'push'
|
60
|
+
if @opts.options.pipe_data
|
61
|
+
push_piped_data
|
62
|
+
else
|
63
|
+
push_files
|
64
|
+
end
|
65
|
+
when 'pull'
|
66
|
+
pull_files()
|
67
|
+
when 'delete'
|
68
|
+
delete_files
|
69
|
+
else
|
70
|
+
show_error()
|
71
|
+
end
|
72
|
+
|
73
|
+
end # run()
|
74
|
+
|
75
|
+
private
|
76
|
+
|
77
|
+
def prep_connection
|
78
|
+
# Establish connection
|
79
|
+
show_verbose "Establishing connection...", false
|
80
|
+
@cf = CloudFiles::Connection.new(@conf["username"], @conf["api_key"]);
|
81
|
+
show_verbose " done."
|
82
|
+
|
83
|
+
# Special option for Slicehost customers in DFW datacenter
|
84
|
+
if @opts.options.local_net
|
85
|
+
@cf.storagehost = 'snet-storage.clouddrive.com'
|
86
|
+
end
|
87
|
+
end # prep_connection()
|
88
|
+
|
89
|
+
def prep_container(create_container = true)
|
90
|
+
# Check for the container. If it doesn't exist, create it if allowed
|
91
|
+
if !@cf.container_exists?(@opts.options.container)
|
92
|
+
if create_container
|
93
|
+
show_verbose "Container '#{@opts.options.container}' does not exist. Creating it...", false
|
94
|
+
@cf.create_container(@opts.options.container)
|
95
|
+
show_verbose " done."
|
96
|
+
else
|
97
|
+
show_error("Error: Container '#{@opts.options.container}' does not exist.")
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
@container = @cf.container(@opts.options.container)
|
102
|
+
end # prep_cnnection()
|
103
|
+
|
104
|
+
def push_piped_data
|
105
|
+
prep_container
|
106
|
+
|
107
|
+
puts "Warning: 5GB maximum filesize"
|
108
|
+
object = @container.create_object(@opts.options.remote_path, true)
|
109
|
+
object.write
|
110
|
+
end # push_piped_data()
|
111
|
+
|
112
|
+
def push_files
|
113
|
+
prep_container
|
114
|
+
|
115
|
+
path = @opts.options.local_path
|
116
|
+
pwd = Dir.getwd # Save current directory so we can come back
|
117
|
+
|
118
|
+
if FileTest::file?(path)
|
119
|
+
glob_options = File.join(File::basename(path))
|
120
|
+
elsif @opts.options.recursive
|
121
|
+
Dir.chdir(path)
|
122
|
+
glob_options = File.join("**", "*")
|
123
|
+
else
|
124
|
+
Dir.chdir(path)
|
125
|
+
glob_options = File.join("*")
|
126
|
+
end
|
127
|
+
files = Dir.glob(glob_options)
|
128
|
+
|
129
|
+
# Upload file(s)
|
130
|
+
counter = 1
|
131
|
+
show_verbose "There are #{files.length} files to process."
|
132
|
+
files.each do |file|
|
133
|
+
file = file.sub(/\.\//, '')
|
134
|
+
if file == "" || file[0,1] == "." || FileTest.directory?(file)
|
135
|
+
show_verbose "(#{counter}/#{files.length}) Skipping #{file}"
|
136
|
+
counter += 1
|
137
|
+
next
|
138
|
+
end
|
139
|
+
|
140
|
+
show_verbose "(#{counter}/#{files.length}) Pushing file #{file}...", false
|
141
|
+
|
142
|
+
if @opts.options.remote_path.to_s == ''
|
143
|
+
remote_path = file
|
144
|
+
else
|
145
|
+
remote_path = File.join(@opts.options.remote_path, file)
|
146
|
+
end
|
147
|
+
|
148
|
+
object = @container.create_object(remote_path, true)
|
149
|
+
object.load_from_filename(file)
|
150
|
+
|
151
|
+
show_verbose " done."
|
152
|
+
counter += 1
|
153
|
+
end # files.each
|
154
|
+
|
155
|
+
Dir.chdir(pwd) # Go back to original directory
|
156
|
+
|
157
|
+
end # push_files()
|
158
|
+
|
159
|
+
def pull_files
|
160
|
+
prep_container(false)
|
161
|
+
|
162
|
+
file = object_file?
|
163
|
+
objects = get_objects_array
|
164
|
+
|
165
|
+
# Process objects
|
166
|
+
counter = 1
|
167
|
+
show_verbose "There are #{objects.length} objects to process."
|
168
|
+
objects.each do |object_name|
|
169
|
+
object = @container.object(object_name)
|
170
|
+
if object.content_type == "application/directory"
|
171
|
+
show_verbose "(#{counter}/#{objects.length}) Pseudo directory #{object.name}"
|
172
|
+
counter += 1
|
173
|
+
next
|
174
|
+
end
|
175
|
+
|
176
|
+
path_info = File.split(@opts.options.local_path.to_s)
|
177
|
+
file_info = File.split(object.name.to_s)
|
178
|
+
|
179
|
+
if file # Dealing with a single file pull
|
180
|
+
if @opts.options.local_path.to_s == ''
|
181
|
+
filepath = file_info[1].to_s # Use current directory and original name
|
182
|
+
else
|
183
|
+
if File.exist?(@opts.options.local_path.to_s)
|
184
|
+
# The file exists, so we will overwrite it
|
185
|
+
filepath = File.join(@opts.options.local_path.to_s)
|
186
|
+
else
|
187
|
+
# If the file doesn't exist, a new name may have been given.
|
188
|
+
# Test the path.
|
189
|
+
if File.exist?(path_info[0])
|
190
|
+
# A new name was given with a valid path
|
191
|
+
filepath = File.join(path_info[0], path_info[1])
|
192
|
+
else
|
193
|
+
# The given path is not valid
|
194
|
+
show_error("cfbackup: #{file_info[0]}: No such file or directory.")
|
195
|
+
end
|
196
|
+
end
|
197
|
+
end
|
198
|
+
else # Dealing with a multi-object pull
|
199
|
+
if @opts.options.local_path.to_s == ''
|
200
|
+
filepath = object.name.to_s # Use current directory with object name
|
201
|
+
dir_path = file_info[0]
|
202
|
+
else
|
203
|
+
if File.directory?(@opts.options.local_path.to_s)
|
204
|
+
filepath = File.join(@opts.options.local_path.to_s, object.name.to_s)
|
205
|
+
dir_path = File.join(@opts.options.local_path, file_info[0])
|
206
|
+
else
|
207
|
+
# We can't copy a directory to a file...
|
208
|
+
show_error("cfbackup: #{@container.name}:#{@opts.options.remote_path.to_s}/ is a directory (not copied).")
|
209
|
+
end
|
210
|
+
end
|
211
|
+
File.makedirs(dir_path) # Create subdirectories as needed
|
212
|
+
end
|
213
|
+
|
214
|
+
show_verbose "(#{counter}/#{objects.length}) Pulling object #{object.name}...", false
|
215
|
+
object.save_to_filename(filepath)
|
216
|
+
show_verbose " done"
|
217
|
+
counter += 1
|
218
|
+
end
|
219
|
+
end # pull_files()
|
220
|
+
|
221
|
+
def delete_files
|
222
|
+
prep_container(false)
|
223
|
+
|
224
|
+
if object_file?
|
225
|
+
show_verbose "Deleting object #{@opts.options.remote_path.to_s}"
|
226
|
+
@container.delete_object(@opts.options.remote_path.to_s)
|
227
|
+
else
|
228
|
+
if !@opts.options.recursive
|
229
|
+
show_error("Error: #{@opts.options.remote_path} is a directory but the the recursive option was not specified. Doing nothing.")
|
230
|
+
else
|
231
|
+
objects = get_objects_array
|
232
|
+
|
233
|
+
# Process objects
|
234
|
+
counter = 1
|
235
|
+
show_verbose "There are #{objects.length} objects to process."
|
236
|
+
objects.each do |object_name|
|
237
|
+
show_verbose "(#{counter}/#{objects.length}) Deleting object #{object_name}...", false
|
238
|
+
@container.delete_object(object_name)
|
239
|
+
show_verbose " done"
|
240
|
+
counter += 1
|
241
|
+
end
|
242
|
+
end
|
243
|
+
end
|
244
|
+
|
245
|
+
end # delete_files()
|
246
|
+
|
247
|
+
|
248
|
+
# Helper methods below
|
249
|
+
|
250
|
+
def object_file?
|
251
|
+
|
252
|
+
file = false
|
253
|
+
unless @opts.options.remote_path.to_s == ''
|
254
|
+
if @container.object_exists?(@opts.options.remote_path)
|
255
|
+
if @container.object(@opts.options.remote_path).content_type != "application/directory"
|
256
|
+
file = true
|
257
|
+
if @opts.options.recursive
|
258
|
+
puts "Warning: This is a file so the recursive option is meaningless."
|
259
|
+
end
|
260
|
+
end
|
261
|
+
else
|
262
|
+
show_error("The object #{@opts.options.remote_path} does not exist")
|
263
|
+
end
|
264
|
+
end
|
265
|
+
|
266
|
+
return file
|
267
|
+
end
|
268
|
+
|
269
|
+
def get_objects_array
|
270
|
+
|
271
|
+
file = object_file?
|
272
|
+
|
273
|
+
# Get array of objects to process
|
274
|
+
objects = Array.new
|
275
|
+
if file
|
276
|
+
objects << @opts.options.remote_path.to_s
|
277
|
+
elsif @opts.options.recursive
|
278
|
+
# Use prefix instead of path so that "subdirectories" are included
|
279
|
+
objects = @container.objects(:prefix => @opts.options.remote_path)
|
280
|
+
else
|
281
|
+
objects = @container.objects(:path => @opts.options.remote_path)
|
282
|
+
end
|
283
|
+
|
284
|
+
return objects
|
285
|
+
end
|
286
|
+
|
287
|
+
# Shows given message if verbose output is turned on
|
288
|
+
def show_verbose(message, line_break = true)
|
289
|
+
unless !@opts.options.verbose
|
290
|
+
if line_break
|
291
|
+
puts message
|
292
|
+
else
|
293
|
+
print message
|
294
|
+
end
|
295
|
+
$stdout.flush
|
296
|
+
end
|
297
|
+
end # show_verbose()
|
298
|
+
|
299
|
+
# Show error message, banner and exit
|
300
|
+
def show_error(message = '')
|
301
|
+
puts message
|
302
|
+
puts @opts.banner
|
303
|
+
exit
|
304
|
+
end # show_error()
|
305
|
+
|
306
|
+
end # class CFBackup
|
data/lib/optcfbackup.rb
ADDED
@@ -0,0 +1,86 @@
|
|
1
|
+
require 'optparse'
|
2
|
+
require 'ostruct'
|
3
|
+
|
4
|
+
class OptCFBackup
|
5
|
+
|
6
|
+
# Options structure
|
7
|
+
attr_reader :options
|
8
|
+
|
9
|
+
# Ussage message
|
10
|
+
attr_reader :banner
|
11
|
+
|
12
|
+
# Initializes object with command line arguments passed
|
13
|
+
def initialize(args)
|
14
|
+
|
15
|
+
@banner = "Usage: cfbackup.rb --action push|pull|delete options --container CONTAINER"
|
16
|
+
|
17
|
+
@options = OpenStruct.new
|
18
|
+
self.options.config = ["#{ENV['HOME']}/.cfconfig.yml", './cfconfig.yml', '/etc/cfconfig.yml']
|
19
|
+
self.options.action = ''
|
20
|
+
self.options.pipe_data = false
|
21
|
+
self.options.show_ver = false
|
22
|
+
self.options.recursive = false
|
23
|
+
self.options.local_net = false
|
24
|
+
self.options.container = ''
|
25
|
+
self.options.local_path = ''
|
26
|
+
self.options.remote_path = ''
|
27
|
+
self.options.verbose = false;
|
28
|
+
|
29
|
+
opts = OptionParser.new do |opts|
|
30
|
+
opts.banner = self.banner
|
31
|
+
|
32
|
+
opts.on("--action ACTION", "Action to perform: push, pull, or delete.") do |action|
|
33
|
+
self.options.action = action
|
34
|
+
end
|
35
|
+
|
36
|
+
opts.on("--pipe_data", "Pipe data from another application and stream it to Cloud Files") do |pipe|
|
37
|
+
self.options.pipe_data = pipe
|
38
|
+
end
|
39
|
+
|
40
|
+
opts.on("-r", "--recursive", "Traverse local path recursivley") do |recursive|
|
41
|
+
self.options.recursive = recursive
|
42
|
+
end
|
43
|
+
|
44
|
+
opts.on("-v", "--verbose", "Run verbosely") do |verbose|
|
45
|
+
self.options.verbose = verbose
|
46
|
+
end
|
47
|
+
|
48
|
+
opts.on("--local_path LOCAL_PATH", "Local path or file") do |local_path|
|
49
|
+
self.options.local_path = local_path
|
50
|
+
end
|
51
|
+
|
52
|
+
opts.on("--container CONTAINER", "Cloud Files container name") do |name|
|
53
|
+
self.options.container, self.options.remote_path = name.split(":", 2)
|
54
|
+
clean_remote_path unless (self.options.remote_path.nil?)
|
55
|
+
end
|
56
|
+
|
57
|
+
opts.on("--version", "Show current version") do |version|
|
58
|
+
self.options.show_ver = version
|
59
|
+
end
|
60
|
+
|
61
|
+
opts.on("--config_file PATH", "Use specified config file, rather than the default") do |config|
|
62
|
+
self.options.config = Array.new()
|
63
|
+
self.options.config << config
|
64
|
+
end
|
65
|
+
|
66
|
+
opts.on("--local_net", "Use unmetered connection in DFW1 (only applicable to Slicehost or Mosso Cloud Server customers)") do |local_net|
|
67
|
+
self.options.local_net = local_net
|
68
|
+
end
|
69
|
+
|
70
|
+
end
|
71
|
+
|
72
|
+
opts.parse!(args)
|
73
|
+
|
74
|
+
end # initialize()
|
75
|
+
|
76
|
+
private
|
77
|
+
|
78
|
+
def clean_remote_path
|
79
|
+
if self.options.remote_path[0,1] == "/"
|
80
|
+
self.options.remote_path.slice!(0)
|
81
|
+
end
|
82
|
+
# Follwoig won't work for piped data. Might result in "text.txt/"
|
83
|
+
# self.options.remote_path = self.options.remote_path + "/" unless (self.options.remote_path[-1,1] == "/")
|
84
|
+
end
|
85
|
+
|
86
|
+
end # class OptCFBackup
|
data/temp/README
ADDED
@@ -0,0 +1,107 @@
|
|
1
|
+
require 'test_helper'
|
2
|
+
|
3
|
+
class CfbackupTest < Test::Unit::TestCase
|
4
|
+
TEST_DIR = 'test/tmp'
|
5
|
+
|
6
|
+
context "A backup" do
|
7
|
+
|
8
|
+
context "with a single file push" do
|
9
|
+
setup do
|
10
|
+
mock_ARGV = ['--action', 'push', '--local_path', 'test/data/data.txt', '--container', 'test', '--config_file', 'test/cfconfig.yml', '-v']
|
11
|
+
@backup = CFBackup.new(mock_ARGV)
|
12
|
+
end
|
13
|
+
|
14
|
+
should "return true when file sucessfully sent" do
|
15
|
+
assert @backup.run
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
context "with a recursive directory push" do
|
20
|
+
setup do
|
21
|
+
mock_ARGV = ['--action', 'push', '-r', '--local_path', 'test/data', '--container', 'test', '--config_file', 'test/cfconfig.yml', '-v']
|
22
|
+
@backup = CFBackup.new(mock_ARGV)
|
23
|
+
end
|
24
|
+
|
25
|
+
should "return true when all files/directories successfully sent" do
|
26
|
+
assert @backup.run
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
end
|
31
|
+
|
32
|
+
context "A restore" do
|
33
|
+
|
34
|
+
context "with a single file pull" do
|
35
|
+
file = 'data.txt'
|
36
|
+
filepath = File.join(TEST_DIR, file)
|
37
|
+
|
38
|
+
setup do
|
39
|
+
mock_ARGV = ['--action', 'pull', '--container', "test:#{file}", '--local_path', "#{filepath}", '--config_file', 'test/cfconfig.yml', '-v']
|
40
|
+
@backup = CFBackup.new(mock_ARGV)
|
41
|
+
end
|
42
|
+
|
43
|
+
should "return true when file succesfully pulled" do
|
44
|
+
assert @backup.run
|
45
|
+
end
|
46
|
+
|
47
|
+
# I don't know why this doesn't work. It's like File is caching results
|
48
|
+
# and not updating until the applicaiton exists. Overcoming by
|
49
|
+
# asserting the file deletion during teardown at the loss of shoulda
|
50
|
+
#
|
51
|
+
# should "result in test/tmp/#{file} existing" do
|
52
|
+
# assert File.exist?(filepath)
|
53
|
+
# end
|
54
|
+
|
55
|
+
teardown do
|
56
|
+
assert File.delete(filepath)
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
60
|
+
context "with a recursive pull" do
|
61
|
+
setup do
|
62
|
+
mock_ARGV = ['--action', 'pull', '--container', "test", '--local_path', "test/tmp", '--config_file', 'test/cfconfig.yml', '-v']
|
63
|
+
@backup = CFBackup.new(mock_ARGV)
|
64
|
+
end
|
65
|
+
|
66
|
+
should "return true when all files pulled" do
|
67
|
+
assert @backup.run
|
68
|
+
end
|
69
|
+
|
70
|
+
# Todo compare directories
|
71
|
+
|
72
|
+
teardown do
|
73
|
+
system "rm -rf test/tmp/*"
|
74
|
+
# I didn't feel safe with TEST_DIR + '/*'
|
75
|
+
# Disaster waiting to happen. rm -rf is already bad enough.
|
76
|
+
end
|
77
|
+
end # context "A recursive pull"
|
78
|
+
|
79
|
+
end
|
80
|
+
|
81
|
+
context "A deletion" do
|
82
|
+
|
83
|
+
context "of a single file" do
|
84
|
+
setup do
|
85
|
+
mock_ARGV = ['--action', 'delete', '--container', "test:folder_1/file1.txt", '--config_file', 'test/cfconfig.yml', '-v']
|
86
|
+
@backup = CFBackup.new(mock_ARGV)
|
87
|
+
end
|
88
|
+
|
89
|
+
should "return true when file deleted" do
|
90
|
+
assert @backup.run
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
context "of a pseudo directory" do
|
95
|
+
setup do
|
96
|
+
mock_ARGV = ['--action', 'delete', '-r', '--container', "test:folder_2", '--config_file', 'test/cfconfig.yml', '-v']
|
97
|
+
@backup = CFBackup.new(mock_ARGV)
|
98
|
+
end
|
99
|
+
|
100
|
+
should "return true when directory deleted" do
|
101
|
+
assert @backup.run
|
102
|
+
end
|
103
|
+
end
|
104
|
+
|
105
|
+
end # context "A deletion"
|
106
|
+
|
107
|
+
end
|
data/test/cfconfig.yml
ADDED
data/test/data/data.txt
ADDED
data/test/test_helper.rb
ADDED
metadata
ADDED
@@ -0,0 +1,89 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: cfbackup
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.7.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Jon Stacey
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
|
12
|
+
date: 2009-05-19 00:00:00 -05:00
|
13
|
+
default_executable: cfbackup
|
14
|
+
dependencies:
|
15
|
+
- !ruby/object:Gem::Dependency
|
16
|
+
name: rackspace-cloudfiles
|
17
|
+
type: :runtime
|
18
|
+
version_requirement:
|
19
|
+
version_requirements: !ruby/object:Gem::Requirement
|
20
|
+
requirements:
|
21
|
+
- - ">="
|
22
|
+
- !ruby/object:Gem::Version
|
23
|
+
version: 1.3.0.3
|
24
|
+
version:
|
25
|
+
description: A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.
|
26
|
+
email: jon@jonsview.com
|
27
|
+
executables:
|
28
|
+
- cfbackup
|
29
|
+
extensions: []
|
30
|
+
|
31
|
+
extra_rdoc_files:
|
32
|
+
- LICENSE
|
33
|
+
- README.markdown
|
34
|
+
files:
|
35
|
+
- .gitignore
|
36
|
+
- CHANGELOG.markdown
|
37
|
+
- LICENSE
|
38
|
+
- README.markdown
|
39
|
+
- Rakefile
|
40
|
+
- VERSION.yml
|
41
|
+
- bin/cfbackup
|
42
|
+
- cfbackup.gemspec
|
43
|
+
- cfconfig.yml
|
44
|
+
- conf/cfconfig.yml
|
45
|
+
- example_scripts/piped.sh
|
46
|
+
- example_scripts/temp_directory.sh
|
47
|
+
- lib/cfbackup.rb
|
48
|
+
- lib/optcfbackup.rb
|
49
|
+
- temp/README
|
50
|
+
- test/cfbackup_test.rb
|
51
|
+
- test/cfconfig.yml
|
52
|
+
- test/data/data.txt
|
53
|
+
- test/data/folder_1/file1.txt
|
54
|
+
- test/data/folder_1/file2.txt
|
55
|
+
- test/data/folder_1/folder_3/file1.txt
|
56
|
+
- test/data/folder_2/file1.txt
|
57
|
+
- test/data/folder_2/file2.txt
|
58
|
+
- test/test_helper.rb
|
59
|
+
has_rdoc: true
|
60
|
+
homepage: http://github.com/jmstacey/cfbackup
|
61
|
+
licenses: []
|
62
|
+
|
63
|
+
post_install_message:
|
64
|
+
rdoc_options:
|
65
|
+
- --charset=UTF-8
|
66
|
+
require_paths:
|
67
|
+
- lib
|
68
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
69
|
+
requirements:
|
70
|
+
- - ">="
|
71
|
+
- !ruby/object:Gem::Version
|
72
|
+
version: "0"
|
73
|
+
version:
|
74
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
75
|
+
requirements:
|
76
|
+
- - ">="
|
77
|
+
- !ruby/object:Gem::Version
|
78
|
+
version: "0"
|
79
|
+
version:
|
80
|
+
requirements: []
|
81
|
+
|
82
|
+
rubyforge_project:
|
83
|
+
rubygems_version: 1.3.5
|
84
|
+
signing_key:
|
85
|
+
specification_version: 2
|
86
|
+
summary: A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.
|
87
|
+
test_files:
|
88
|
+
- test/cfbackup_test.rb
|
89
|
+
- test/test_helper.rb
|