cfbackup 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +6 -0
- data/CHANGELOG.markdown +45 -0
- data/LICENSE +14 -0
- data/README.markdown +63 -0
- data/Rakefile +64 -0
- data/VERSION.yml +4 -0
- data/bin/cfbackup +6 -0
- data/cfbackup.gemspec +66 -0
- data/cfconfig.yml +2 -0
- data/conf/cfconfig.yml +2 -0
- data/example_scripts/piped.sh +17 -0
- data/example_scripts/temp_directory.sh +24 -0
- data/lib/cfbackup.rb +306 -0
- data/lib/optcfbackup.rb +86 -0
- data/temp/README +5 -0
- data/test/cfbackup_test.rb +107 -0
- data/test/cfconfig.yml +2 -0
- data/test/data/data.txt +11 -0
- data/test/data/folder_1/file1.txt +2 -0
- data/test/data/folder_1/file2.txt +2 -0
- data/test/data/folder_1/folder_3/file1.txt +2 -0
- data/test/data/folder_2/file1.txt +2 -0
- data/test/data/folder_2/file2.txt +2 -0
- data/test/test_helper.rb +10 -0
- metadata +89 -0
data/.gitignore
ADDED
data/CHANGELOG.markdown
ADDED
@@ -0,0 +1,45 @@
|
|
1
|
+
CFBackup ChangeLog
|
2
|
+
==================
|
3
|
+
|
4
|
+
0.7.1 2009-05-19
|
5
|
+
-----------------
|
6
|
+
* Version bump and re-release of 0.6.1 because the published gem by GitHub was marked as 0.7.0.
|
7
|
+
|
8
|
+
0.6.1 2009-05-19
|
9
|
+
-----------------
|
10
|
+
* Use official Rackspace CloudFiles API from GitHub
|
11
|
+
* Corrected case issue on case-sensitive filesystems
|
12
|
+
|
13
|
+
0.6.0 2009-05-03
|
14
|
+
-----------------
|
15
|
+
* Added example_scripts to RDoc.
|
16
|
+
* gem creates cfconfig.yml in /etc for reference
|
17
|
+
* CFBackup looks in multiple places for config file
|
18
|
+
* Hidden directory in $HOME ($HOME/.cfconfig.yml)
|
19
|
+
* Non-hidden in current directory (./cfconfig.yml)
|
20
|
+
* In /etc/cfconfig.yml
|
21
|
+
* Refactoring and new usage with --action push|pull|delete
|
22
|
+
* Pulling files implemented
|
23
|
+
* Deleting files implemented
|
24
|
+
* Initial unit tests completed
|
25
|
+
|
26
|
+
0.5.0 2009-04-18
|
27
|
+
-----------------
|
28
|
+
|
29
|
+
* Conversion to a Ruby Gem
|
30
|
+
* cfbackup put in bin so that it can be called from anywhere
|
31
|
+
* Looks in a few standard locations for the config file
|
32
|
+
* Unit test framework prepped
|
33
|
+
* Cloud Files API has been moved to a separate repository and made available as a gem
|
34
|
+
* Most doc files converted to markdown
|
35
|
+
|
36
|
+
0.0.4 2009-04-11
|
37
|
+
-----------------
|
38
|
+
|
39
|
+
* Pipe data straight to container
|
40
|
+
* Specify remote path or file name
|
41
|
+
|
42
|
+
0.0.3 2009-04-08
|
43
|
+
-------------------
|
44
|
+
|
45
|
+
* Initial release to public
|
data/LICENSE
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
Copyright (C) 2009 Jon Stacey
|
2
|
+
|
3
|
+
This program is free software: you can redistribute it and/or modify
|
4
|
+
it under the terms of the GNU General Public License as published by
|
5
|
+
the Free Software Foundation, either version 3 of the License, or
|
6
|
+
(at your option) any later version.
|
7
|
+
|
8
|
+
This program is distributed in the hope that it will be useful,
|
9
|
+
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
10
|
+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
11
|
+
GNU General Public License for more details.
|
12
|
+
|
13
|
+
You should have received a copy of the GNU General Public License
|
14
|
+
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
data/README.markdown
ADDED
@@ -0,0 +1,63 @@
|
|
1
|
+
CFBackup
|
2
|
+
=========
|
3
|
+
|
4
|
+
CFBackup is a small ruby program that transfers files or directories from the local machine to a Cloud Files container. It is meant to serve as a useful tool for automated backups.
|
5
|
+
|
6
|
+
Features
|
7
|
+
-----------
|
8
|
+
|
9
|
+
* Push (backup) a single file or directory (uses pseudo directories)
|
10
|
+
* Pull (restore) a single file or directory
|
11
|
+
* Delete (rotate) remote objects
|
12
|
+
* Pipe data straight to container
|
13
|
+
* Free transfers over local Rackspace network for Slicehost/Cloud Server
|
14
|
+
customers in DFW1 datacenter
|
15
|
+
|
16
|
+
Requirements
|
17
|
+
--------------
|
18
|
+
|
19
|
+
* ruby-cloudfiles
|
20
|
+
|
21
|
+
Notes:
|
22
|
+
* If you install CFBackup as a gem, all of the dependencies _should_ automatically be installed for you.
|
23
|
+
* Ubuntu Users: The Ubuntu rubygems package will installs executables outside your normal PATH. You will
|
24
|
+
need to update it or create a symlink to access cfbackup from anywhere. See the wiki for more information.reating a symlink.
|
25
|
+
|
26
|
+
Install
|
27
|
+
-----------
|
28
|
+
|
29
|
+
* gem sources -a http://gems.github.com
|
30
|
+
* sudo gem install jmstacey-cfbackup
|
31
|
+
|
32
|
+
Configuration
|
33
|
+
-----------
|
34
|
+
|
35
|
+
A sample configuration file named cfconfig.yml should have be placed in /etc if installed as a gem.
|
36
|
+
CFBackup will look in the following places (in order) for the configuration file named cfconfig.yml:
|
37
|
+
|
38
|
+
* Hidden in home directory (~/.cfbackup.yml)
|
39
|
+
* Non-hidden in present working directory (./cfbackup.yml)
|
40
|
+
* In etc (/etc/cfbackup.yml)
|
41
|
+
|
42
|
+
The configuration file can be overridden at any time with the --config_file option
|
43
|
+
|
44
|
+
Configuration
|
45
|
+
-----------
|
46
|
+
|
47
|
+
Usage: cfbackup.rb --action push|pull|delete options --container CONTAINER
|
48
|
+
--action ACTION Action to perform: push, pull, or delete.
|
49
|
+
--pipe_data Pipe data from another application and stream it to Cloud Files
|
50
|
+
-r, --recursive Traverse local path recursivley
|
51
|
+
-v, --verbose Run verbosely
|
52
|
+
--local_path LOCAL_PATH Local path or file
|
53
|
+
--container CONTAINER Cloud Files container name
|
54
|
+
--version Show current version
|
55
|
+
--config_file PATH Use specified config file, rather than the default
|
56
|
+
--local_net Use unmetered connection in DFW1 (only applicable to Slicehost or Mosso Cloud Server customers)
|
57
|
+
|
58
|
+
The wiki has usage examples and some sample automation scripts can be found in the example_scripts directory.
|
59
|
+
|
60
|
+
Copyright
|
61
|
+
------------
|
62
|
+
|
63
|
+
Copyright (c) 2009 Jon Stacey. See LICENSE for details.
|
data/Rakefile
ADDED
@@ -0,0 +1,64 @@
|
|
1
|
+
require 'rake'
|
2
|
+
|
3
|
+
$LOAD_PATH.unshift('lib')
|
4
|
+
|
5
|
+
begin
|
6
|
+
require 'jeweler'
|
7
|
+
Jeweler::Tasks.new do |gem|
|
8
|
+
gem.name = "cfbackup"
|
9
|
+
gem.summary = "A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files."
|
10
|
+
gem.description = "A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files."
|
11
|
+
gem.email = "jon@jonsview.com"
|
12
|
+
gem.homepage = "http://github.com/jmstacey/cfbackup"
|
13
|
+
gem.authors = ["Jon Stacey"]
|
14
|
+
|
15
|
+
# Dependencies
|
16
|
+
gem.add_dependency('rackspace-cloudfiles', '>=1.3.0.3')
|
17
|
+
|
18
|
+
# Include Files
|
19
|
+
gem.files.include %w(lib/optcfbackup.rb)
|
20
|
+
|
21
|
+
# gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
|
22
|
+
end
|
23
|
+
rescue LoadError
|
24
|
+
puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
|
25
|
+
end
|
26
|
+
|
27
|
+
require 'rake/testtask'
|
28
|
+
Rake::TestTask.new(:test) do |test|
|
29
|
+
test.libs << 'lib' << 'test'
|
30
|
+
test.pattern = 'test/**/*_test.rb'
|
31
|
+
test.verbose = true
|
32
|
+
end
|
33
|
+
|
34
|
+
begin
|
35
|
+
require 'rcov/rcovtask'
|
36
|
+
Rcov::RcovTask.new do |test|
|
37
|
+
test.libs << 'test'
|
38
|
+
test.pattern = 'test/**/*_test.rb'
|
39
|
+
test.verbose = true
|
40
|
+
end
|
41
|
+
rescue LoadError
|
42
|
+
task :rcov do
|
43
|
+
abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
task :default => :test
|
48
|
+
|
49
|
+
require 'rake/rdoctask'
|
50
|
+
Rake::RDocTask.new do |rdoc|
|
51
|
+
if File.exist?('VERSION.yml')
|
52
|
+
config = YAML.load(File.read('VERSION.yml'))
|
53
|
+
version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
|
54
|
+
else
|
55
|
+
version = ""
|
56
|
+
end
|
57
|
+
|
58
|
+
rdoc.rdoc_dir = 'rdoc'
|
59
|
+
rdoc.title = "cfbackup #{version}"
|
60
|
+
rdoc.rdoc_files.include('README*')
|
61
|
+
rdoc.rdoc_files.include('example_scripts/**')
|
62
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
63
|
+
end
|
64
|
+
|
data/VERSION.yml
ADDED
data/bin/cfbackup
ADDED
data/cfbackup.gemspec
ADDED
@@ -0,0 +1,66 @@
|
|
1
|
+
Gem::Specification.new do |s|
|
2
|
+
s.name = %q{cfbackup}
|
3
|
+
s.version = "0.7.1"
|
4
|
+
|
5
|
+
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
6
|
+
s.authors = ["Jon Stacey"]
|
7
|
+
s.date = %q{2009-05-19}
|
8
|
+
s.default_executable = %q{cfbackup}
|
9
|
+
s.description = %q{A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.}
|
10
|
+
s.email = %q{jon@jonsview.com}
|
11
|
+
s.executables = ["cfbackup"]
|
12
|
+
s.extra_rdoc_files = [
|
13
|
+
"LICENSE",
|
14
|
+
"README.markdown"
|
15
|
+
]
|
16
|
+
s.files = [
|
17
|
+
".gitignore",
|
18
|
+
"CHANGELOG.markdown",
|
19
|
+
"LICENSE",
|
20
|
+
"README.markdown",
|
21
|
+
"Rakefile",
|
22
|
+
"VERSION.yml",
|
23
|
+
"bin/cfbackup",
|
24
|
+
"cfbackup.gemspec",
|
25
|
+
"cfconfig.yml",
|
26
|
+
"conf/cfconfig.yml",
|
27
|
+
"example_scripts/piped.sh",
|
28
|
+
"example_scripts/temp_directory.sh",
|
29
|
+
"lib/cfbackup.rb",
|
30
|
+
"lib/optcfbackup.rb",
|
31
|
+
"lib/optcfbackup.rb",
|
32
|
+
"temp/README",
|
33
|
+
"test/cfbackup_test.rb",
|
34
|
+
"test/cfconfig.yml",
|
35
|
+
"test/data/data.txt",
|
36
|
+
"test/data/folder_1/file1.txt",
|
37
|
+
"test/data/folder_1/file2.txt",
|
38
|
+
"test/data/folder_1/folder_3/file1.txt",
|
39
|
+
"test/data/folder_2/file1.txt",
|
40
|
+
"test/data/folder_2/file2.txt",
|
41
|
+
"test/test_helper.rb"
|
42
|
+
]
|
43
|
+
s.has_rdoc = true
|
44
|
+
s.homepage = %q{http://github.com/jmstacey/cfbackup}
|
45
|
+
s.rdoc_options = ["--charset=UTF-8"]
|
46
|
+
s.require_paths = ["lib"]
|
47
|
+
s.rubygems_version = %q{1.2.0}
|
48
|
+
s.summary = %q{A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.}
|
49
|
+
s.test_files = [
|
50
|
+
"test/cfbackup_test.rb",
|
51
|
+
"test/test_helper.rb"
|
52
|
+
]
|
53
|
+
|
54
|
+
if s.respond_to? :specification_version then
|
55
|
+
current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
|
56
|
+
s.specification_version = 2
|
57
|
+
|
58
|
+
if current_version >= 3 then
|
59
|
+
s.add_runtime_dependency(%q<rackspace-cloudfiles>, [">= 1.3.0.3"])
|
60
|
+
else
|
61
|
+
s.add_dependency(%q<rackspace-cloudfiles>, [">= 1.3.0.3"])
|
62
|
+
end
|
63
|
+
else
|
64
|
+
s.add_dependency(%q<rackspace-cloudfiles>, [">= 1.3.0.3"])
|
65
|
+
end
|
66
|
+
end
|
data/cfconfig.yml
ADDED
data/conf/cfconfig.yml
ADDED
@@ -0,0 +1,17 @@
|
|
1
|
+
#!/bin/sh
|
2
|
+
|
3
|
+
# In this example, data is piped directly from the backup to a Cloud Files
|
4
|
+
# container, bypassing the intermediate temp directory step
|
5
|
+
#
|
6
|
+
# This is an example script that you could use in combination with CFBackup.
|
7
|
+
# Modify it to suit your needs. Rename to file without an extension if you
|
8
|
+
# are going to place in /etc/cron.daily.
|
9
|
+
|
10
|
+
CONTAINER=backups
|
11
|
+
NOW=$(date +_%b_%d_%y)
|
12
|
+
|
13
|
+
# Backup all MySQL databases
|
14
|
+
mysqldump -u root -pPASSWORD --all-databases --flush-logs --lock-all-tables | gzip -9 | cfbackup --action push --pipe_data --container $CONTAINER:mysql_all_backup$NOW.sql.gz
|
15
|
+
|
16
|
+
# Create main backup archive
|
17
|
+
tar -cvf - /home/user /etc /usr/local/nginx | gzip -9 | cfbackup --action push --pipe_data --container $CONTAINER:main_backup$NOW.tar.gz
|
@@ -0,0 +1,24 @@
|
|
1
|
+
#!/bin/sh
|
2
|
+
|
3
|
+
# In this example, backups are first created and placed in a temporary
|
4
|
+
# holding directory. Then, the entire directory is uploaded to
|
5
|
+
# Cloud Files. Finally, the temp directory is cleared in preparation
|
6
|
+
# for the next backup.
|
7
|
+
#
|
8
|
+
# This is an example script that you could use in combination with CFBackup.
|
9
|
+
# Modify it to suit your needs. Rename to file without an extension if you
|
10
|
+
# are going to place in /etc/cron.daily.
|
11
|
+
|
12
|
+
cd ~/cfbackup/temp
|
13
|
+
CONTAINER=backups
|
14
|
+
NOW=$(date +_%b_%d_%y)
|
15
|
+
|
16
|
+
# Dump all MySQL databases
|
17
|
+
mysqldump -u root -pPASSWORD --all-databases --flush-logs --lock-all-tables | gzip -9 > mysql_all_backup$NOW.sql.gz
|
18
|
+
|
19
|
+
# Create main backup archive
|
20
|
+
tar -czvf main_backup$NOW.tar.gz /home/user /etc /usr/local/nginx
|
21
|
+
|
22
|
+
cd ~/cfbackup
|
23
|
+
cfbackup --action push --local_path temp/ --container $CONTAINER
|
24
|
+
rm -f temp/*
|
data/lib/cfbackup.rb
ADDED
@@ -0,0 +1,306 @@
|
|
1
|
+
# CFBackup, a small utility script to backup files to Mosso Cloud Files
|
2
|
+
# Copyright (C) 2009 Jon Stacey
|
3
|
+
#
|
4
|
+
# This program is free software: you can redistribute it and/or modify
|
5
|
+
# it under the terms of the GNU General Public License as published by
|
6
|
+
# the Free Software Foundation, either version 3 of the License, or
|
7
|
+
# (at your option) any later version.
|
8
|
+
#
|
9
|
+
# This program is distributed in the hope that it will be useful,
|
10
|
+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
11
|
+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
12
|
+
# GNU General Public License for more details.
|
13
|
+
#
|
14
|
+
# You should have received a copy of the GNU General Public License
|
15
|
+
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
16
|
+
|
17
|
+
require 'rubygems'
|
18
|
+
require 'ftools'
|
19
|
+
require 'cloudfiles'
|
20
|
+
require 'optcfbackup'
|
21
|
+
require 'yaml'
|
22
|
+
|
23
|
+
class CFBackup
|
24
|
+
|
25
|
+
def initialize(args)
|
26
|
+
@opts = OptCFBackup.new(args)
|
27
|
+
|
28
|
+
# Special case if the version is requested
|
29
|
+
if @opts.options.show_ver
|
30
|
+
version_file = File.join(File.dirname(__FILE__), '..', 'VERSION.yml')
|
31
|
+
if File.exist?(version_file)
|
32
|
+
config = YAML.load(File.read(version_file))
|
33
|
+
version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
|
34
|
+
else
|
35
|
+
version = "unkown version"
|
36
|
+
end
|
37
|
+
show_error("CFBackup #{version}")
|
38
|
+
end
|
39
|
+
|
40
|
+
# Locate and load config file
|
41
|
+
@opts.options.config.each do |path|
|
42
|
+
if (File.exist?(path))
|
43
|
+
@conf = YAML::load(File.open(path))
|
44
|
+
break
|
45
|
+
end
|
46
|
+
end
|
47
|
+
show_error('Error: Unable to locate config file.') unless (@conf != nil)
|
48
|
+
|
49
|
+
prep_connection
|
50
|
+
|
51
|
+
end # initialize()
|
52
|
+
|
53
|
+
def run
|
54
|
+
|
55
|
+
show_error() unless (@opts.options.container != "")
|
56
|
+
|
57
|
+
# Run appropriate action
|
58
|
+
case @opts.options.action
|
59
|
+
when 'push'
|
60
|
+
if @opts.options.pipe_data
|
61
|
+
push_piped_data
|
62
|
+
else
|
63
|
+
push_files
|
64
|
+
end
|
65
|
+
when 'pull'
|
66
|
+
pull_files()
|
67
|
+
when 'delete'
|
68
|
+
delete_files
|
69
|
+
else
|
70
|
+
show_error()
|
71
|
+
end
|
72
|
+
|
73
|
+
end # run()
|
74
|
+
|
75
|
+
private
|
76
|
+
|
77
|
+
def prep_connection
|
78
|
+
# Establish connection
|
79
|
+
show_verbose "Establishing connection...", false
|
80
|
+
@cf = CloudFiles::Connection.new(@conf["username"], @conf["api_key"]);
|
81
|
+
show_verbose " done."
|
82
|
+
|
83
|
+
# Special option for Slicehost customers in DFW datacenter
|
84
|
+
if @opts.options.local_net
|
85
|
+
@cf.storagehost = 'snet-storage.clouddrive.com'
|
86
|
+
end
|
87
|
+
end # prep_connection()
|
88
|
+
|
89
|
+
def prep_container(create_container = true)
|
90
|
+
# Check for the container. If it doesn't exist, create it if allowed
|
91
|
+
if !@cf.container_exists?(@opts.options.container)
|
92
|
+
if create_container
|
93
|
+
show_verbose "Container '#{@opts.options.container}' does not exist. Creating it...", false
|
94
|
+
@cf.create_container(@opts.options.container)
|
95
|
+
show_verbose " done."
|
96
|
+
else
|
97
|
+
show_error("Error: Container '#{@opts.options.container}' does not exist.")
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
@container = @cf.container(@opts.options.container)
|
102
|
+
end # prep_cnnection()
|
103
|
+
|
104
|
+
def push_piped_data
|
105
|
+
prep_container
|
106
|
+
|
107
|
+
puts "Warning: 5GB maximum filesize"
|
108
|
+
object = @container.create_object(@opts.options.remote_path, true)
|
109
|
+
object.write
|
110
|
+
end # push_piped_data()
|
111
|
+
|
112
|
+
def push_files
|
113
|
+
prep_container
|
114
|
+
|
115
|
+
path = @opts.options.local_path
|
116
|
+
pwd = Dir.getwd # Save current directory so we can come back
|
117
|
+
|
118
|
+
if FileTest::file?(path)
|
119
|
+
glob_options = File.join(File::basename(path))
|
120
|
+
elsif @opts.options.recursive
|
121
|
+
Dir.chdir(path)
|
122
|
+
glob_options = File.join("**", "*")
|
123
|
+
else
|
124
|
+
Dir.chdir(path)
|
125
|
+
glob_options = File.join("*")
|
126
|
+
end
|
127
|
+
files = Dir.glob(glob_options)
|
128
|
+
|
129
|
+
# Upload file(s)
|
130
|
+
counter = 1
|
131
|
+
show_verbose "There are #{files.length} files to process."
|
132
|
+
files.each do |file|
|
133
|
+
file = file.sub(/\.\//, '')
|
134
|
+
if file == "" || file[0,1] == "." || FileTest.directory?(file)
|
135
|
+
show_verbose "(#{counter}/#{files.length}) Skipping #{file}"
|
136
|
+
counter += 1
|
137
|
+
next
|
138
|
+
end
|
139
|
+
|
140
|
+
show_verbose "(#{counter}/#{files.length}) Pushing file #{file}...", false
|
141
|
+
|
142
|
+
if @opts.options.remote_path.to_s == ''
|
143
|
+
remote_path = file
|
144
|
+
else
|
145
|
+
remote_path = File.join(@opts.options.remote_path, file)
|
146
|
+
end
|
147
|
+
|
148
|
+
object = @container.create_object(remote_path, true)
|
149
|
+
object.load_from_filename(file)
|
150
|
+
|
151
|
+
show_verbose " done."
|
152
|
+
counter += 1
|
153
|
+
end # files.each
|
154
|
+
|
155
|
+
Dir.chdir(pwd) # Go back to original directory
|
156
|
+
|
157
|
+
end # push_files()
|
158
|
+
|
159
|
+
def pull_files
|
160
|
+
prep_container(false)
|
161
|
+
|
162
|
+
file = object_file?
|
163
|
+
objects = get_objects_array
|
164
|
+
|
165
|
+
# Process objects
|
166
|
+
counter = 1
|
167
|
+
show_verbose "There are #{objects.length} objects to process."
|
168
|
+
objects.each do |object_name|
|
169
|
+
object = @container.object(object_name)
|
170
|
+
if object.content_type == "application/directory"
|
171
|
+
show_verbose "(#{counter}/#{objects.length}) Pseudo directory #{object.name}"
|
172
|
+
counter += 1
|
173
|
+
next
|
174
|
+
end
|
175
|
+
|
176
|
+
path_info = File.split(@opts.options.local_path.to_s)
|
177
|
+
file_info = File.split(object.name.to_s)
|
178
|
+
|
179
|
+
if file # Dealing with a single file pull
|
180
|
+
if @opts.options.local_path.to_s == ''
|
181
|
+
filepath = file_info[1].to_s # Use current directory and original name
|
182
|
+
else
|
183
|
+
if File.exist?(@opts.options.local_path.to_s)
|
184
|
+
# The file exists, so we will overwrite it
|
185
|
+
filepath = File.join(@opts.options.local_path.to_s)
|
186
|
+
else
|
187
|
+
# If the file doesn't exist, a new name may have been given.
|
188
|
+
# Test the path.
|
189
|
+
if File.exist?(path_info[0])
|
190
|
+
# A new name was given with a valid path
|
191
|
+
filepath = File.join(path_info[0], path_info[1])
|
192
|
+
else
|
193
|
+
# The given path is not valid
|
194
|
+
show_error("cfbackup: #{file_info[0]}: No such file or directory.")
|
195
|
+
end
|
196
|
+
end
|
197
|
+
end
|
198
|
+
else # Dealing with a multi-object pull
|
199
|
+
if @opts.options.local_path.to_s == ''
|
200
|
+
filepath = object.name.to_s # Use current directory with object name
|
201
|
+
dir_path = file_info[0]
|
202
|
+
else
|
203
|
+
if File.directory?(@opts.options.local_path.to_s)
|
204
|
+
filepath = File.join(@opts.options.local_path.to_s, object.name.to_s)
|
205
|
+
dir_path = File.join(@opts.options.local_path, file_info[0])
|
206
|
+
else
|
207
|
+
# We can't copy a directory to a file...
|
208
|
+
show_error("cfbackup: #{@container.name}:#{@opts.options.remote_path.to_s}/ is a directory (not copied).")
|
209
|
+
end
|
210
|
+
end
|
211
|
+
File.makedirs(dir_path) # Create subdirectories as needed
|
212
|
+
end
|
213
|
+
|
214
|
+
show_verbose "(#{counter}/#{objects.length}) Pulling object #{object.name}...", false
|
215
|
+
object.save_to_filename(filepath)
|
216
|
+
show_verbose " done"
|
217
|
+
counter += 1
|
218
|
+
end
|
219
|
+
end # pull_files()
|
220
|
+
|
221
|
+
def delete_files
|
222
|
+
prep_container(false)
|
223
|
+
|
224
|
+
if object_file?
|
225
|
+
show_verbose "Deleting object #{@opts.options.remote_path.to_s}"
|
226
|
+
@container.delete_object(@opts.options.remote_path.to_s)
|
227
|
+
else
|
228
|
+
if !@opts.options.recursive
|
229
|
+
show_error("Error: #{@opts.options.remote_path} is a directory but the the recursive option was not specified. Doing nothing.")
|
230
|
+
else
|
231
|
+
objects = get_objects_array
|
232
|
+
|
233
|
+
# Process objects
|
234
|
+
counter = 1
|
235
|
+
show_verbose "There are #{objects.length} objects to process."
|
236
|
+
objects.each do |object_name|
|
237
|
+
show_verbose "(#{counter}/#{objects.length}) Deleting object #{object_name}...", false
|
238
|
+
@container.delete_object(object_name)
|
239
|
+
show_verbose " done"
|
240
|
+
counter += 1
|
241
|
+
end
|
242
|
+
end
|
243
|
+
end
|
244
|
+
|
245
|
+
end # delete_files()
|
246
|
+
|
247
|
+
|
248
|
+
# Helper methods below
|
249
|
+
|
250
|
+
def object_file?
|
251
|
+
|
252
|
+
file = false
|
253
|
+
unless @opts.options.remote_path.to_s == ''
|
254
|
+
if @container.object_exists?(@opts.options.remote_path)
|
255
|
+
if @container.object(@opts.options.remote_path).content_type != "application/directory"
|
256
|
+
file = true
|
257
|
+
if @opts.options.recursive
|
258
|
+
puts "Warning: This is a file so the recursive option is meaningless."
|
259
|
+
end
|
260
|
+
end
|
261
|
+
else
|
262
|
+
show_error("The object #{@opts.options.remote_path} does not exist")
|
263
|
+
end
|
264
|
+
end
|
265
|
+
|
266
|
+
return file
|
267
|
+
end
|
268
|
+
|
269
|
+
def get_objects_array
|
270
|
+
|
271
|
+
file = object_file?
|
272
|
+
|
273
|
+
# Get array of objects to process
|
274
|
+
objects = Array.new
|
275
|
+
if file
|
276
|
+
objects << @opts.options.remote_path.to_s
|
277
|
+
elsif @opts.options.recursive
|
278
|
+
# Use prefix instead of path so that "subdirectories" are included
|
279
|
+
objects = @container.objects(:prefix => @opts.options.remote_path)
|
280
|
+
else
|
281
|
+
objects = @container.objects(:path => @opts.options.remote_path)
|
282
|
+
end
|
283
|
+
|
284
|
+
return objects
|
285
|
+
end
|
286
|
+
|
287
|
+
# Shows given message if verbose output is turned on
|
288
|
+
def show_verbose(message, line_break = true)
|
289
|
+
unless !@opts.options.verbose
|
290
|
+
if line_break
|
291
|
+
puts message
|
292
|
+
else
|
293
|
+
print message
|
294
|
+
end
|
295
|
+
$stdout.flush
|
296
|
+
end
|
297
|
+
end # show_verbose()
|
298
|
+
|
299
|
+
# Show error message, banner and exit
|
300
|
+
def show_error(message = '')
|
301
|
+
puts message
|
302
|
+
puts @opts.banner
|
303
|
+
exit
|
304
|
+
end # show_error()
|
305
|
+
|
306
|
+
end # class CFBackup
|
data/lib/optcfbackup.rb
ADDED
@@ -0,0 +1,86 @@
|
|
1
|
+
require 'optparse'
|
2
|
+
require 'ostruct'
|
3
|
+
|
4
|
+
class OptCFBackup
|
5
|
+
|
6
|
+
# Options structure
|
7
|
+
attr_reader :options
|
8
|
+
|
9
|
+
# Ussage message
|
10
|
+
attr_reader :banner
|
11
|
+
|
12
|
+
# Initializes object with command line arguments passed
|
13
|
+
def initialize(args)
|
14
|
+
|
15
|
+
@banner = "Usage: cfbackup.rb --action push|pull|delete options --container CONTAINER"
|
16
|
+
|
17
|
+
@options = OpenStruct.new
|
18
|
+
self.options.config = ["#{ENV['HOME']}/.cfconfig.yml", './cfconfig.yml', '/etc/cfconfig.yml']
|
19
|
+
self.options.action = ''
|
20
|
+
self.options.pipe_data = false
|
21
|
+
self.options.show_ver = false
|
22
|
+
self.options.recursive = false
|
23
|
+
self.options.local_net = false
|
24
|
+
self.options.container = ''
|
25
|
+
self.options.local_path = ''
|
26
|
+
self.options.remote_path = ''
|
27
|
+
self.options.verbose = false;
|
28
|
+
|
29
|
+
opts = OptionParser.new do |opts|
|
30
|
+
opts.banner = self.banner
|
31
|
+
|
32
|
+
opts.on("--action ACTION", "Action to perform: push, pull, or delete.") do |action|
|
33
|
+
self.options.action = action
|
34
|
+
end
|
35
|
+
|
36
|
+
opts.on("--pipe_data", "Pipe data from another application and stream it to Cloud Files") do |pipe|
|
37
|
+
self.options.pipe_data = pipe
|
38
|
+
end
|
39
|
+
|
40
|
+
opts.on("-r", "--recursive", "Traverse local path recursivley") do |recursive|
|
41
|
+
self.options.recursive = recursive
|
42
|
+
end
|
43
|
+
|
44
|
+
opts.on("-v", "--verbose", "Run verbosely") do |verbose|
|
45
|
+
self.options.verbose = verbose
|
46
|
+
end
|
47
|
+
|
48
|
+
opts.on("--local_path LOCAL_PATH", "Local path or file") do |local_path|
|
49
|
+
self.options.local_path = local_path
|
50
|
+
end
|
51
|
+
|
52
|
+
opts.on("--container CONTAINER", "Cloud Files container name") do |name|
|
53
|
+
self.options.container, self.options.remote_path = name.split(":", 2)
|
54
|
+
clean_remote_path unless (self.options.remote_path.nil?)
|
55
|
+
end
|
56
|
+
|
57
|
+
opts.on("--version", "Show current version") do |version|
|
58
|
+
self.options.show_ver = version
|
59
|
+
end
|
60
|
+
|
61
|
+
opts.on("--config_file PATH", "Use specified config file, rather than the default") do |config|
|
62
|
+
self.options.config = Array.new()
|
63
|
+
self.options.config << config
|
64
|
+
end
|
65
|
+
|
66
|
+
opts.on("--local_net", "Use unmetered connection in DFW1 (only applicable to Slicehost or Mosso Cloud Server customers)") do |local_net|
|
67
|
+
self.options.local_net = local_net
|
68
|
+
end
|
69
|
+
|
70
|
+
end
|
71
|
+
|
72
|
+
opts.parse!(args)
|
73
|
+
|
74
|
+
end # initialize()
|
75
|
+
|
76
|
+
private
|
77
|
+
|
78
|
+
def clean_remote_path
|
79
|
+
if self.options.remote_path[0,1] == "/"
|
80
|
+
self.options.remote_path.slice!(0)
|
81
|
+
end
|
82
|
+
# Follwoig won't work for piped data. Might result in "text.txt/"
|
83
|
+
# self.options.remote_path = self.options.remote_path + "/" unless (self.options.remote_path[-1,1] == "/")
|
84
|
+
end
|
85
|
+
|
86
|
+
end # class OptCFBackup
|
data/temp/README
ADDED
@@ -0,0 +1,107 @@
|
|
1
|
+
require 'test_helper'
|
2
|
+
|
3
|
+
class CfbackupTest < Test::Unit::TestCase
|
4
|
+
TEST_DIR = 'test/tmp'
|
5
|
+
|
6
|
+
context "A backup" do
|
7
|
+
|
8
|
+
context "with a single file push" do
|
9
|
+
setup do
|
10
|
+
mock_ARGV = ['--action', 'push', '--local_path', 'test/data/data.txt', '--container', 'test', '--config_file', 'test/cfconfig.yml', '-v']
|
11
|
+
@backup = CFBackup.new(mock_ARGV)
|
12
|
+
end
|
13
|
+
|
14
|
+
should "return true when file sucessfully sent" do
|
15
|
+
assert @backup.run
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
context "with a recursive directory push" do
|
20
|
+
setup do
|
21
|
+
mock_ARGV = ['--action', 'push', '-r', '--local_path', 'test/data', '--container', 'test', '--config_file', 'test/cfconfig.yml', '-v']
|
22
|
+
@backup = CFBackup.new(mock_ARGV)
|
23
|
+
end
|
24
|
+
|
25
|
+
should "return true when all files/directories successfully sent" do
|
26
|
+
assert @backup.run
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
end
|
31
|
+
|
32
|
+
context "A restore" do
|
33
|
+
|
34
|
+
context "with a single file pull" do
|
35
|
+
file = 'data.txt'
|
36
|
+
filepath = File.join(TEST_DIR, file)
|
37
|
+
|
38
|
+
setup do
|
39
|
+
mock_ARGV = ['--action', 'pull', '--container', "test:#{file}", '--local_path', "#{filepath}", '--config_file', 'test/cfconfig.yml', '-v']
|
40
|
+
@backup = CFBackup.new(mock_ARGV)
|
41
|
+
end
|
42
|
+
|
43
|
+
should "return true when file succesfully pulled" do
|
44
|
+
assert @backup.run
|
45
|
+
end
|
46
|
+
|
47
|
+
# I don't know why this doesn't work. It's like File is caching results
|
48
|
+
# and not updating until the applicaiton exists. Overcoming by
|
49
|
+
# asserting the file deletion during teardown at the loss of shoulda
|
50
|
+
#
|
51
|
+
# should "result in test/tmp/#{file} existing" do
|
52
|
+
# assert File.exist?(filepath)
|
53
|
+
# end
|
54
|
+
|
55
|
+
teardown do
|
56
|
+
assert File.delete(filepath)
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
60
|
+
context "with a recursive pull" do
|
61
|
+
setup do
|
62
|
+
mock_ARGV = ['--action', 'pull', '--container', "test", '--local_path', "test/tmp", '--config_file', 'test/cfconfig.yml', '-v']
|
63
|
+
@backup = CFBackup.new(mock_ARGV)
|
64
|
+
end
|
65
|
+
|
66
|
+
should "return true when all files pulled" do
|
67
|
+
assert @backup.run
|
68
|
+
end
|
69
|
+
|
70
|
+
# Todo compare directories
|
71
|
+
|
72
|
+
teardown do
|
73
|
+
system "rm -rf test/tmp/*"
|
74
|
+
# I didn't feel safe with TEST_DIR + '/*'
|
75
|
+
# Disaster waiting to happen. rm -rf is already bad enough.
|
76
|
+
end
|
77
|
+
end # context "A recursive pull"
|
78
|
+
|
79
|
+
end
|
80
|
+
|
81
|
+
context "A deletion" do
|
82
|
+
|
83
|
+
context "of a single file" do
|
84
|
+
setup do
|
85
|
+
mock_ARGV = ['--action', 'delete', '--container', "test:folder_1/file1.txt", '--config_file', 'test/cfconfig.yml', '-v']
|
86
|
+
@backup = CFBackup.new(mock_ARGV)
|
87
|
+
end
|
88
|
+
|
89
|
+
should "return true when file deleted" do
|
90
|
+
assert @backup.run
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
context "of a pseudo directory" do
|
95
|
+
setup do
|
96
|
+
mock_ARGV = ['--action', 'delete', '-r', '--container', "test:folder_2", '--config_file', 'test/cfconfig.yml', '-v']
|
97
|
+
@backup = CFBackup.new(mock_ARGV)
|
98
|
+
end
|
99
|
+
|
100
|
+
should "return true when directory deleted" do
|
101
|
+
assert @backup.run
|
102
|
+
end
|
103
|
+
end
|
104
|
+
|
105
|
+
end # context "A deletion"
|
106
|
+
|
107
|
+
end
|
data/test/cfconfig.yml
ADDED
data/test/data/data.txt
ADDED
data/test/test_helper.rb
ADDED
metadata
ADDED
@@ -0,0 +1,89 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: cfbackup
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.7.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Jon Stacey
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
|
12
|
+
date: 2009-05-19 00:00:00 -05:00
|
13
|
+
default_executable: cfbackup
|
14
|
+
dependencies:
|
15
|
+
- !ruby/object:Gem::Dependency
|
16
|
+
name: rackspace-cloudfiles
|
17
|
+
type: :runtime
|
18
|
+
version_requirement:
|
19
|
+
version_requirements: !ruby/object:Gem::Requirement
|
20
|
+
requirements:
|
21
|
+
- - ">="
|
22
|
+
- !ruby/object:Gem::Version
|
23
|
+
version: 1.3.0.3
|
24
|
+
version:
|
25
|
+
description: A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.
|
26
|
+
email: jon@jonsview.com
|
27
|
+
executables:
|
28
|
+
- cfbackup
|
29
|
+
extensions: []
|
30
|
+
|
31
|
+
extra_rdoc_files:
|
32
|
+
- LICENSE
|
33
|
+
- README.markdown
|
34
|
+
files:
|
35
|
+
- .gitignore
|
36
|
+
- CHANGELOG.markdown
|
37
|
+
- LICENSE
|
38
|
+
- README.markdown
|
39
|
+
- Rakefile
|
40
|
+
- VERSION.yml
|
41
|
+
- bin/cfbackup
|
42
|
+
- cfbackup.gemspec
|
43
|
+
- cfconfig.yml
|
44
|
+
- conf/cfconfig.yml
|
45
|
+
- example_scripts/piped.sh
|
46
|
+
- example_scripts/temp_directory.sh
|
47
|
+
- lib/cfbackup.rb
|
48
|
+
- lib/optcfbackup.rb
|
49
|
+
- temp/README
|
50
|
+
- test/cfbackup_test.rb
|
51
|
+
- test/cfconfig.yml
|
52
|
+
- test/data/data.txt
|
53
|
+
- test/data/folder_1/file1.txt
|
54
|
+
- test/data/folder_1/file2.txt
|
55
|
+
- test/data/folder_1/folder_3/file1.txt
|
56
|
+
- test/data/folder_2/file1.txt
|
57
|
+
- test/data/folder_2/file2.txt
|
58
|
+
- test/test_helper.rb
|
59
|
+
has_rdoc: true
|
60
|
+
homepage: http://github.com/jmstacey/cfbackup
|
61
|
+
licenses: []
|
62
|
+
|
63
|
+
post_install_message:
|
64
|
+
rdoc_options:
|
65
|
+
- --charset=UTF-8
|
66
|
+
require_paths:
|
67
|
+
- lib
|
68
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
69
|
+
requirements:
|
70
|
+
- - ">="
|
71
|
+
- !ruby/object:Gem::Version
|
72
|
+
version: "0"
|
73
|
+
version:
|
74
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
75
|
+
requirements:
|
76
|
+
- - ">="
|
77
|
+
- !ruby/object:Gem::Version
|
78
|
+
version: "0"
|
79
|
+
version:
|
80
|
+
requirements: []
|
81
|
+
|
82
|
+
rubyforge_project:
|
83
|
+
rubygems_version: 1.3.5
|
84
|
+
signing_key:
|
85
|
+
specification_version: 2
|
86
|
+
summary: A simple ruby program intended to serve as a useful tool for automated backups to Mosso Cloud Files.
|
87
|
+
test_files:
|
88
|
+
- test/cfbackup_test.rb
|
89
|
+
- test/test_helper.rb
|