backup 3.0.27 → 3.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/LICENSE.md +1 -1
- data/README.md +139 -386
- data/bin/backup +1 -7
- data/lib/backup.rb +3 -9
- data/lib/backup/archive.rb +26 -20
- data/lib/backup/cleaner.rb +2 -2
- data/lib/backup/cli.rb +366 -0
- data/lib/backup/compressor/base.rb +2 -2
- data/lib/backup/compressor/gzip.rb +35 -1
- data/lib/backup/config.rb +1 -2
- data/lib/backup/database/base.rb +2 -2
- data/lib/backup/database/mongodb.rb +3 -3
- data/lib/backup/database/mysql.rb +3 -2
- data/lib/backup/database/postgresql.rb +3 -2
- data/lib/backup/database/riak.rb +18 -5
- data/lib/backup/dependency.rb +144 -93
- data/lib/backup/encryptor/base.rb +2 -2
- data/lib/backup/logger.rb +108 -110
- data/lib/backup/logger/console.rb +51 -0
- data/lib/backup/logger/logfile.rb +113 -0
- data/lib/backup/logger/syslog.rb +116 -0
- data/lib/backup/model.rb +67 -65
- data/lib/backup/notifier/base.rb +1 -1
- data/lib/backup/notifier/hipchat.rb +1 -1
- data/lib/backup/notifier/mail.rb +1 -1
- data/lib/backup/notifier/pushover.rb +6 -3
- data/lib/backup/packager.rb +4 -4
- data/lib/backup/pipeline.rb +17 -3
- data/lib/backup/splitter.rb +2 -2
- data/lib/backup/storage/base.rb +2 -2
- data/lib/backup/storage/cloudfiles.rb +2 -2
- data/lib/backup/storage/dropbox.rb +4 -4
- data/lib/backup/storage/ftp.rb +2 -2
- data/lib/backup/storage/local.rb +2 -2
- data/lib/backup/storage/ninefold.rb +2 -2
- data/lib/backup/storage/rsync.rb +3 -3
- data/lib/backup/storage/s3.rb +2 -2
- data/lib/backup/storage/scp.rb +2 -6
- data/lib/backup/storage/sftp.rb +2 -5
- data/lib/backup/syncer/base.rb +1 -1
- data/lib/backup/syncer/cloud/base.rb +15 -8
- data/lib/backup/syncer/rsync/local.rb +1 -1
- data/lib/backup/syncer/rsync/pull.rb +1 -1
- data/lib/backup/syncer/rsync/push.rb +1 -1
- data/lib/backup/utilities.rb +211 -0
- data/lib/backup/version.rb +1 -1
- data/templates/cli/{utility/archive → archive} +4 -8
- data/templates/cli/{utility/compressor → compressor}/bzip2 +0 -0
- data/templates/cli/{utility/compressor → compressor}/custom +0 -0
- data/templates/cli/{utility/compressor → compressor}/gzip +0 -0
- data/templates/cli/{utility/compressor → compressor}/lzma +0 -0
- data/templates/cli/{utility/compressor → compressor}/pbzip2 +0 -0
- data/templates/cli/config +68 -0
- data/templates/cli/{utility/database → database}/mongodb +1 -1
- data/templates/cli/{utility/database → database}/mysql +1 -1
- data/templates/cli/{utility/database → database}/postgresql +1 -1
- data/templates/cli/{utility/database → database}/redis +0 -0
- data/templates/cli/database/riak +20 -0
- data/templates/cli/{utility/encryptor → encryptor}/gpg +0 -0
- data/templates/cli/{utility/encryptor → encryptor}/openssl +0 -0
- data/templates/cli/{utility/model.erb → model.erb} +4 -4
- data/templates/cli/{utility/notifier → notifier}/campfire +0 -0
- data/templates/cli/{utility/notifier → notifier}/hipchat +0 -0
- data/templates/cli/{utility/notifier → notifier}/mail +0 -0
- data/templates/cli/{utility/notifier → notifier}/prowl +0 -0
- data/templates/cli/{utility/notifier → notifier}/pushover +0 -0
- data/templates/cli/{utility/notifier → notifier}/twitter +0 -0
- data/templates/cli/{utility/splitter → splitter} +0 -0
- data/templates/cli/{utility/storage → storage}/cloud_files +0 -0
- data/templates/cli/{utility/storage → storage}/dropbox +0 -0
- data/templates/cli/{utility/storage → storage}/ftp +0 -0
- data/templates/cli/{utility/storage → storage}/local +0 -0
- data/templates/cli/{utility/storage → storage}/ninefold +0 -0
- data/templates/cli/{utility/storage → storage}/rsync +0 -0
- data/templates/cli/{utility/storage → storage}/s3 +0 -0
- data/templates/cli/{utility/storage → storage}/scp +0 -0
- data/templates/cli/{utility/storage → storage}/sftp +0 -0
- data/templates/cli/{utility/syncer → syncer}/cloud_files +0 -0
- data/templates/cli/{utility/syncer → syncer}/rsync_local +0 -0
- data/templates/cli/{utility/syncer → syncer}/rsync_pull +0 -0
- data/templates/cli/{utility/syncer → syncer}/rsync_push +0 -0
- data/templates/cli/{utility/syncer → syncer}/s3 +0 -0
- metadata +55 -131
- data/.gitignore +0 -8
- data/.travis.yml +0 -10
- data/Gemfile +0 -28
- data/Guardfile +0 -23
- data/backup.gemspec +0 -32
- data/lib/backup/cli/helpers.rb +0 -93
- data/lib/backup/cli/utility.rb +0 -255
- data/spec-live/.gitignore +0 -6
- data/spec-live/README +0 -7
- data/spec-live/backups/config.rb +0 -83
- data/spec-live/backups/config.yml.template +0 -46
- data/spec-live/backups/models.rb +0 -184
- data/spec-live/compressor/custom_spec.rb +0 -30
- data/spec-live/compressor/gzip_spec.rb +0 -30
- data/spec-live/encryptor/gpg_keys.rb +0 -239
- data/spec-live/encryptor/gpg_spec.rb +0 -287
- data/spec-live/notifier/mail_spec.rb +0 -121
- data/spec-live/spec_helper.rb +0 -151
- data/spec-live/storage/dropbox_spec.rb +0 -151
- data/spec-live/storage/local_spec.rb +0 -83
- data/spec-live/storage/scp_spec.rb +0 -193
- data/spec-live/syncer/cloud/s3_spec.rb +0 -124
- data/spec/archive_spec.rb +0 -335
- data/spec/cleaner_spec.rb +0 -312
- data/spec/cli/helpers_spec.rb +0 -301
- data/spec/cli/utility_spec.rb +0 -411
- data/spec/compressor/base_spec.rb +0 -52
- data/spec/compressor/bzip2_spec.rb +0 -217
- data/spec/compressor/custom_spec.rb +0 -106
- data/spec/compressor/gzip_spec.rb +0 -217
- data/spec/compressor/lzma_spec.rb +0 -123
- data/spec/compressor/pbzip2_spec.rb +0 -165
- data/spec/config_spec.rb +0 -321
- data/spec/configuration/helpers_spec.rb +0 -247
- data/spec/configuration/store_spec.rb +0 -39
- data/spec/configuration_spec.rb +0 -62
- data/spec/database/base_spec.rb +0 -63
- data/spec/database/mongodb_spec.rb +0 -510
- data/spec/database/mysql_spec.rb +0 -411
- data/spec/database/postgresql_spec.rb +0 -353
- data/spec/database/redis_spec.rb +0 -334
- data/spec/database/riak_spec.rb +0 -176
- data/spec/dependency_spec.rb +0 -51
- data/spec/encryptor/base_spec.rb +0 -40
- data/spec/encryptor/gpg_spec.rb +0 -909
- data/spec/encryptor/open_ssl_spec.rb +0 -148
- data/spec/errors_spec.rb +0 -306
- data/spec/logger_spec.rb +0 -367
- data/spec/model_spec.rb +0 -666
- data/spec/notifier/base_spec.rb +0 -104
- data/spec/notifier/campfire_spec.rb +0 -217
- data/spec/notifier/hipchat_spec.rb +0 -211
- data/spec/notifier/mail_spec.rb +0 -316
- data/spec/notifier/prowl_spec.rb +0 -138
- data/spec/notifier/pushover_spec.rb +0 -123
- data/spec/notifier/twitter_spec.rb +0 -153
- data/spec/package_spec.rb +0 -61
- data/spec/packager_spec.rb +0 -213
- data/spec/pipeline_spec.rb +0 -259
- data/spec/spec_helper.rb +0 -60
- data/spec/splitter_spec.rb +0 -120
- data/spec/storage/base_spec.rb +0 -166
- data/spec/storage/cloudfiles_spec.rb +0 -254
- data/spec/storage/cycler_spec.rb +0 -247
- data/spec/storage/dropbox_spec.rb +0 -480
- data/spec/storage/ftp_spec.rb +0 -271
- data/spec/storage/local_spec.rb +0 -259
- data/spec/storage/ninefold_spec.rb +0 -343
- data/spec/storage/rsync_spec.rb +0 -362
- data/spec/storage/s3_spec.rb +0 -245
- data/spec/storage/scp_spec.rb +0 -233
- data/spec/storage/sftp_spec.rb +0 -244
- data/spec/syncer/base_spec.rb +0 -109
- data/spec/syncer/cloud/base_spec.rb +0 -515
- data/spec/syncer/cloud/cloud_files_spec.rb +0 -181
- data/spec/syncer/cloud/s3_spec.rb +0 -174
- data/spec/syncer/rsync/base_spec.rb +0 -98
- data/spec/syncer/rsync/local_spec.rb +0 -149
- data/spec/syncer/rsync/pull_spec.rb +0 -98
- data/spec/syncer/rsync/push_spec.rb +0 -333
- data/spec/version_spec.rb +0 -21
- data/templates/cli/utility/config +0 -32
- data/templates/cli/utility/database/riak +0 -11
data/LICENSE.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
|
|
2
|
-
Copyright (c) 2009-
|
|
2
|
+
Copyright (c) 2009-2013 Michael van Rooijen ( [@meskyanichi](http://twitter.com/#!/meskyanichi) )
|
|
3
3
|
=================================================================================================
|
|
4
4
|
|
|
5
5
|
The "Backup" RubyGem is released under the **MIT LICENSE**
|
data/README.md
CHANGED
|
@@ -1,38 +1,45 @@
|
|
|
1
1
|
Backup
|
|
2
2
|
======
|
|
3
3
|
|
|
4
|
-
Backup is a
|
|
4
|
+
Backup is a system utility for Linux and Mac OS X, distributed as a RubyGem, that allows you to easily perform backup
|
|
5
|
+
operations. It provides an elegant DSL in Ruby for _modeling_ your backups. Backup has built-in support for various
|
|
6
|
+
databases, storage protocols/services, syncers, compressors, encryptors and notifiers which you can mix and match. It
|
|
7
|
+
was built with modularity, extensibility and simplicity in mind.
|
|
5
8
|
|
|
6
|
-
|
|
7
|
-
[](http://stillmaintained.com/meskyanichi/backup)
|
|
9
|
+
## Installation
|
|
8
10
|
|
|
11
|
+
To get the latest stable version:
|
|
9
12
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
**[Michael van Rooijen](http://michaelvanrooijen.com/) ( [@meskyanichi](http://twitter.com/#!/meskyanichi) )**
|
|
13
|
-
|
|
14
|
-
Drop me a message for any questions, suggestions, requests, bugs or submit them to the [issue log](https://github.com/meskyanichi/backup/issues).
|
|
15
|
-
|
|
13
|
+
gem install backup
|
|
16
14
|
|
|
17
|
-
|
|
15
|
+
See [Release Notes](https://github.com/meskyanichi/backup/wiki/Release-Notes) in the wiki for changes in the latest
|
|
16
|
+
version.
|
|
18
17
|
|
|
19
|
-
|
|
18
|
+
Backup supports Ruby versions 1.8.7, 1.9.2, 1.9.3 and 2.0.0.
|
|
20
19
|
|
|
21
|
-
|
|
20
|
+
## Overview
|
|
22
21
|
|
|
23
|
-
|
|
22
|
+
Backup allows you to _model_ your backup jobs using a Ruby DSL:
|
|
24
23
|
|
|
24
|
+
```rb
|
|
25
|
+
Backup::Model.new(:my_backup, 'Description for my_backup') do
|
|
26
|
+
# ... components here ...
|
|
27
|
+
end
|
|
28
|
+
```
|
|
25
29
|
|
|
26
|
-
|
|
30
|
+
The `:my_backup` symbol is the model's `trigger` and used to perform the job:
|
|
27
31
|
|
|
28
|
-
|
|
32
|
+
$ backup perform --trigger my_backup
|
|
29
33
|
|
|
30
|
-
|
|
31
|
-
|
|
34
|
+
Backup's _components_ are added to the backup _model_ to define the actions to be performed.
|
|
35
|
+
All of Backup's components are fully documented in the [Backup Wiki](https://github.com/meskyanichi/backup/wiki).
|
|
36
|
+
The following is brief overview of the components Backup provides:
|
|
32
37
|
|
|
33
|
-
|
|
38
|
+
### Archives and Databases
|
|
34
39
|
|
|
35
|
-
|
|
40
|
+
[Archives](https://github.com/meskyanichi/backup/wiki/Archives) create basic `tar` archives. Both **GNU** and **BSD**
|
|
41
|
+
`tar` are supported.
|
|
42
|
+
[Databases](https://github.com/meskyanichi/backup/wiki/Databases) create backups of one of the following supported databases:
|
|
36
43
|
|
|
37
44
|
- MySQL
|
|
38
45
|
- PostgreSQL
|
|
@@ -40,78 +47,76 @@ Below you find a list of components that Backup currently supports. If you'd lik
|
|
|
40
47
|
- Redis
|
|
41
48
|
- Riak
|
|
42
49
|
|
|
43
|
-
|
|
50
|
+
Any number of Archives and Databases may be defined within a backup _model_.
|
|
44
51
|
|
|
45
|
-
###
|
|
52
|
+
### Compressors and Encryptors
|
|
46
53
|
|
|
47
|
-
|
|
48
|
-
|
|
54
|
+
Adding a [Compressor](https://github.com/meskyanichi/backup/wiki/Compressors) to your backup will compress all the
|
|
55
|
+
Archives and Database backups within your final archive package.
|
|
56
|
+
`Gzip`, `Bzip2` and other similar compressors are supported.
|
|
49
57
|
|
|
50
|
-
|
|
58
|
+
Adding a [Encryptor](https://github.com/meskyanichi/backup/wiki/Encryptors) allows you to encrypt your final backup package.
|
|
59
|
+
Both `OpenSSL` and `GPG` are supported.
|
|
51
60
|
|
|
52
|
-
|
|
61
|
+
Your final backup _package_ might look something like this:
|
|
62
|
+
|
|
63
|
+
```text
|
|
64
|
+
$ gpg --decrypt my_backup.tar.gpg --outfile my_backup.tar
|
|
65
|
+
$ tar -tvf my_backup.tar
|
|
66
|
+
my_backup/
|
|
67
|
+
my_backup/archives/
|
|
68
|
+
my_backup/archives/user_avatars.tar.gz
|
|
69
|
+
my_backup/archives/log_files.tar.gz
|
|
70
|
+
my_backup/databases/
|
|
71
|
+
my_backup/databases/PostgreSQL/
|
|
72
|
+
my_backup/databases/PostgreSQL/pg_db_name.sql.gz
|
|
73
|
+
my_backup/databases/Redis/
|
|
74
|
+
my_backup/databases/Redis/redis_db_name.rdb.gz
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### Storages
|
|
78
|
+
|
|
79
|
+
Once your final backup package is ready, you can use any number of the following
|
|
80
|
+
[Storages](https://github.com/meskyanichi/backup/wiki/Storages) to store it:
|
|
53
81
|
|
|
54
82
|
- Amazon Simple Storage Service (S3)
|
|
55
83
|
- Rackspace Cloud Files (Mosso)
|
|
56
84
|
- Ninefold Cloud Storage
|
|
57
85
|
- Dropbox Web Service
|
|
58
|
-
- Remote Servers
|
|
59
|
-
- Local Storage
|
|
60
|
-
|
|
61
|
-
[Storage Wiki Page](https://github.com/meskyanichi/backup/wiki/Storages)
|
|
62
|
-
|
|
63
|
-
### Storage Features
|
|
86
|
+
- Remote Servers _(Available Protocols: FTP, SFTP, SCP and RSync)_
|
|
87
|
+
- Local Storage _(including network mounted locations)_
|
|
64
88
|
|
|
65
|
-
|
|
66
|
-
- Amazon Simple Storage Service (S3)
|
|
67
|
-
- Rackspace Cloud Files (Mosso)
|
|
68
|
-
- Ninefold Cloud Storage
|
|
69
|
-
- Dropbox Web Service
|
|
70
|
-
- Remote Servers *(Only Protocols: FTP, SFTP, SCP)*
|
|
71
|
-
- Local Storage
|
|
89
|
+
All of the above Storages _(except RSync)_ support:
|
|
72
90
|
|
|
73
|
-
[Cycling
|
|
91
|
+
- [Cycling](https://github.com/meskyanichi/backup/wiki/Cycling) to keep and rotate multiple copies
|
|
92
|
+
of your stored backups.
|
|
74
93
|
|
|
75
|
-
-
|
|
76
|
-
|
|
77
|
-
- Rackspace Cloud Files (Mosso)
|
|
78
|
-
- Ninefold Cloud Storage
|
|
79
|
-
- Dropbox Web Service
|
|
80
|
-
- Remote Servers *(Only Protocols: FTP, SFTP, SCP)*
|
|
81
|
-
- Local Storage
|
|
94
|
+
- [Splitter](https://github.com/meskyanichi/backup/wiki/Splitter) to break up a large
|
|
95
|
+
backup package into smaller files.
|
|
82
96
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
- **Incremental Backups, applies to:**
|
|
86
|
-
- Remote Servers *(Only Protocols: RSync)*
|
|
97
|
+
When using the RSync Storage, once a full backup has been stored, subsequent backups only need to
|
|
98
|
+
transmit the changed portions of the final archive to bring the remote copy up-to-date.
|
|
87
99
|
|
|
88
100
|
### Syncers
|
|
89
101
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
- Rackspce Cloud Files
|
|
93
|
-
|
|
94
|
-
[Syncer Wiki Page](https://github.com/meskyanichi/backup/wiki/Syncers)
|
|
102
|
+
[Syncers](https://github.com/meskyanichi/backup/wiki/Syncers) are processed after your final backup archive has been
|
|
103
|
+
stored and allow you to perform file synchronization.
|
|
95
104
|
|
|
96
|
-
|
|
105
|
+
Backup includes two types of Syncers:
|
|
97
106
|
|
|
98
|
-
-
|
|
99
|
-
-
|
|
100
|
-
- Pbzip2
|
|
101
|
-
- Lzma
|
|
107
|
+
- `RSync`: Used to sync files locally, local-to-remote (`Push`), or remote-to-local (`Pull`).
|
|
108
|
+
- `Cloud`: Used to sync files to remote storage services like Amazon S3 and Rackspace.
|
|
102
109
|
|
|
103
|
-
|
|
110
|
+
A backup _model_ may contain _only_ Syncers as well.
|
|
104
111
|
|
|
105
|
-
###
|
|
112
|
+
### Notifiers
|
|
106
113
|
|
|
107
|
-
|
|
108
|
-
|
|
114
|
+
[Notifiers](https://github.com/meskyanichi/backup/wiki/Notifiers) are used to send notifications upon successful and/or
|
|
115
|
+
failed completion of your backup _model_.
|
|
109
116
|
|
|
110
|
-
|
|
117
|
+
Supported notification services include:
|
|
111
118
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
- Mail
|
|
119
|
+
- Email _(SMTP, Sendmail, Exim and File delivery)_
|
|
115
120
|
- Twitter
|
|
116
121
|
- Campfire
|
|
117
122
|
- Presently
|
|
@@ -119,360 +124,108 @@ Below you find a list of components that Backup currently supports. If you'd lik
|
|
|
119
124
|
- Hipchat
|
|
120
125
|
- Pushover
|
|
121
126
|
|
|
122
|
-
[Notifiers Wiki Page](https://github.com/meskyanichi/backup/wiki/Notifiers)
|
|
123
|
-
|
|
124
|
-
### Supported Ruby versions (Tested with RSpec)
|
|
125
|
-
|
|
126
|
-
- Ruby 1.9.3
|
|
127
|
-
- Ruby 1.9.2
|
|
128
|
-
- Ruby 1.8.7
|
|
129
|
-
|
|
130
127
|
|
|
131
|
-
|
|
132
|
-
----------------------------------
|
|
128
|
+
## Generators
|
|
133
129
|
|
|
134
|
-
|
|
135
|
-
Backup has a [great wiki](https://github.com/meskyanichi/backup/wiki) which explains each component of Backup in detail.
|
|
130
|
+
Backup makes it easy to setup new backup _model_ files with it's [Generator](https://github.com/meskyanichi/backup/wiki/Generator) command.
|
|
136
131
|
|
|
137
|
-
```
|
|
138
|
-
|
|
132
|
+
```
|
|
133
|
+
$ backup generate:model -t my_backup --archives --databases=postgresql,redis --compressors=gzip \
|
|
134
|
+
--encryptors=gpg --storages=sftp,s3 --notifiers=mail,twitter
|
|
135
|
+
```
|
|
139
136
|
|
|
140
|
-
|
|
137
|
+
Simply generate a new _model_ using the options you need, then update the configuration for each component using the
|
|
138
|
+
[Wiki](https://github.com/meskyanichi/backup/wiki) documentation.
|
|
141
139
|
|
|
142
|
-
|
|
143
|
-
database.name = 'my_sample_mysql_db'
|
|
144
|
-
database.username = 'my_username'
|
|
145
|
-
database.password = 'my_password'
|
|
146
|
-
database.skip_tables = ['logs']
|
|
147
|
-
database.additional_options = ['--single-transaction', '--quick']
|
|
148
|
-
end
|
|
140
|
+
The following is an example of a what this Backup _model_ might look like:
|
|
149
141
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
end
|
|
142
|
+
```rb
|
|
143
|
+
Backup::Model.new(:my_backup, 'Description for my_backup') do
|
|
144
|
+
split_into_chunks_of 250
|
|
154
145
|
|
|
155
146
|
archive :user_avatars do |archive|
|
|
156
147
|
archive.add '/var/apps/my_sample_app/public/avatars'
|
|
157
148
|
end
|
|
158
149
|
|
|
159
|
-
archive :
|
|
160
|
-
archive.add '/var/apps/my_sample_app/logs
|
|
161
|
-
archive.
|
|
162
|
-
archive.add '/var/apps/my_sample_app/logs/other/'
|
|
163
|
-
archive.exclude '/var/apps/my_sample_app/logs/other/exclude-this.log'
|
|
150
|
+
archive :log_files do |archive|
|
|
151
|
+
archive.add '/var/apps/my_sample_app/logs'
|
|
152
|
+
archive.exclude '/var/apps/my_sample_app/logs/exclude-this.log'
|
|
164
153
|
end
|
|
165
154
|
|
|
166
|
-
|
|
167
|
-
|
|
155
|
+
database PostgreSQL do |db|
|
|
156
|
+
db.name = "pg_db_name"
|
|
157
|
+
db.username = "username"
|
|
158
|
+
db.password = "password"
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
database Redis do |db|
|
|
162
|
+
db.name = "redis_db_name"
|
|
163
|
+
db.path = "/usr/local/var/db/redis"
|
|
164
|
+
db.password = "password"
|
|
165
|
+
db.invoke_save = true
|
|
168
166
|
end
|
|
169
167
|
|
|
170
168
|
compress_with Gzip
|
|
171
169
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
server.ip = 'a.my-backup-server.com'
|
|
176
|
-
server.port = 22
|
|
177
|
-
server.path = '~/backups'
|
|
178
|
-
server.keep = 25
|
|
170
|
+
encrypt_with GPG do |encryption|
|
|
171
|
+
encryption.mode = :symmetric
|
|
172
|
+
encryption.passphrase = 'my_password'
|
|
179
173
|
end
|
|
180
174
|
|
|
181
|
-
store_with SFTP
|
|
182
|
-
server.username
|
|
183
|
-
server.password
|
|
184
|
-
server.ip
|
|
185
|
-
server.port
|
|
186
|
-
server.path
|
|
187
|
-
server.keep
|
|
175
|
+
store_with SFTP do |server|
|
|
176
|
+
server.username = "my_username"
|
|
177
|
+
server.password = "my_password"
|
|
178
|
+
server.ip = "123.45.678.90"
|
|
179
|
+
server.port = 22
|
|
180
|
+
server.path = "~/backups/"
|
|
181
|
+
server.keep = 5
|
|
188
182
|
end
|
|
189
183
|
|
|
190
184
|
store_with S3 do |s3|
|
|
191
|
-
s3.access_key_id = 'my_access_key_id'
|
|
192
|
-
s3.secret_access_key = 'my_secret_access_key'
|
|
193
|
-
s3.region = 'us-east-1'
|
|
194
|
-
s3.bucket = 'my_bucket/backups'
|
|
195
|
-
s3.keep = 20
|
|
196
|
-
end
|
|
197
|
-
|
|
198
|
-
sync_with Cloud::S3 do |s3|
|
|
199
185
|
s3.access_key_id = "my_access_key_id"
|
|
200
186
|
s3.secret_access_key = "my_secret_access_key"
|
|
201
|
-
s3.
|
|
202
|
-
s3.
|
|
203
|
-
s3.
|
|
204
|
-
|
|
205
|
-
s3.directories do |directory|
|
|
206
|
-
directory.add "/var/apps/my_app/public/videos"
|
|
207
|
-
directory.add "/var/apps/my_app/public/music"
|
|
208
|
-
end
|
|
187
|
+
s3.region = "us-east-1"
|
|
188
|
+
s3.bucket = "bucket-name"
|
|
189
|
+
s3.path = "/path/to/my/backups"
|
|
190
|
+
s3.keep = 10
|
|
209
191
|
end
|
|
210
192
|
|
|
211
193
|
notify_by Mail do |mail|
|
|
212
|
-
mail.on_success
|
|
213
|
-
|
|
214
|
-
mail.
|
|
194
|
+
mail.on_success = false
|
|
195
|
+
|
|
196
|
+
mail.from = "sender@email.com"
|
|
197
|
+
mail.to = "receiver@email.com"
|
|
198
|
+
mail.address = "smtp.gmail.com"
|
|
199
|
+
mail.port = 587
|
|
200
|
+
mail.user_name = "sender@email.com"
|
|
201
|
+
mail.password = "my_password"
|
|
202
|
+
mail.authentication = "plain"
|
|
203
|
+
mail.enable_starttls_auto = true
|
|
215
204
|
end
|
|
216
205
|
|
|
217
206
|
notify_by Twitter do |tweet|
|
|
218
|
-
tweet.
|
|
219
|
-
tweet.
|
|
220
|
-
tweet.
|
|
207
|
+
tweet.consumer_key = "my_consumer_key"
|
|
208
|
+
tweet.consumer_secret = "my_consumer_secret"
|
|
209
|
+
tweet.oauth_token = "my_oauth_token"
|
|
210
|
+
tweet.oauth_token_secret = "my_oauth_token_secret"
|
|
221
211
|
end
|
|
222
|
-
|
|
223
212
|
end
|
|
224
213
|
```
|
|
225
214
|
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
First, it will dump the two Databases (MySQL and MongoDB). The MySQL dump will be piped through the Gzip Compressor into
|
|
229
|
-
`sample_backup/databases/MySQL/my_sample_mysql_db.sql.gz`. The MongoDB dump will be dumped into
|
|
230
|
-
`sample_backup/databases/MongoDB/`, which will then be packaged into `sample_backup/databases/MongoDB-#####.tar.gz`
|
|
231
|
-
(`#####` will be a simple unique identifier, in case multiple dumps are performed.)
|
|
232
|
-
Next, it will create two _tar_ Archives (user\_avatars and logs). Each will be piped through the Gzip Compressor into
|
|
233
|
-
`sample_backup/archives/` as `user_archives.tar.gz` and `logs.tar.gz`.
|
|
234
|
-
Finally, the `sample_backup` directory will be packaged into an uncompressed _tar_ archive, which will be piped through
|
|
235
|
-
the OpenSSL Encryptor to encrypt this final package into `YYYY-MM-DD-hh-mm-ss.sample_backup.tar.enc`. This final
|
|
236
|
-
encrypted archive will then be transfered to your Amazon S3 account. If all goes well, and no exceptions are raised,
|
|
237
|
-
you'll be notified via the Twitter notifier that the backup succeeded. If any warnings were issued or there was an
|
|
238
|
-
exception raised during the backup process, then you'd receive an email in your inbox containing detailed exception
|
|
239
|
-
information, as well as receive a simple Twitter message that something went wrong.
|
|
240
|
-
|
|
241
|
-
Aside of S3, we have also defined two `SFTP` storage methods, and given them two unique identifiers `Server A` and
|
|
242
|
-
`Server B` to distinguish between the two. With these in place, a copy of the backup will now also be stored on two
|
|
243
|
-
separate servers: `a.my-backup-server.com` and `b.my-backup-server.com`.
|
|
244
|
-
|
|
245
|
-
As you can see, you can freely mix and match **archives**, **databases**, **compressors**, **encryptors**, **storages**
|
|
246
|
-
and **notifiers** for your backups. You could even specify 4 storage locations if you wanted: Amazon S3, Rackspace Cloud
|
|
247
|
-
Files, Ninefold and Dropbox, it'd then store your packaged backup to 4 separate locations for high redundancy.
|
|
248
|
-
|
|
249
|
-
Also, notice the `split_into_chunks_of 4000` at the top of the configuration. This tells Backup to split any backups
|
|
250
|
-
that exceed in 4000 MEGABYTES of size in to multiple smaller chunks. Assuming your backup file is 12000 MEGABYTES (12GB)
|
|
251
|
-
in size, then Backup will take the output which was piped from _tar_ into the OpenSSL Compressor and additionally pipe
|
|
252
|
-
that output through the _split_ utility, which will result in 3 chunks of 4000 MEGABYTES with additional file extensions
|
|
253
|
-
of `-aa`, `-ab` and `-ac`. These files will then be individually transfered. This is useful for when you are using
|
|
254
|
-
Amazon S3, Rackspace Cloud Files, or other 3rd party storage services which limit you to "5GB per file" uploads. So with
|
|
255
|
-
this, the backup file size is no longer a constraint.
|
|
256
|
-
|
|
257
|
-
Additionally we have also defined a **S3 Syncer** ( `sync_with Cloud::S3` ), which does not follow the above process of
|
|
258
|
-
archiving/compression/encryption, but instead will directly sync the whole `videos` and `music` folder structures from
|
|
259
|
-
your machine to your Amazon S3 account. (very efficient and cost-effective since it will only transfer files that were
|
|
260
|
-
added/changed. Additionally, since we flagged it to 'mirror', it'll also remove files from S3 that no longer exist). If
|
|
261
|
-
you simply wanted to sync to a separate backup server that you own, you could also use the RSync syncer for even more
|
|
262
|
-
efficient backups that only transfer the **bytes** of each file that changed.
|
|
263
|
-
|
|
264
|
-
There are more **archives**, **databases**, **compressors**, **encryptors**, **storages** and **notifiers** than
|
|
265
|
-
displayed in the example, all available components are listed at the top of this README, as well as in the
|
|
266
|
-
[Wiki](https://github.com/meskyanichi/backup/wiki) with more detailed information.
|
|
267
|
-
|
|
268
|
-
### Running the example
|
|
269
|
-
|
|
270
|
-
Notice the `Backup::Model.new(:sample_backup, 'A sample backup configuration') do` at the top of the above example. The
|
|
271
|
-
`:sample_backup` is called the **trigger**. This is used to identify the backup procedure/file and initialize it.
|
|
272
|
-
|
|
273
|
-
``` sh
|
|
274
|
-
backup perform -t [--trigger] sample_backup
|
|
275
|
-
```
|
|
215
|
+
The [Getting Started](https://github.com/meskyanichi/backup/wiki/Getting-Started) page provides a simple
|
|
216
|
+
walk-through to familiarize you with setting up, configuring and running a backup job.
|
|
276
217
|
|
|
277
|
-
|
|
218
|
+
## Suggestions, Issues, etc...
|
|
278
219
|
|
|
279
|
-
|
|
220
|
+
If you have any suggestions or problems, please submit an Issue or Pull Request using Backup's
|
|
221
|
+
[Issue Log](https://github.com/meskyanichi/backup/issues).
|
|
280
222
|
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
using pure Ruby, and it provides an elegant DSL to do so. Here's an example:
|
|
223
|
+
If you find any errors or omissions in Backup's documentation [Wiki](https://github.com/meskyanichi/backup/wiki),
|
|
224
|
+
please feel free to edit it!
|
|
284
225
|
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
end
|
|
289
|
-
```
|
|
226
|
+
Backup has seen many improvements over the years thanks to it's
|
|
227
|
+
[Contributors](https://github.com/meskyanichi/backup/contributors), as well as those who have help discuss issues and
|
|
228
|
+
improve the documentation, and looks forward to continuing to provide users with a reliable backup solution.
|
|
290
229
|
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
Whenever project page for more information.
|
|
294
|
-
|
|
295
|
-
### Documentation
|
|
296
|
-
|
|
297
|
-
See the [Wiki Pages](https://github.com/meskyanichi/backup/wiki).
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
### Suggestions, Bugs, Requests, Questions
|
|
301
|
-
|
|
302
|
-
View the [issue log](https://github.com/meskyanichi/backup/issues) and post them there.
|
|
303
|
-
|
|
304
|
-
### Contributors
|
|
305
|
-
|
|
306
|
-
<table>
|
|
307
|
-
<tr>
|
|
308
|
-
<th>Contributor</th>
|
|
309
|
-
<th>Contribution</th>
|
|
310
|
-
</tr>
|
|
311
|
-
<tr>
|
|
312
|
-
<td><a href="https://github.com/burns" target="_blank"><b>Brian D. Burns ( burns )</b></a></td>
|
|
313
|
-
<td><b>Core Contributor</b></td>
|
|
314
|
-
</tr>
|
|
315
|
-
<tr>
|
|
316
|
-
<td><a href="https://github.com/asanghi" target="_blank">Aditya Sanghi ( asanghi )</a></td>
|
|
317
|
-
<td>Twitter Notifier, Dropbox Timeout Configuration</td>
|
|
318
|
-
</tr>
|
|
319
|
-
<tr>
|
|
320
|
-
<td><a href="https://github.com/phlipper" target="_blank">Phil Cohen ( phlipper )</a></td>
|
|
321
|
-
<td>Exclude Option for Archives</td>
|
|
322
|
-
</tr>
|
|
323
|
-
<tr>
|
|
324
|
-
<td><a href="https://github.com/arunagw" target="_blank">Arun Agrawal ( arunagw )</a></td>
|
|
325
|
-
<td>Campfire notifier</td>
|
|
326
|
-
</tr>
|
|
327
|
-
<tr>
|
|
328
|
-
<td><a href="https://github.com/szimmermann" target="_blank">Stefan Zimmermann ( szimmermann )</a></td>
|
|
329
|
-
<td>Enabling package/archive (tar utility) support for more Linux distro's (FreeBSD, etc)</td>
|
|
330
|
-
</tr>
|
|
331
|
-
<tr>
|
|
332
|
-
<td><a href="https://github.com/trystant" target="_blank">Mark Nyon ( trystant )</a></td>
|
|
333
|
-
<td>Helping discuss MongoDump Lock/FSync problem</td>
|
|
334
|
-
</tr>
|
|
335
|
-
<tr>
|
|
336
|
-
<td><a href="https://github.com/imanel" target="_blank">Bernard Potocki ( imanel )</a></td>
|
|
337
|
-
<td>Helping discuss MongoDump Lock/FSync problem + Submitting a patch</td>
|
|
338
|
-
</tr>
|
|
339
|
-
<tr>
|
|
340
|
-
<td><a href="https://github.com/tomash" target="_blank">Tomasz Stachewicz ( tomash )</a></td>
|
|
341
|
-
<td>Helping discuss MongoDump Lock/FSync problem + Submitting a patch</td>
|
|
342
|
-
</tr>
|
|
343
|
-
<tr>
|
|
344
|
-
<td><a href="https://github.com/lapluviosilla" target="_blank">Paul Strong ( lapluviosilla )</a></td>
|
|
345
|
-
<td>Helping discuss MongoDump Lock/FSync problem</td>
|
|
346
|
-
</tr>
|
|
347
|
-
<tr>
|
|
348
|
-
<td><a href="https://github.com/rgnitz" target="_blank">Ryan ( rgnitz )</a></td>
|
|
349
|
-
<td>Helping discuss MongoDump Lock/FSync problem</td>
|
|
350
|
-
</tr>
|
|
351
|
-
<tr>
|
|
352
|
-
<td><a href="https://github.com/tsigo" target="_blank">Robert Speicher ( tsigo )</a></td>
|
|
353
|
-
<td>Adding the --quiet [-q] feature to Backup to silence console logging</td>
|
|
354
|
-
</tr>
|
|
355
|
-
<tr>
|
|
356
|
-
<td><a href="https://github.com/jwhitcraft" target="_blank">Jon Whitcraft ( jwhitcraft )</a></td>
|
|
357
|
-
<td>Adding the ability to add additional options to the S3Syncer</td>
|
|
358
|
-
</tr>
|
|
359
|
-
<tr>
|
|
360
|
-
<td><a href="https://github.com/bgarret" target="_blank">Benoit Garret ( bgarret )</a></td>
|
|
361
|
-
<td>Presently notifier</td>
|
|
362
|
-
</tr>
|
|
363
|
-
<tr>
|
|
364
|
-
<td><a href="https://github.com/lleirborras" target="_blank">Lleïr Borràs Metje ( lleirborras )</a></td>
|
|
365
|
-
<td>Lzma Compressor</td>
|
|
366
|
-
</tr>
|
|
367
|
-
<tr>
|
|
368
|
-
<td><a href="https://github.com/jof" target="_blank">Jonathan Lassoff ( jof )</a></td>
|
|
369
|
-
<td>Bugfixes and more secure GPG storage</td>
|
|
370
|
-
</tr>
|
|
371
|
-
<tr>
|
|
372
|
-
<td><a href="https://github.com/mikz" target="_blank">Michal Cichra ( mikz )</a></td>
|
|
373
|
-
<td>Wildcard Triggers</td>
|
|
374
|
-
</tr>
|
|
375
|
-
<tr>
|
|
376
|
-
<td><a href="https://github.com/trybeee" target="_blank">Dmitry Novotochinov ( trybeee )</a></td>
|
|
377
|
-
<td>Dropbox Storage</td>
|
|
378
|
-
</tr>
|
|
379
|
-
<tr>
|
|
380
|
-
<td><a href="https://github.com/Emerson" target="_blank">Emerson Lackey ( Emerson )</a></td>
|
|
381
|
-
<td>Local RSync Storage</td>
|
|
382
|
-
</tr>
|
|
383
|
-
<tr>
|
|
384
|
-
<td><a href="https://github.com/digilord" target="_blank">digilord</a></td>
|
|
385
|
-
<td>OpenSSL Verify Mode for Mail Notifier</td>
|
|
386
|
-
</tr>
|
|
387
|
-
<tr>
|
|
388
|
-
<td><a href="https://github.com/stemps" target="_blank">stemps</a></td>
|
|
389
|
-
<td>FTP Passive Mode</td>
|
|
390
|
-
</tr>
|
|
391
|
-
<tr>
|
|
392
|
-
<td><a href="https://github.com/dkowis" target="_blank">David Kowis ( dkowis )</a></td>
|
|
393
|
-
<td>Fixed PostgreSQL Password issues</td>
|
|
394
|
-
</tr>
|
|
395
|
-
<tr>
|
|
396
|
-
<td><a href="https://github.com/jotto" target="_blank">Jonathan Otto ( jotto )</a></td>
|
|
397
|
-
<td>Allow for running PostgreSQL as another UNIX user</td>
|
|
398
|
-
</tr>
|
|
399
|
-
<tr>
|
|
400
|
-
<td><a href="https://github.com/joaovitor" target="_blank">João Vitor ( joaovitor )</a></td>
|
|
401
|
-
<td>Changed default PostgreSQL example options to appropriate ones</td>
|
|
402
|
-
</tr>
|
|
403
|
-
<tr>
|
|
404
|
-
<td><a href="https://github.com/swissmanu" target="_blank">Manuel Alabor ( swissmanu )</a></td>
|
|
405
|
-
<td>Prowl Notifier</td>
|
|
406
|
-
</tr>
|
|
407
|
-
<tr>
|
|
408
|
-
<td><a href="https://github.com/josephcrim" target="_blank">Joseph Crim ( josephcrim )</a></td>
|
|
409
|
-
<td>Riak Database, exit() suggestions</td>
|
|
410
|
-
</tr>
|
|
411
|
-
<tr>
|
|
412
|
-
<td><a href="https://github.com/fearoffish" target="_blank">Jamie van Dyke ( fearoffish )</a></td>
|
|
413
|
-
<td>POpen4 implementation</td>
|
|
414
|
-
</tr>
|
|
415
|
-
<tr>
|
|
416
|
-
<td><a href="https://github.com/hmarr" target="_blank">Harry Marr ( hmarr )</a></td>
|
|
417
|
-
<td>Auth URL for Rackspace Cloud Files Storage</td>
|
|
418
|
-
</tr>
|
|
419
|
-
<tr>
|
|
420
|
-
<td><a href="https://github.com/manuelmeurer" target="_blank">Manuel Meurer ( manuelmeurer )</a></td>
|
|
421
|
-
<td>Ensure the storage file (YAML dump) has content before reading it</td>
|
|
422
|
-
</tr>
|
|
423
|
-
<tr>
|
|
424
|
-
<td><a href="https://github.com/jessedearing" target="_blank">Jesse Dearing ( jessedearing )</a></td>
|
|
425
|
-
<td>Hipchat Notifier</td>
|
|
426
|
-
</tr>
|
|
427
|
-
<tr>
|
|
428
|
-
<td><a href="https://github.com/szymonpk" target="_blank">Szymon ( szymonpk )</a></td>
|
|
429
|
-
<td>Pbzip2 compressor</td>
|
|
430
|
-
</tr>
|
|
431
|
-
<tr>
|
|
432
|
-
<td><a href="https://github.com/SteveNewson" target="_blank">Steve Newson ( SteveNewson )</a></td>
|
|
433
|
-
<td>Pushover Notifier</td>
|
|
434
|
-
</tr>
|
|
435
|
-
</table>
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
### Want to contribute?
|
|
439
|
-
|
|
440
|
-
- Fork the project
|
|
441
|
-
- Write RSpec tests, and test against:
|
|
442
|
-
- Ruby 1.9.3
|
|
443
|
-
- Ruby 1.9.2
|
|
444
|
-
- Ruby 1.8.7
|
|
445
|
-
- Try to keep the overall *structure / design* of the gem the same
|
|
446
|
-
|
|
447
|
-
I can't guarantee I'll pull every pull request. Also, I may accept your pull request and drastically change parts to improve readability/maintainability. Feel free to discuss about improvements, new functionality/features in the [issue log](https://github.com/meskyanichi/backup/issues) before contributing if you need/want more information.
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
### Easily run tests against all three Ruby versions
|
|
451
|
-
|
|
452
|
-
Install [RVM](https://rvm.beginrescueend.com/) and use it to install Ruby 1.9.3, 1.9.2 and 1.8.7.
|
|
453
|
-
|
|
454
|
-
rvm get latest && rvm reload
|
|
455
|
-
rvm install 1.9.3 && rvm install 1.9.2 && rvm install 1.8.7
|
|
456
|
-
|
|
457
|
-
Once these are installed, go ahead and install all the necessary dependencies.
|
|
458
|
-
|
|
459
|
-
cd backup
|
|
460
|
-
rvm use 1.9.3 && gem install bundler && bundle install
|
|
461
|
-
rvm use 1.9.2 && gem install bundler && bundle install
|
|
462
|
-
rvm use 1.8.7 && gem install bundler && bundle install
|
|
463
|
-
|
|
464
|
-
The Backup gem uses [Guard](https://github.com/guard/guard) along with [Guard::RSpec](https://github.com/guard/guard-rspec) to quickly and easily test Backup's code against all four Rubies. If you've done the above, all you have to do is run:
|
|
465
|
-
|
|
466
|
-
bundle exec guard
|
|
467
|
-
|
|
468
|
-
from Backup's root and that's it. It'll now test against all Ruby versions each time you adjust a file in the `lib` or `spec` directories.
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
### Or contribute by writing blogs/tutorials/use cases
|
|
472
|
-
|
|
473
|
-
- http://freelancing-gods.com/posts/backing_up_with_backup
|
|
474
|
-
- http://erik.debill.org/2011/03/26/csing-backup-with-rails
|
|
475
|
-
- http://blog.noizeramp.com/2011/03/31/backing-up-backup-ruby-gem/
|
|
476
|
-
- http://www.sebaugereau.com/using-ruby-to-backup-with-beauty
|
|
477
|
-
- http://outofti.me/post/4159686269/backup-with-pgbackups
|
|
478
|
-
- http://h2ik.co/2011/03/backing-up-with-ruby/
|
|
230
|
+
**Copyright (c) 2009-2013 [Michael van Rooijen](http://michaelvanrooijen.com/) ( [@meskyanichi](http://twitter.com/#!/meskyanichi) )**
|
|
231
|
+
Released under the **MIT** [License](LICENSE.md).
|