synco 1.1.0 → 1.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- checksums.yaml.gz.sig +3 -0
- data/bin/synco +3 -3
- data/lib/synco/command/disk.rb +8 -23
- data/lib/synco/command/prune.rb +10 -25
- data/lib/synco/command/rotate.rb +46 -61
- data/lib/synco/command/spawn.rb +10 -25
- data/lib/synco/command.rb +23 -35
- data/lib/synco/controller.rb +4 -19
- data/lib/synco/directory.rb +8 -23
- data/lib/synco/disk.rb +4 -19
- data/lib/synco/method.rb +11 -26
- data/lib/synco/methods/rsync.rb +14 -29
- data/lib/synco/methods/scp.rb +8 -23
- data/lib/synco/methods/zfs.rb +7 -22
- data/lib/synco/scope.rb +33 -68
- data/lib/synco/script.rb +15 -32
- data/lib/synco/server.rb +9 -24
- data/lib/synco/shell.rb +5 -20
- data/lib/synco/shells/ssh.rb +10 -25
- data/lib/synco/version.rb +5 -20
- data/lib/synco.rb +13 -29
- data/license.md +21 -0
- data/readme.md +248 -0
- data.tar.gz.sig +0 -0
- metadata +60 -120
- metadata.gz.sig +0 -0
- data/.gitignore +0 -22
- data/.rspec +0 -4
- data/.simplecov +0 -9
- data/.travis.yml +0 -14
- data/Gemfile +0 -11
- data/README.md +0 -246
- data/Rakefile +0 -8
- data/lib/synco/compact_formatter.rb +0 -115
- data/media/LSync Logo.artx/Preview/preview.png +0 -0
- data/media/LSync Logo.artx/QuickLook/Preview.pdf +0 -0
- data/media/LSync Logo.artx/doc.thread +0 -0
- data/media/LSync Logo.png +0 -0
- data/spec/synco/backup_script.rb +0 -63
- data/spec/synco/directory_spec.rb +0 -33
- data/spec/synco/local_backup.rb +0 -56
- data/spec/synco/local_sync.rb +0 -91
- data/spec/synco/method_spec.rb +0 -62
- data/spec/synco/rsync_spec.rb +0 -89
- data/spec/synco/scp_spec.rb +0 -58
- data/spec/synco/script_spec.rb +0 -51
- data/spec/synco/shell_spec.rb +0 -42
- data/spec/synco/usb_spec.rb +0 -76
- data/spec/synco/zfs_spec.rb +0 -50
- data/synco.gemspec +0 -35
data/README.md
DELETED
@@ -1,246 +0,0 @@
|
|
1
|
-
# Synco
|
2
|
-
|
3
|
-
Synco is a tool for scripted synchronization and backups. It provides a custom Ruby DSL for describing backup and synchronization tasks involving one more more system and disk. It is designed to provide flexibility while reducing the complexity multi-server backups.
|
4
|
-
|
5
|
-
* Single and multi-server data synchronization.
|
6
|
-
* Incremental backups both locally and remotely.
|
7
|
-
* Backup staging and coordination.
|
8
|
-
* Backup verification using [Fingerprint](https://github.com/ioquatix/fingerprint).
|
9
|
-
* Data backup redundancy controlled via DNS.
|
10
|
-
|
11
|
-
[![Build Status](https://secure.travis-ci.org/ioquatix/synco.svg)](http://travis-ci.org/ioquatix/synco)
|
12
|
-
[![Code Climate](https://codeclimate.com/github/ioquatix/synco.svg)](https://codeclimate.com/github/ioquatix/synco)
|
13
|
-
[![Coverage Status](https://coveralls.io/repos/ioquatix/synco/badge.svg)](https://coveralls.io/r/ioquatix/synco)
|
14
|
-
|
15
|
-
## Installation
|
16
|
-
|
17
|
-
Add this line to your application's Gemfile:
|
18
|
-
|
19
|
-
gem 'synco'
|
20
|
-
|
21
|
-
And then execute:
|
22
|
-
|
23
|
-
$ bundle
|
24
|
-
|
25
|
-
Or install it yourself as:
|
26
|
-
|
27
|
-
$ gem install synco
|
28
|
-
|
29
|
-
## Usage
|
30
|
-
|
31
|
-
Synco imposes a particular structure regarding the organisation of backup scripts: Backup scripts involve a set of servers and directories. A server is a logical unit where files are available or stored. Directories are relative paths which are resolved relative to a server's root path.
|
32
|
-
|
33
|
-
A simple backup script might look something like this:
|
34
|
-
|
35
|
-
#!/usr/bin/env ruby
|
36
|
-
|
37
|
-
require 'synco'
|
38
|
-
require 'synco/methods/rsync'
|
39
|
-
|
40
|
-
Synco::run_script do |script|
|
41
|
-
script.method = Synco::Methods::RSync.new(archive: true)
|
42
|
-
|
43
|
-
server(:master) do |server|
|
44
|
-
server.host = "server.example.com"
|
45
|
-
server.root = "/"
|
46
|
-
end
|
47
|
-
|
48
|
-
server(:backup) do |server|
|
49
|
-
server.host = "backup.example.com"
|
50
|
-
server.root = "/tank/backups/server.example.com"
|
51
|
-
end
|
52
|
-
|
53
|
-
backup('etc', 'var', 'srv', 'home',
|
54
|
-
arguments: %W{--exclude cache/ --exclude tmp/}
|
55
|
-
)
|
56
|
-
end
|
57
|
-
|
58
|
-
This will produce an identical copy using rsync of the specified directories. As an example `server.example.com:/etc` would be copied to `backup.example.com:/tank/backups/server.example.com/etc`.
|
59
|
-
|
60
|
-
### RSync Snapshots
|
61
|
-
|
62
|
-
Building on the above backup, you can use `Synco::Methods::RSyncSnapshot` which supports snapshot based backups. It creates a snapshot into a sub-directory called `latest.snapshot` and uses RSync's `--link-dest` to hard-link files when unchanged. Synco provides scripts to rotate and prune these backups as required, but you must invoke them as part of the script:
|
63
|
-
|
64
|
-
server(:backup) do |server|
|
65
|
-
server.host = "backup.example.com"
|
66
|
-
server.root = "/"
|
67
|
-
|
68
|
-
server.on(:success) do
|
69
|
-
run "synco", "rotate", chdir: target_server.root
|
70
|
-
run "synco", "prune", chdir: target_server.root
|
71
|
-
end
|
72
|
-
end
|
73
|
-
|
74
|
-
These commands can also be run from the command line.
|
75
|
-
|
76
|
-
rotate [--format <name>] [--latest <name>] [--snapshot <name>]
|
77
|
-
Rotate a backup snapshot into a timestamped directory.
|
78
|
-
|
79
|
-
[--format <name>] Set the name of the backup rotations, including strftime expansions. Default: %Y.%m.%d-%H.%M.%S
|
80
|
-
[--latest <name>] The name of the latest backup symlink. Default: latest
|
81
|
-
[--snapshot <name>] The name of the in-progress backup snapshot. Default: latest.snapshot
|
82
|
-
|
83
|
-
prune [--hourly <count>] [--daily <count>] [--weekly <count>] [--monthly <count>] [--quarterly <count>] [--yearly <count>] [--format <name>] [--latest <name>] [--keep <new|old>] [--dry]
|
84
|
-
Prune old backups to reduce disk usage according to a given policy.
|
85
|
-
|
86
|
-
[--hourly <count>] Set the number of hourly backups to keep. Default: 24
|
87
|
-
[--daily <count>] Set the number of daily backups to keep. Default: 28
|
88
|
-
[--weekly <count>] Set the number of weekly backups to keep. Default: 52
|
89
|
-
[--monthly <count>] Set the number of monthly backups to keep. Default: 36
|
90
|
-
[--quarterly <count>] Set the number of quaterly backups to keep. Default: 40
|
91
|
-
[--yearly <count>] Set the number of yearly backups to keep. Default: 20
|
92
|
-
[--format <name>] Set the name of the backup rotations, including strftime expansions. Default: %Y.%m.%d-%H.%M.%S
|
93
|
-
[--latest <name>] The name of the latest backup symlink. Default: latest
|
94
|
-
[--keep <new|old>] Keep the younger or older backups within the same period division Default: old
|
95
|
-
[--dry] Print out what would be done rather than doing it.
|
96
|
-
|
97
|
-
### Mounting Disks
|
98
|
-
|
99
|
-
Synco supports mounting disks before the backup begins and unmounting them after done. The specifics of this process may require some adjustment based on your OS. For example on linux, `sudo` is used to invoke `mount` and `umount`.
|
100
|
-
|
101
|
-
server(:destination) do |server|
|
102
|
-
self.mountpoint = '/mnt/backups'
|
103
|
-
self.root = File.join(server.mountpoint, 'laptop')
|
104
|
-
|
105
|
-
server.on(:prepare) do
|
106
|
-
# synco mount uses labels, e.g. the disk partition has LABEL=backups
|
107
|
-
target_server.run "synco", "mount", target_server.mountpoint, 'backups'
|
108
|
-
end
|
109
|
-
|
110
|
-
server.on(:finish) do
|
111
|
-
target_server.run "synco", "unmount", target_server.mountpoint
|
112
|
-
end
|
113
|
-
end
|
114
|
-
|
115
|
-
On Linux, you might want to create the file `/etc/sudoers.d/synco` with the following contents:
|
116
|
-
|
117
|
-
%wheel ALL=(root) NOPASSWD: /bin/mount
|
118
|
-
%wheel ALL=(root) NOPASSWD: /bin/umount
|
119
|
-
|
120
|
-
Please make sure you take the time to educate yourself on the security of such a setup.
|
121
|
-
|
122
|
-
### Database Backups
|
123
|
-
|
124
|
-
If you'd like to dump data before running the backup, it's possible using the event handling mechanisms:
|
125
|
-
|
126
|
-
server(:master) do |server|
|
127
|
-
server.host = "server.example.com"
|
128
|
-
server.root = "/"
|
129
|
-
|
130
|
-
server.on(:prepare) do
|
131
|
-
# Dump MySQL to /srv/mysql
|
132
|
-
run '/etc/lsync/mysql-backup.sh'
|
133
|
-
end
|
134
|
-
end
|
135
|
-
|
136
|
-
The exact contents of `mysql-backup.sh` will depend on your requirements, but here is an example:
|
137
|
-
|
138
|
-
#!/usr/bin/env bash
|
139
|
-
|
140
|
-
BACKUP_DIR=/srv/mysql
|
141
|
-
MYSQL=/usr/bin/mysql
|
142
|
-
MYSQLDUMP=/usr/bin/mysqldump
|
143
|
-
|
144
|
-
databases=`mysql --user=backup -e "SHOW DATABASES;" | grep -Ev "(Database|information_schema|performance_schema|_test|_restore)"`
|
145
|
-
|
146
|
-
# http://stackoverflow.com/questions/451404/how-to-obtain-a-correct-dump-using-mysqldump-and-single-transaction-when-ddl-is
|
147
|
-
MYSQLDUMP_OPTIONS="--force --skip-opt --single-transaction --add-drop-table --create-options --quick --extended-insert --set-charset --disable-keys"
|
148
|
-
|
149
|
-
for db in $databases; do
|
150
|
-
echo "Dumping database $db to $BACKUP_DIR/$db.sql.xz..."
|
151
|
-
mysqldump --user=backup $MYSQLDUMP_OPTIONS --databases $db | xz > "$BACKUP_DIR/$db.sql.xz"
|
152
|
-
done
|
153
|
-
|
154
|
-
### Fingerprint Integration
|
155
|
-
|
156
|
-
It is possible to make a [cryptographic checksum of the data](https://github.com/ioquatix/fingerprint). On a filesystem that support immutable snapshots, you can do this before the data is copied. For traditional filesystems, you generally need to do this afterwards.
|
157
|
-
|
158
|
-
server(:master) do |server|
|
159
|
-
server.host = "server.example.com"
|
160
|
-
server.root = "/"
|
161
|
-
|
162
|
-
server.on(:success) do
|
163
|
-
# Run fingerprint on the backup data:
|
164
|
-
run 'fingerprint', '--root', target_server.root, 'analyze'
|
165
|
-
end
|
166
|
-
end
|
167
|
-
|
168
|
-
Fingerprint is used in many of the specs to verify file copies.
|
169
|
-
|
170
|
-
### ZFS Snapshots
|
171
|
-
|
172
|
-
*This part of Synco is still under heavy development*
|
173
|
-
|
174
|
-
Synco can manage synchronization and backups of ZFS partitions. However, to use the standard tools, it is necessary to enable `zfs_admin_snapshot`, in `/etc/modprobe.d/zfs.conf`:
|
175
|
-
|
176
|
-
options zfs zfs_admin_snapshot=1
|
177
|
-
|
178
|
-
Propagate user permissions for the ZFS partition:
|
179
|
-
|
180
|
-
sudo zfs allow -ld -u `whoami` create,mount,send,receive,snapshot tank/test
|
181
|
-
|
182
|
-
### Backup staging
|
183
|
-
|
184
|
-
Synco in a previous life supported backup staging. However, at this time it's not available except in a very limited form: backup scripts which use `Synco.run_script` use explicit locking so that it's not possible to run the same backup at the same time. In the future, staging sequential and parallel backups will be added.
|
185
|
-
|
186
|
-
### DNS Failover
|
187
|
-
|
188
|
-
**This behaviour is not well tested**
|
189
|
-
|
190
|
-
Synco uses DNS to resolve the master server. This allows for bi-directional synchronization and other interesting setups.
|
191
|
-
|
192
|
-
Firstly, a backup script defaults to the server with the name `:master` as the master, where data is replicated FROM.
|
193
|
-
|
194
|
-
However, it is possible instead to specify a hostname, e.g. `primary.example.com`. Then, specify several servers, e.g. `s01.example.com`, `s02.example.com` and so on:
|
195
|
-
|
196
|
-
Synco::run_script do |script|
|
197
|
-
script.method = Synco::Methods::RSync.new(archive: true)
|
198
|
-
|
199
|
-
script.master = "primary.example.com"
|
200
|
-
|
201
|
-
server("s01.example.com") do |server|
|
202
|
-
server.root = "/"
|
203
|
-
end
|
204
|
-
|
205
|
-
server("s02.example.com") do |server|
|
206
|
-
server.root = "/"
|
207
|
-
end
|
208
|
-
|
209
|
-
backup('srv/http',
|
210
|
-
arguments: %W{--exclude cache/ --exclude tmp/}
|
211
|
-
)
|
212
|
-
end
|
213
|
-
|
214
|
-
When you run the script, the behaviour will depend on whether `primary.example.com` points to `s01.example.com` or `s02.example.com`. The data will always be copied from the master server to the other servers.
|
215
|
-
|
216
|
-
## Contributing
|
217
|
-
|
218
|
-
1. Fork it
|
219
|
-
2. Create your feature branch (`git checkout -b my-new-feature`)
|
220
|
-
3. Commit your changes (`git commit -am 'Add some feature'`)
|
221
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
222
|
-
5. Create new Pull Request
|
223
|
-
|
224
|
-
## License
|
225
|
-
|
226
|
-
Released under the MIT license.
|
227
|
-
|
228
|
-
Copyright, 2016, by [Samuel G. D. Williams](http://www.codeotaku.com/samuel-williams).
|
229
|
-
|
230
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
231
|
-
of this software and associated documentation files (the "Software"), to deal
|
232
|
-
in the Software without restriction, including without limitation the rights
|
233
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
234
|
-
copies of the Software, and to permit persons to whom the Software is
|
235
|
-
furnished to do so, subject to the following conditions:
|
236
|
-
|
237
|
-
The above copyright notice and this permission notice shall be included in
|
238
|
-
all copies or substantial portions of the Software.
|
239
|
-
|
240
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
241
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
242
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
243
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
244
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
245
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
246
|
-
THE SOFTWARE.
|
data/Rakefile
DELETED
@@ -1,115 +0,0 @@
|
|
1
|
-
# Copyright, 2016, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
2
|
-
#
|
3
|
-
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
-
# of this software and associated documentation files (the "Software"), to deal
|
5
|
-
# in the Software without restriction, including without limitation the rights
|
6
|
-
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
7
|
-
# copies of the Software, and to permit persons to whom the Software is
|
8
|
-
# furnished to do so, subject to the following conditions:
|
9
|
-
#
|
10
|
-
# The above copyright notice and this permission notice shall be included in
|
11
|
-
# all copies or substantial portions of the Software.
|
12
|
-
#
|
13
|
-
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
-
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
-
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
16
|
-
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
17
|
-
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
18
|
-
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
19
|
-
# THE SOFTWARE.
|
20
|
-
|
21
|
-
require 'rainbow'
|
22
|
-
require 'shellwords'
|
23
|
-
|
24
|
-
module Synco
|
25
|
-
class CompactFormatter
|
26
|
-
def initialize(verbose: true)
|
27
|
-
@start = Time.now
|
28
|
-
@verbose = verbose
|
29
|
-
end
|
30
|
-
|
31
|
-
def time_offset_prefix
|
32
|
-
offset = Time.now - @start
|
33
|
-
minutes = (offset/60).floor
|
34
|
-
seconds = (offset - (minutes*60))
|
35
|
-
|
36
|
-
if minutes > 0
|
37
|
-
"#{minutes}m#{seconds.floor}s"
|
38
|
-
else
|
39
|
-
"#{seconds.round(2)}s"
|
40
|
-
end.rjust(6)
|
41
|
-
end
|
42
|
-
|
43
|
-
def chdir_string(options)
|
44
|
-
if options[:chdir]
|
45
|
-
" in #{options[:chdir]}"
|
46
|
-
else
|
47
|
-
""
|
48
|
-
end
|
49
|
-
end
|
50
|
-
|
51
|
-
def format_command(arguments, buffer)
|
52
|
-
arguments = arguments.dup
|
53
|
-
|
54
|
-
# environment = arguments.first.is_a?(Hash) ? arguments.shift : nil
|
55
|
-
options = arguments.last.is_a?(Hash) ? arguments.pop : nil
|
56
|
-
|
57
|
-
arguments = arguments.flatten.collect(&:to_s)
|
58
|
-
|
59
|
-
buffer << Rainbow(arguments.shelljoin).bright.blue
|
60
|
-
|
61
|
-
if options
|
62
|
-
buffer << chdir_string(options)
|
63
|
-
end
|
64
|
-
|
65
|
-
buffer << "\n"
|
66
|
-
|
67
|
-
# if environment
|
68
|
-
# environment.each do |key,value|
|
69
|
-
# buffer << "\texport #{key}=#{value.dump}\n"
|
70
|
-
# end
|
71
|
-
# end
|
72
|
-
end
|
73
|
-
|
74
|
-
def format_exception(exception, buffer)
|
75
|
-
buffer << Rainbow("#{exception.class}: #{exception}").bright.red << "\n"
|
76
|
-
exception.backtrace.each do |line|
|
77
|
-
buffer << "\t" << Rainbow(line).red << "\n"
|
78
|
-
end
|
79
|
-
end
|
80
|
-
|
81
|
-
def format_line(message, severity)
|
82
|
-
if severity == 'ERROR'
|
83
|
-
Rainbow(message).red
|
84
|
-
else
|
85
|
-
message
|
86
|
-
end
|
87
|
-
end
|
88
|
-
|
89
|
-
def call(severity, datetime, progname, message)
|
90
|
-
buffer = []
|
91
|
-
prefix = ""
|
92
|
-
|
93
|
-
if @verbose
|
94
|
-
prefix = time_offset_prefix
|
95
|
-
buffer << Rainbow(prefix).cyan + ": "
|
96
|
-
prefix = " " * (prefix.size) + "| "
|
97
|
-
end
|
98
|
-
|
99
|
-
if progname == 'shell' and message.kind_of? Array
|
100
|
-
format_command(message, buffer)
|
101
|
-
elsif message.kind_of? Exception
|
102
|
-
format_exception(message, buffer)
|
103
|
-
else
|
104
|
-
buffer << format_line(message, severity) << "\n"
|
105
|
-
end
|
106
|
-
|
107
|
-
result = buffer.join
|
108
|
-
|
109
|
-
# This fancy regex indents lines correctly depending on the prefix:
|
110
|
-
result.gsub!(/\n(?!$)/, "\n#{prefix}") unless prefix.empty?
|
111
|
-
|
112
|
-
return result
|
113
|
-
end
|
114
|
-
end
|
115
|
-
end
|
Binary file
|
Binary file
|
Binary file
|
data/media/LSync Logo.png
DELETED
Binary file
|
data/spec/synco/backup_script.rb
DELETED
@@ -1,63 +0,0 @@
|
|
1
|
-
#!/usr/bin/env rspec
|
2
|
-
|
3
|
-
# Copyright, 2015, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
4
|
-
#
|
5
|
-
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
-
# of this software and associated documentation files (the "Software"), to deal
|
7
|
-
# in the Software without restriction, including without limitation the rights
|
8
|
-
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
-
# copies of the Software, and to permit persons to whom the Software is
|
10
|
-
# furnished to do so, subject to the following conditions:
|
11
|
-
#
|
12
|
-
# The above copyright notice and this permission notice shall be included in
|
13
|
-
# all copies or substantial portions of the Software.
|
14
|
-
#
|
15
|
-
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
-
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
-
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
-
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
-
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
-
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
-
# THE SOFTWARE.
|
22
|
-
|
23
|
-
require 'fingerprint'
|
24
|
-
require 'process/group'
|
25
|
-
require 'fileutils'
|
26
|
-
require 'digest'
|
27
|
-
|
28
|
-
require 'synco/scope'
|
29
|
-
require 'synco/script'
|
30
|
-
|
31
|
-
SYNCO_PATH = File.expand_path("../../bin/synco", __dir__)
|
32
|
-
|
33
|
-
RSpec.shared_context "backup script" do
|
34
|
-
def create_files(master_path, target_path)
|
35
|
-
FileUtils.rm_rf master_path
|
36
|
-
FileUtils.rm_rf target_path
|
37
|
-
|
38
|
-
FileUtils.mkdir_p master_path
|
39
|
-
FileUtils.mkdir_p target_path
|
40
|
-
|
41
|
-
(1...10).each do |i|
|
42
|
-
path = File.join(master_path, i.to_s)
|
43
|
-
|
44
|
-
FileUtils.mkdir(path)
|
45
|
-
|
46
|
-
text = Digest::MD5.hexdigest(i.to_s)
|
47
|
-
|
48
|
-
File.open(File.join(path, i.to_s), "w") { |f| f.write(text) }
|
49
|
-
end
|
50
|
-
end
|
51
|
-
|
52
|
-
let(:tmp_path) {File.join(__dir__, 'tmp')}
|
53
|
-
let(:master_path) {File.join(__dir__, 'tmp/master')}
|
54
|
-
let(:target_path) {File.join(__dir__, 'tmp/target')}
|
55
|
-
|
56
|
-
before(:each) do
|
57
|
-
create_files(master_path, target_path)
|
58
|
-
end
|
59
|
-
|
60
|
-
after(:each) do
|
61
|
-
# FileUtils.rm_rf tmp_path
|
62
|
-
end
|
63
|
-
end
|
@@ -1,33 +0,0 @@
|
|
1
|
-
#!/usr/bin/env rspec
|
2
|
-
|
3
|
-
# Copyright, 2016, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
4
|
-
#
|
5
|
-
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
-
# of this software and associated documentation files (the "Software"), to deal
|
7
|
-
# in the Software without restriction, including without limitation the rights
|
8
|
-
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
-
# copies of the Software, and to permit persons to whom the Software is
|
10
|
-
# furnished to do so, subject to the following conditions:
|
11
|
-
#
|
12
|
-
# The above copyright notice and this permission notice shall be included in
|
13
|
-
# all copies or substantial portions of the Software.
|
14
|
-
#
|
15
|
-
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
-
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
-
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
-
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
-
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
-
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
-
# THE SOFTWARE.
|
22
|
-
|
23
|
-
require 'synco/directory'
|
24
|
-
|
25
|
-
describe Synco::Directory.new(".", arguments: ['--foo']) do
|
26
|
-
it "should have arguments" do
|
27
|
-
expect(subject.arguments).to include('--foo')
|
28
|
-
end
|
29
|
-
|
30
|
-
it "must be relative path" do
|
31
|
-
expect{Synco::Directory.new("/var")}.to raise_error(ArgumentError)
|
32
|
-
end
|
33
|
-
end
|
data/spec/synco/local_backup.rb
DELETED
@@ -1,56 +0,0 @@
|
|
1
|
-
#!/usr/bin/env ruby
|
2
|
-
|
3
|
-
require 'synco'
|
4
|
-
require 'synco/shells/ssh'
|
5
|
-
require 'synco/methods/rsync'
|
6
|
-
|
7
|
-
# Check we've got two paths for sync.
|
8
|
-
unless ARGV.size == 2
|
9
|
-
$stderr.puts "Usage: #{$0} [master] [copy]"
|
10
|
-
exit(255)
|
11
|
-
end
|
12
|
-
|
13
|
-
# $growl = Growl.new "localhost", "Synco", ["Backup Status"]
|
14
|
-
# $growl.register
|
15
|
-
|
16
|
-
$stdout.sync = true
|
17
|
-
$stderr.sync = true
|
18
|
-
|
19
|
-
Synco::run_script do |script|
|
20
|
-
self.method = Synco::Methods::RSyncSnapshot.new(:push, "--archive", "--compress", "--stats")
|
21
|
-
|
22
|
-
self.master = :src
|
23
|
-
|
24
|
-
server(:src) do |server|
|
25
|
-
server.root = ARGV[0]
|
26
|
-
end
|
27
|
-
|
28
|
-
server(:dst) do |server|
|
29
|
-
server.root = ARGV[1]
|
30
|
-
|
31
|
-
# Runs after all directories have been successfully backed up.
|
32
|
-
server.on(:success) do
|
33
|
-
target.run "synco", "rotate"
|
34
|
-
target.run "synco", "prune"
|
35
|
-
end
|
36
|
-
end
|
37
|
-
|
38
|
-
# This event is fired before the backup starts
|
39
|
-
#script.on(:prepare) do
|
40
|
-
# $growl.notify "Backup Status", "Starting Backup", "Starting at #{Time.now.to_s}"
|
41
|
-
#end
|
42
|
-
#
|
43
|
-
## This event occurs if the backup is successful
|
44
|
-
#script.on(:success) do
|
45
|
-
# $growl.notify "Backup Status", "Backup Successful", script.log.string
|
46
|
-
#end
|
47
|
-
#
|
48
|
-
## This event occurs if any part of the backup fails and is not handled elsewhere.
|
49
|
-
#script.on(:failure) do |failure|
|
50
|
-
# $growl.notify "Backup Status", "Backup Failure", failure.to_s
|
51
|
-
#
|
52
|
-
# raise failure
|
53
|
-
#end
|
54
|
-
|
55
|
-
backup('./')
|
56
|
-
end
|
data/spec/synco/local_sync.rb
DELETED
@@ -1,91 +0,0 @@
|
|
1
|
-
#!/usr/bin/env ruby
|
2
|
-
|
3
|
-
# You need to have the FSSM gem installed for this to work.
|
4
|
-
require 'fssm'
|
5
|
-
|
6
|
-
require 'synco'
|
7
|
-
require 'synco/shells/ssh'
|
8
|
-
require 'synco/methods/rsync'
|
9
|
-
require 'synco/event_timer'
|
10
|
-
|
11
|
-
MONITOR = ARGV.delete "-m"
|
12
|
-
|
13
|
-
# Check we've got two paths for sync.
|
14
|
-
unless ARGV.size == 2
|
15
|
-
$stderr.puts "Usage: #{$0} [master] [copy]"
|
16
|
-
exit(255)
|
17
|
-
end
|
18
|
-
|
19
|
-
script = Synco::Script.new do |script|
|
20
|
-
script.method = Synco::Methods::RSync.new(:push, :arguments => ["--archive", "--delete"])
|
21
|
-
|
22
|
-
script.master = :src
|
23
|
-
|
24
|
-
server(:src) do |server|
|
25
|
-
server.root = ARGV[0]
|
26
|
-
end
|
27
|
-
|
28
|
-
server(:dst) do |server|
|
29
|
-
server.root = ARGV[1]
|
30
|
-
|
31
|
-
#server.on(:prepare) do
|
32
|
-
# logger.warn "Aborting backup!"
|
33
|
-
# server.abort!
|
34
|
-
#end
|
35
|
-
end
|
36
|
-
|
37
|
-
#server(:remote) do |server|
|
38
|
-
# server.host = "ayako.oriontransfer.org"
|
39
|
-
# server.shell = Synco::Shells::SSH.new(:user => "nobody")
|
40
|
-
# server.root = "/tmp/sync-test"
|
41
|
-
#end
|
42
|
-
|
43
|
-
#server(:remote2) do |server|
|
44
|
-
# server.host = "remote2.example.com"
|
45
|
-
# server.root = "/tmp/sync-test"
|
46
|
-
#
|
47
|
-
# server.on(:prepare) do |controller|
|
48
|
-
# controller.run! "mysqldump", "..."
|
49
|
-
# end
|
50
|
-
#
|
51
|
-
# server.on(:success) do |controller|
|
52
|
-
# controller.run! "uname", "-a"
|
53
|
-
# end
|
54
|
-
#end
|
55
|
-
|
56
|
-
backup('./')
|
57
|
-
end
|
58
|
-
|
59
|
-
# Initial sync:
|
60
|
-
|
61
|
-
context = Synco::Context.new(script)
|
62
|
-
context.run!
|
63
|
-
|
64
|
-
if MONITOR
|
65
|
-
# Monitor directories for changes:
|
66
|
-
monitor = FSSM::Monitor.new(:directories => true)
|
67
|
-
|
68
|
-
# The event timer aggregates events into a single callback which will be called at most
|
69
|
-
# once every k seconds (where k = 10 in this case).
|
70
|
-
$event_timer = Synco::EventTimer.new(10) do
|
71
|
-
$script.run!
|
72
|
-
end
|
73
|
-
|
74
|
-
$script.directories.each do |directory|
|
75
|
-
full_path = $script[:src].full_path(directory)
|
76
|
-
puts "Monitoring path: #{full_path}"
|
77
|
-
|
78
|
-
monitor.path(full_path, "**/*") do
|
79
|
-
update { $event_timer.trigger! }
|
80
|
-
delete { $event_timer.trigger! }
|
81
|
-
create { $event_timer.trigger! }
|
82
|
-
end
|
83
|
-
end
|
84
|
-
|
85
|
-
begin
|
86
|
-
monitor.run
|
87
|
-
ensure
|
88
|
-
# We should wait for the backup to complete nicely.
|
89
|
-
$event_timer.join
|
90
|
-
end
|
91
|
-
end
|
data/spec/synco/method_spec.rb
DELETED
@@ -1,62 +0,0 @@
|
|
1
|
-
#!/usr/bin/env rspec
|
2
|
-
|
3
|
-
# Copyright, 2016, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
4
|
-
#
|
5
|
-
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
-
# of this software and associated documentation files (the "Software"), to deal
|
7
|
-
# in the Software without restriction, including without limitation the rights
|
8
|
-
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
-
# copies of the Software, and to permit persons to whom the Software is
|
10
|
-
# furnished to do so, subject to the following conditions:
|
11
|
-
#
|
12
|
-
# The above copyright notice and this permission notice shall be included in
|
13
|
-
# all copies or substantial portions of the Software.
|
14
|
-
#
|
15
|
-
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
-
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
-
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
-
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
-
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
-
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
-
# THE SOFTWARE.
|
22
|
-
|
23
|
-
require 'synco/scope'
|
24
|
-
require 'synco/methods/scp'
|
25
|
-
|
26
|
-
require_relative 'backup_script'
|
27
|
-
|
28
|
-
describe Synco::Methods::SCP do
|
29
|
-
include_context "backup script"
|
30
|
-
|
31
|
-
let(:test_server) {ENV.fetch('TEST_SERVER', 'localhost')}
|
32
|
-
|
33
|
-
let(:script) {Synco::Script.new}
|
34
|
-
let(:logger) {Logger.new($stderr)}
|
35
|
-
let(:master_server) {Synco::Server.new('localhost', root: master_path)}
|
36
|
-
let(:target_server) {Synco::Server.new(test_server, root: target_path)}
|
37
|
-
|
38
|
-
let(:group) {Process::Group.new}
|
39
|
-
let(:script_scope) {Synco::ScriptScope.new(script, logger, group)}
|
40
|
-
let(:sync_scope) {Synco::TargetScope.new(script_scope, target_server)}
|
41
|
-
|
42
|
-
let(:directory) {Synco::Directory.new(".", arguments: ['--archive'])}
|
43
|
-
let(:directory_scope) {Synco::DirectoryScope.new(sync_scope, directory)}
|
44
|
-
|
45
|
-
# This example shows all the state which goes into one single method invocation:
|
46
|
-
it 'should copy files from master to target' do
|
47
|
-
script.instance_variable_set(:@master_server, master_server)
|
48
|
-
script.instance_variable_set(:@current_server, master_server)
|
49
|
-
|
50
|
-
expect(directory_scope.master_server).to_not be == nil
|
51
|
-
expect(directory_scope.target_server).to_not be == nil
|
52
|
-
expect(directory_scope.current_server).to_not be == nil
|
53
|
-
|
54
|
-
group.wait do
|
55
|
-
Fiber.new do
|
56
|
-
subject.call(directory_scope)
|
57
|
-
end.resume
|
58
|
-
end
|
59
|
-
|
60
|
-
expect(Fingerprint).to be_identical(master_path, target_path)
|
61
|
-
end
|
62
|
-
end
|