anjea_backup 0.0.1 → 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +8 -8
- data/Gemfile.lock +1 -1
- data/README.md +30 -2
- data/bin/anjea +22 -2
- data/lib/anjea_backup.rb +2 -1
- data/lib/anjea_backup/anjea_backup.rb +15 -32
- data/lib/anjea_backup/backup_item.rb +19 -0
- data/lib/anjea_backup/inifile.rb +4 -2
- data/lib/anjea_backup/logger.rb +20 -0
- data/lib/anjea_backup/version.rb +1 -1
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
---
|
2
2
|
!binary "U0hBMQ==":
|
3
3
|
metadata.gz: !binary |-
|
4
|
-
|
4
|
+
MjE2N2EwNThkYWZmY2E4OTFiM2UyYzc4YWUxZjIxMWU2M2M1YWI3Mw==
|
5
5
|
data.tar.gz: !binary |-
|
6
|
-
|
6
|
+
ZDc1ZDMyZGNlNTY4MmIxZDFkY2RmODEwMmM4ZmU2NWUyZTVjZjI1Zg==
|
7
7
|
SHA512:
|
8
8
|
metadata.gz: !binary |-
|
9
|
-
|
10
|
-
|
11
|
-
|
9
|
+
ZDlhMzk5ZDQyYTczN2Q2MDI4MDQwN2FjZGYzY2YxZWViZWU2MjBiNDE3NDA4
|
10
|
+
NGQyYzk3MTk0OTJiZWE5MGE1ZTkxZDYyYjNmN2IwMGU5MWU4M2QxYjg3MGNk
|
11
|
+
YWNiYzRkOGVhOGYyYzMwZDBkNWQ5MTg2NTJlZGE2ZjZkOGY1YzY=
|
12
12
|
data.tar.gz: !binary |-
|
13
|
-
|
14
|
-
|
15
|
-
|
13
|
+
MGNlNzU2N2M3NmFmYWMwNDM5NzQyMDgxZTFhYTlkZTQ4ODY4MDU0YzAyNDg2
|
14
|
+
YTU4ZDhkODZiMDE5MDRlZjFmNGE2NWY2ZTFlNGI3MjcxZDM5NzE1MDA5NTY2
|
15
|
+
OTZjOGJjZTAyY2FkZTdlNDVlNmI0YWQyNzlhMTc0ZjhhOTJmOGI=
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -4,11 +4,34 @@ Warning: *I do not consider AnjeaBackup anything close to production-ready* .
|
|
4
4
|
|
5
5
|
It is however a fun toy, and some things do work.
|
6
6
|
|
7
|
+
# Mission
|
8
|
+
|
9
|
+
AnjeaBackup should
|
10
|
+
|
11
|
+
- not depend on many gems (ideally a plain ruby installation should suffice).
|
12
|
+
- use rsync and hard links (it helps a lot if you understand these).
|
13
|
+
- be configurable through editing configuration files.
|
14
|
+
- not depend on other services (e.g. not use an external database).
|
15
|
+
- concentrate on doing the file synchronization, stuff like mailing or regular execution can be left to your other favorite tools (see #Example).
|
16
|
+
|
7
17
|
AnjeaBackup will create copies of local or remote (via ssh) files and directories, hard-linked to former versions of the same files and directories.
|
8
18
|
|
9
19
|
The approach has been beautifully implemented by numerous people using e.g. bash or perl scripts.
|
10
20
|
I aimed for quick and easy configurability and some oppinionated choices that were given my usage scenario.
|
11
21
|
|
22
|
+
With anjea_backup you define what folders and files to backup (these can be on a remote server to which you have passwordless key-based ssh access to, or use sshfs to 'tunnel' it to your local filesystem). On each execution of anjea_backup, a copy of these files and folders will be created in a defined location. Hard links are used which means:
|
23
|
+
- only changed files need to be transferred
|
24
|
+
- unchanged files are represented by inodes, taking away virtually no disk space. Always the last backup serves as reference for the next ones (against which changes are computed). The drawback here is that if a single byte in a big file changes, the whole file is transferred and created again - thus it is not a solution to sparsely backup e.g. VM images.
|
25
|
+
|
26
|
+
anjea_backup preserves the original full path and will put your files in a folder structure similar to this:
|
27
|
+
|
28
|
+
/backup/2014-11-21-04/symbolic_name/path/to/asset
|
29
|
+
/backup/2014-11-21-04/other_name/path/to_other/asset
|
30
|
+
|
31
|
+
where `/backup` can be configured, `symbolic_name` is a user-given name and `/path/to/asset` is the path on the machine that receives the backup.
|
32
|
+
|
33
|
+
Note that symbolic links the in source can be tricky.
|
34
|
+
|
12
35
|
## Installation
|
13
36
|
|
14
37
|
You'll probably need a decent linux installation and have rsync installed.
|
@@ -19,7 +42,7 @@ You'll probably need a decent linux installation and have rsync installed.
|
|
19
42
|
|
20
43
|
AnjeaBackup is meant to be a script. If you find 'library' usage for it, get in touch. Also, I think if you want to use it, get in touch.
|
21
44
|
|
22
|
-
The script is in
|
45
|
+
The script is in `bin/anjea` .
|
23
46
|
|
24
47
|
The working basic use case is to just call it. It will pick up two configuration files (see next sections) and immediately start a backup-attempt.
|
25
48
|
|
@@ -57,7 +80,9 @@ Defines *what* to backup and how to access it. Example:
|
|
57
80
|
key=/home/anjea/.anjea/keys/otherhostkey_rsa
|
58
81
|
user=anjea
|
59
82
|
|
60
|
-
|
83
|
+
The group names (e.g. [files]) serve as human readable name and as directory name within the `dst` location specified in `anjea.conf`.
|
84
|
+
|
85
|
+
In this example, the second group (_remotebox_) saves files from a remote box (`anjea@otherhost.mynetwork:/home/remoteanjea/stuff`) using the public key defined in `key` .
|
61
86
|
|
62
87
|
## Example
|
63
88
|
|
@@ -77,11 +102,14 @@ This will need work.
|
|
77
102
|
* Continue with other backups in case one faults
|
78
103
|
* Allow manual targeted backups of single sources
|
79
104
|
* Revive tests
|
105
|
+
* Nested groups in ".ini" files
|
106
|
+
* Proper ini file parser
|
80
107
|
* (missing: send mail) -> easy to do from outside
|
81
108
|
* (missing: do full, non incremental, non hardlinked backups into vault from time to time)
|
82
109
|
|
83
110
|
## Contributing
|
84
111
|
|
112
|
+
0. Contact me
|
85
113
|
1. Fork it ( https://github.com/[my-github-username]/anjea_backup/fork )
|
86
114
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
87
115
|
3. Commit your changes (`git commit -am 'Add some feature'`)
|
data/bin/anjea
CHANGED
@@ -2,6 +2,19 @@
|
|
2
2
|
|
3
3
|
require 'anjea_backup'
|
4
4
|
|
5
|
+
usage_doc = <<eos
|
6
|
+
anjea [backup|vault|list|help]
|
7
|
+
|
8
|
+
anjea_backup uses rsync to synchronize files.
|
9
|
+
It uses hard links which are suitable for large sets of small, chaning files.
|
10
|
+
Hard links are not so optimal for large files, a single changed byte
|
11
|
+
will leed to a full copy of that file.
|
12
|
+
|
13
|
+
Configuration takes place in two files, anjea.conf and backup.conf .
|
14
|
+
The the README for details.
|
15
|
+
eos
|
16
|
+
|
17
|
+
|
5
18
|
action = :backup
|
6
19
|
if ARGV.length > 0
|
7
20
|
case ARGV[0].downcase
|
@@ -11,6 +24,10 @@ if ARGV.length > 0
|
|
11
24
|
action = :vault
|
12
25
|
when 'list'
|
13
26
|
action = :list
|
27
|
+
when 'help'
|
28
|
+
action = :help
|
29
|
+
else
|
30
|
+
action = :help
|
14
31
|
end
|
15
32
|
end
|
16
33
|
|
@@ -19,11 +36,14 @@ begin
|
|
19
36
|
when :backup
|
20
37
|
AnjeaBackup::Backup.new.backup
|
21
38
|
when :list
|
22
|
-
AnjeaBackup.new.cleanup
|
39
|
+
AnjeaBackup::Backup.new.cleanup
|
23
40
|
when :vault
|
24
|
-
AnjeaBackup.new.to_vault
|
41
|
+
AnjeaBackup::Backup.new.to_vault
|
42
|
+
when :help
|
43
|
+
puts usage_doc
|
25
44
|
else
|
26
45
|
STDERR.puts "unknown action"
|
46
|
+
puts usage_doc
|
27
47
|
exit 3
|
28
48
|
end
|
29
49
|
rescue NoIniFileError => ex
|
data/lib/anjea_backup.rb
CHANGED
@@ -4,38 +4,22 @@ require 'pathname'
|
|
4
4
|
require_relative 'inifile'
|
5
5
|
|
6
6
|
module AnjeaBackup
|
7
|
-
class BackupItem
|
8
|
-
attr_accessor :name
|
9
|
-
attr_accessor :description
|
10
|
-
attr_accessor :src_dir
|
11
|
-
attr_accessor :ssh_url
|
12
|
-
attr_accessor :ssh_key
|
13
|
-
|
14
|
-
def initialize hash
|
15
|
-
@name = hash[:name]
|
16
|
-
@description = hash['description']
|
17
|
-
@src_dir = hash['src']
|
18
|
-
if hash['host'] && hash['user'] && hash['key']
|
19
|
-
@ssh_url = "#{hash['user']}@#{hash['host']}:#{@src_dir}"
|
20
|
-
@ssh_key = hash['key']
|
21
|
-
end
|
22
|
-
end
|
23
|
-
end
|
24
|
-
|
25
7
|
class Backup
|
8
|
+
include AnjeaBackup::Logger
|
9
|
+
|
26
10
|
def initialize
|
27
11
|
read_system_conf
|
12
|
+
setup_dirs
|
28
13
|
if !lock!
|
29
14
|
log_err "Aborting, anjea already running. Delete #{@lock_file} if not."
|
30
15
|
exit 2
|
31
16
|
end
|
32
17
|
read_backups_conf
|
33
|
-
setup_dirs
|
34
18
|
end
|
35
19
|
|
36
20
|
def backup
|
37
|
-
yyyymmdd =
|
38
|
-
|
21
|
+
yyyymmdd = now_with_hour
|
22
|
+
|
39
23
|
# TODO work in a tmp work dir
|
40
24
|
@backup_items.each do |item|
|
41
25
|
last_backup = File.join(@last, item.name)
|
@@ -46,7 +30,11 @@ module AnjeaBackup
|
|
46
30
|
source = item.ssh_url ? "-e \"ssh -i #{item.ssh_key}\" #{item.ssh_url}"
|
47
31
|
: item.src_dir
|
48
32
|
|
49
|
-
rsync_cmd = "rsync -avz
|
33
|
+
rsync_cmd = "rsync -avz "\
|
34
|
+
"--delete --relative --stats "\
|
35
|
+
"--log-file #{log_file_for(yyyymmdd, item)} "\
|
36
|
+
"--link-dest #{last_backup} "\
|
37
|
+
"#{source} #{today_backup}"
|
50
38
|
|
51
39
|
log item, "rsync start"
|
52
40
|
if system(rsync_cmd)
|
@@ -100,19 +88,13 @@ module AnjeaBackup
|
|
100
88
|
end
|
101
89
|
|
102
90
|
def log_err item=nil, msg
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
STDERR.puts "#{DateTime.now.strftime("%Y-%m-%d-%H:%M")} - #{msg}"
|
107
|
-
end
|
108
|
-
end
|
109
|
-
|
110
|
-
def log item, msg
|
111
|
-
puts "[#{item.name}] #{DateTime.now.strftime("%Y-%m-%d-%H-%M")} - #{msg}"
|
91
|
+
log_msg = (!item.nil?) ? "[#{item.name}] #{now_with_minutes} - #{msg}"
|
92
|
+
: "#{now_with_minutes} - #{msg}"
|
93
|
+
STDERR.puts log_msg
|
112
94
|
end
|
113
95
|
|
114
96
|
def lock!
|
115
|
-
File.new(@lock_file,'w').flock(
|
97
|
+
File.new(@lock_file, 'w').flock(File::LOCK_NB | File::LOCK_EX)
|
116
98
|
end
|
117
99
|
|
118
100
|
def link_last_backup today_backup, last_backup
|
@@ -134,6 +116,7 @@ module AnjeaBackup
|
|
134
116
|
@failed = File.join(@destination, 'failed')
|
135
117
|
@partial = File.join(@destination, 'partial')
|
136
118
|
@lock_file = system_conf[0]['lock']
|
119
|
+
# rescue from malformed config
|
137
120
|
end
|
138
121
|
|
139
122
|
def log_file_for yyyymmdd, item
|
@@ -0,0 +1,19 @@
|
|
1
|
+
module AnjeaBackup
|
2
|
+
class BackupItem
|
3
|
+
attr_accessor :name
|
4
|
+
attr_accessor :description
|
5
|
+
attr_accessor :src_dir
|
6
|
+
attr_accessor :ssh_url
|
7
|
+
attr_accessor :ssh_key
|
8
|
+
|
9
|
+
def initialize hash
|
10
|
+
@name = hash[:name]
|
11
|
+
@description = hash['description']
|
12
|
+
@src_dir = hash['src']
|
13
|
+
if hash['host'] && hash['user'] && hash['key']
|
14
|
+
@ssh_url = "#{hash['user']}@#{hash['host']}:#{@src_dir}"
|
15
|
+
@ssh_key = hash['key']
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
data/lib/anjea_backup/inifile.rb
CHANGED
@@ -18,9 +18,11 @@ end
|
|
18
18
|
|
19
19
|
def read_ini_file filename
|
20
20
|
ini_objs = []
|
21
|
-
|
22
|
-
|
21
|
+
begin
|
22
|
+
file_contents = File.readlines(filename)
|
23
|
+
rescue Errno::ENOENT
|
23
24
|
raise NoIniFileError, "#{filename} config file error"
|
25
|
+
end
|
24
26
|
ini_obj = {}
|
25
27
|
file_contents.each do |line|
|
26
28
|
next if(line.strip.empty? || line.start_with?("#"))
|
@@ -0,0 +1,20 @@
|
|
1
|
+
module AnjeaBackup
|
2
|
+
module Logger
|
3
|
+
def now_with_minutes
|
4
|
+
DateTime.now.strftime("%Y-%m-%d-%H-%M")
|
5
|
+
end
|
6
|
+
|
7
|
+
def now_with_hour
|
8
|
+
DateTime.now.strftime("%Y-%m-%d-%H")
|
9
|
+
end
|
10
|
+
|
11
|
+
def log item, msg
|
12
|
+
if item
|
13
|
+
puts "[#{item.name}] #{now_with_minutes} - #{msg}"
|
14
|
+
else
|
15
|
+
puts "#{now_with_minutes} - #{msg}"
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
data/lib/anjea_backup/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: anjea_backup
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0
|
4
|
+
version: 0.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Felix Wolfsteller
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-01-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -57,7 +57,9 @@ files:
|
|
57
57
|
- bin/anjea
|
58
58
|
- lib/anjea_backup.rb
|
59
59
|
- lib/anjea_backup/anjea_backup.rb
|
60
|
+
- lib/anjea_backup/backup_item.rb
|
60
61
|
- lib/anjea_backup/inifile.rb
|
62
|
+
- lib/anjea_backup/logger.rb
|
61
63
|
- lib/anjea_backup/version.rb
|
62
64
|
homepage: https://github.com/fwolfst/anjea_backup/
|
63
65
|
licenses:
|