baya 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.rdoc ADDED
@@ -0,0 +1,132 @@
1
+ = Baya - Simple backup and archive automation tool
2
+
3
+ Baya is a tool aiming at making it easy to archive and backup all kind of
4
+ data, from a vast variety of places.
5
+
6
+ At the moment, Baya supports archives from the following:
7
+
8
+ * Git repositories
9
+ * Github accounts
10
+ * Local and remote machines via rsync
11
+
12
+ == Usage
13
+
14
+ Baya aims at being simple to use. Just typing
15
+
16
+ baya
17
+
18
+ should be enough in most cases.
19
+
20
+ The command will by default load its configuration from the `baya.json` file.
21
+
22
+ You can specify the location of the configuration file to use:
23
+
24
+ baya --config my_config.json
25
+ baya -c my_config.json
26
+
27
+ To see all available options, consult the command line help:
28
+
29
+ baya --help
30
+ baya -h
31
+
32
+ == Configuration
33
+
34
+ The configuration is writen in a single JSON file, and is built around the
35
+ following concepts:
36
+
37
+ * All backups are stored in a dedicated folder. All data will be stored in
38
+ subfolders from that root folder.
39
+ * Adapters are the basic storage unit. You can specify a list of adapters that
40
+ will represent all the data sources you want to backup. For example, all the
41
+ following are individual adapters you could configure independently:
42
+ * A Git repository
43
+ * A single Github account
44
+ * One remore repository
45
+
46
+ The JSON configuration document is structured in the following way:
47
+
48
+ {
49
+ "root": "The root backup folder",
50
+ "adapters": [
51
+ {
52
+ "type": "The adapter type. Can be `git`, `github` or `rsync`",
53
+ "mode": "`archive` or `backup`. At the moment, only `archive` is supported",
54
+ "config": {
55
+ // Adapter-specific cofiguration
56
+ }
57
+ }
58
+ ]
59
+ }
60
+
61
+ === Git adapter
62
+
63
+ The Git adapter accepts two arguments, and both are mandatory:
64
+
65
+ {
66
+ "type": "git",
67
+ "mode": "`archive` or `backup`",
68
+ "config": {
69
+ "origin": "The Git origin to clone from",
70
+ "destination": "The backup destination, relative to the global root"
71
+ }
72
+ }
73
+
74
+ === Github adapter
75
+
76
+ The Github adapter has three parameters:
77
+
78
+ * `user` specifies a user account to backup Git repositories from.
79
+ * `org` is used when you want to backup Git repositories from an origanisation
80
+ instead of a user account.
81
+ * `destination` is the backup destination folder, relative to the global root.
82
+
83
+ The `user` and `org` parameters are mutually exclusive, and one of them is
84
+ required. The `destination` parameter is always required.
85
+
86
+ Note: Due to a limitation in the current implementation, only public
87
+ repositories are backed up at the moment.
88
+
89
+ === Rsync adapter
90
+
91
+ In archive mode, the `rsync` adapter accepts only two parameters. Both are
92
+ mandatory:
93
+
94
+ * `source` specifies where to copy from. You can use any rsync compliant
95
+ syntax to specify remote folders.
96
+ * `destination` is the backup destination, relative to the root folder.
97
+
98
+ In archive mode, rsync will be invoked with the `-az` flags to perform the
99
+ archive operation. This will enable compression when copying data over the
100
+ network and the rsync "archive" mode.
101
+
102
+ In backup mode, rsync will be invoked with the following arguments:
103
+
104
+ * `-az` to archive all assets using compression over the wire
105
+ * `--delete` to delete old assets
106
+ * `--link-dest=previous_backup` to hard link unchanged files instead of
107
+ copying them. This is in order to save disk space.
108
+ * A timestamp will be added to the destination folder, in order to create a
109
+ different snapshot for every run
110
+
111
+ == License
112
+
113
+ The MIT License
114
+
115
+ Copyright (c) 2013 - Vivien Barousse
116
+
117
+ Permission is hereby granted, free of charge, to any person obtaining a copy of
118
+ this software and associated documentation files (the "Software"), to deal in
119
+ the Software without restriction, including without limitation the rights to
120
+ use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
121
+ the Software, and to permit persons to whom the Software is furnished to do so,
122
+ subject to the following conditions:
123
+
124
+ The above copyright notice and this permission notice shall be included in all
125
+ copies or substantial portions of the Software.
126
+
127
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
128
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
129
+ FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
130
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
131
+ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
132
+ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/bin/baya ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ ROOT = File.expand_path('../../lib', __FILE__)
4
+
5
+ require ROOT + '/baya'
6
+
7
+ b = Baya::Binaries::Baya.new(ARGV)
8
+ b.run
@@ -0,0 +1,35 @@
1
+ require 'git'
2
+
3
+ module Baya
4
+ module Adapters
5
+ class Git
6
+
7
+ def initialize(config)
8
+ @config = config
9
+ check_config
10
+ end
11
+
12
+ def archive(root)
13
+ destination = root + '/' + @config['destination']
14
+ if File.directory?(destination)
15
+ begin
16
+ g = ::Git.open(destination)
17
+ g.pull
18
+ rescue ArgumentError => e
19
+ raise "Folder exists and is not a Git repository: #{destination}"
20
+ end
21
+ else
22
+ ::Git.clone(@config['origin'], destination)
23
+ end
24
+ end
25
+
26
+ private
27
+
28
+ def check_config
29
+ raise "`origin` is mandatory" unless @config['origin']
30
+ raise "`destination` is mandatory" unless @config['destination']
31
+ end
32
+
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,56 @@
1
+ module Baya
2
+ module Adapters
3
+ class Github
4
+
5
+ require 'curb'
6
+ require 'yajl'
7
+ require ROOT + "/baya/adapters/git"
8
+
9
+ API_ROOT = "https://api.github.com/"
10
+
11
+ def initialize(config)
12
+ @config = config
13
+ check_config
14
+ end
15
+
16
+ def archive(root)
17
+ repos.each do |url|
18
+ name = url.split('/').last.gsub(/\.git$/, "")
19
+ git = Git.new('origin' => url, 'destination' => name)
20
+ git.archive(root + '/' + @config['destination'])
21
+ end
22
+ end
23
+
24
+ def repos
25
+ api_url = API_ROOT + target
26
+ http = Curl.get(api_url)
27
+ json = http.body_str
28
+ data = Yajl::Parser.parse(json)
29
+ data.map do |repo|
30
+ repo['clone_url']
31
+ end
32
+ end
33
+
34
+ private
35
+
36
+ def target
37
+ if @config['user']
38
+ "users/#{@config['user']}/repos"
39
+ elsif @config['org']
40
+ "orgs/#{@config['org']}/repos"
41
+ end
42
+ end
43
+
44
+ def check_config
45
+ unless @config["user"] or @config["org"]
46
+ raise "`user` or `org` is required"
47
+ end
48
+ if @config['user'] and @config['org']
49
+ raise "`user` and `org` are exclusive"
50
+ end
51
+ raise "`destination` is required" unless @config['destination']
52
+ end
53
+
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,78 @@
1
+ module Baya
2
+ module Adapters
3
+ class Rsync
4
+
5
+ require 'open3'
6
+ require 'fileutils'
7
+
8
+ def initialize(config)
9
+ @config = config
10
+ end
11
+
12
+ def archive(root)
13
+ target = root + "/" + @config['destination']
14
+ source = @config['source']
15
+
16
+ rsync_archive(source, target)
17
+ end
18
+
19
+ def backup(root)
20
+ target = root + "/" + @config['destination']
21
+ source = @config['source']
22
+ date = Time.now.strftime("%Y%m%d%H%M%S")
23
+
24
+ previous = Dir[target + "/*/"].sort_by { |a| File.basename(a) }
25
+
26
+ rsync_backup(source, target + "/" + date, previous.last)
27
+
28
+ if keep = @config['keepBackups'] && @config['keepBackups'].to_i
29
+ if previous.count - keep > 0
30
+ to_delete = previous[0..previous.count - keep]
31
+ to_delete.each do |f|
32
+ FileUtils.rmtree(f)
33
+ end
34
+ end
35
+ end
36
+ end
37
+
38
+ private
39
+
40
+ def rsync_archive(source, target)
41
+ check_folder(target, "destination")
42
+ Open3.popen3("rsync", "-az", source, target) do |i, o, e, process|
43
+ if process.value != 0
44
+ raise "Non-zero value from `rsync`."
45
+ end
46
+ end
47
+ end
48
+
49
+ def rsync_backup(source, target, link)
50
+ check_folder(target, "destination")
51
+ options = [
52
+ "rsync",
53
+ "-az",
54
+ "--delete",
55
+ source,
56
+ target
57
+ ]
58
+ options << "--link-dest=#{link}" if link
59
+
60
+ Open3.popen3(*options) do |i, o, e, process|
61
+ if process.value != 0
62
+ raise "Non-zero value from `rsync`."
63
+ end
64
+ end
65
+ end
66
+
67
+ def check_folder(dir, name)
68
+ if File.exist?(dir) && !File.directory?(dir)
69
+ raise "`#{name}` already exists, and is not a directory"
70
+ end
71
+ unless File.exist?(dir)
72
+ FileUtils.mkdir_p(dir)
73
+ end
74
+ end
75
+
76
+ end
77
+ end
78
+ end
@@ -0,0 +1,17 @@
1
+ module Baya
2
+ module Adapters
3
+ require ROOT + '/baya/adapters/git'
4
+ require ROOT + '/baya/adapters/github'
5
+ require ROOT + '/baya/adapters/rsync'
6
+
7
+ ADAPTERS = {
8
+ 'git' => Git,
9
+ 'github' => Github,
10
+ 'rsync' => Rsync
11
+ }
12
+
13
+ def self.from_name(name)
14
+ ADAPTERS[name]
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,45 @@
1
+ module Baya
2
+ module Binaries
3
+ class Baya
4
+
5
+ require ROOT + '/baya/configuration/command_line'
6
+ require ROOT + '/baya/configuration/file'
7
+ require ROOT + '/baya/runner'
8
+
9
+ def initialize(args)
10
+ @args = Configuration::CommandLine.new(args)
11
+ end
12
+
13
+ def run
14
+ if @args.version
15
+ out.puts "Baya v#{VERSION}"
16
+ return
17
+ end
18
+ if @args.help
19
+ out.puts @args.opts.help
20
+ return
21
+ end
22
+
23
+ unless File.file?(@args.config)
24
+ err.puts "Can't read configuration file '#{@args.config}'."
25
+ err.puts "Make sure it exists and it is a file."
26
+ return
27
+ end
28
+
29
+ @config = Configuration::File.new(@args.config)
30
+
31
+ runner = Runner.new(@config)
32
+ runner.run
33
+ end
34
+
35
+ def out
36
+ STDOUT
37
+ end
38
+
39
+ def err
40
+ STDERR
41
+ end
42
+
43
+ end
44
+ end
45
+ end
@@ -0,0 +1,5 @@
1
+ module Baya
2
+ module Binaries
3
+ require ROOT + "/baya/binaries/baya"
4
+ end
5
+ end
@@ -0,0 +1,38 @@
1
+
2
+ module Baya
3
+ module Configuration
4
+ require 'optparse'
5
+ require 'ostruct'
6
+
7
+ class CommandLine < OpenStruct
8
+
9
+ DEFAULTS = {
10
+ 'config' => 'baya.json'
11
+ }
12
+
13
+ def initialize(args)
14
+ super
15
+ DEFAULTS.each do |k, v|
16
+ self.send(:"#{k}=", v)
17
+ end
18
+ opts.parse(args)
19
+ end
20
+
21
+ def opts
22
+ @opts ||= OptionParser.new do |opts|
23
+ opts.on("-c", "--config CONFIG",
24
+ "Set path to the configuration file") do |config|
25
+ self.config = config
26
+ end
27
+ opts.on_tail("-v", "--version", "Show version") do
28
+ self.version = true
29
+ end
30
+ opts.on_tail("-h", "--help", "Show help") do
31
+ self.help = true
32
+ end
33
+ end
34
+ end
35
+
36
+ end
37
+ end
38
+ end
@@ -0,0 +1,28 @@
1
+ module Baya
2
+ module Configuration
3
+ class File
4
+ require 'yajl'
5
+ require 'ostruct'
6
+
7
+ def initialize(file)
8
+ json = ::File.open(file).read
9
+ @data = Yajl::Parser.parse(json)
10
+ end
11
+
12
+ def root
13
+ @root ||= ::File.expand_path(@data['root'])
14
+ end
15
+
16
+ def adapters
17
+ (@data['adapters'] || []).map do |a|
18
+ adapter = OpenStruct.new
19
+ adapter.type = a["type"]
20
+ adapter.mode = a["mode"]
21
+ adapter.config = a["config"]
22
+ adapter
23
+ end
24
+ end
25
+
26
+ end
27
+ end
28
+ end
@@ -0,0 +1,28 @@
1
+ module Baya
2
+ class Runner
3
+
4
+ def initialize(config)
5
+ @config = config
6
+ end
7
+
8
+ def run
9
+ @config.adapters.each do |a|
10
+ if klass = Adapters.from_name(a.type)
11
+ adapter = klass.new(a.config)
12
+ else
13
+ raise "Unknown adapter `#{a.type}`" unless adapter
14
+ end
15
+
16
+ case a.mode
17
+ when 'archive'
18
+ adapter.archive(@config.root)
19
+ when 'backup'
20
+ adapter.backup(@config.root)
21
+ else
22
+ raise "Unknown mode `#{a.mode}` for adapter `#{a.type}`"
23
+ end
24
+ end
25
+ end
26
+
27
+ end
28
+ end
data/lib/baya.rb ADDED
@@ -0,0 +1,7 @@
1
+ module Baya
2
+ VERSION = '0.1.0'
3
+
4
+ ROOT = File.expand_path('..', __FILE__)
5
+ require ROOT + "/baya/adapters"
6
+ require ROOT + "/baya/binaries"
7
+ end
metadata ADDED
@@ -0,0 +1,140 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: baya
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Vivien Barousse
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2013-01-03 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: git
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - '='
20
+ - !ruby/object:Gem::Version
21
+ version: 1.2.5
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - '='
28
+ - !ruby/object:Gem::Version
29
+ version: 1.2.5
30
+ - !ruby/object:Gem::Dependency
31
+ name: curb
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - '='
36
+ - !ruby/object:Gem::Version
37
+ version: 0.8.3
38
+ type: :runtime
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - '='
44
+ - !ruby/object:Gem::Version
45
+ version: 0.8.3
46
+ - !ruby/object:Gem::Dependency
47
+ name: yajl-ruby
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - '='
52
+ - !ruby/object:Gem::Version
53
+ version: 1.1.0
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - '='
60
+ - !ruby/object:Gem::Version
61
+ version: 1.1.0
62
+ - !ruby/object:Gem::Dependency
63
+ name: rspec
64
+ requirement: !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - '='
68
+ - !ruby/object:Gem::Version
69
+ version: 2.12.0
70
+ type: :development
71
+ prerelease: false
72
+ version_requirements: !ruby/object:Gem::Requirement
73
+ none: false
74
+ requirements:
75
+ - - '='
76
+ - !ruby/object:Gem::Version
77
+ version: 2.12.0
78
+ - !ruby/object:Gem::Dependency
79
+ name: simplecov
80
+ requirement: !ruby/object:Gem::Requirement
81
+ none: false
82
+ requirements:
83
+ - - '='
84
+ - !ruby/object:Gem::Version
85
+ version: 0.7.1
86
+ type: :development
87
+ prerelease: false
88
+ version_requirements: !ruby/object:Gem::Requirement
89
+ none: false
90
+ requirements:
91
+ - - '='
92
+ - !ruby/object:Gem::Version
93
+ version: 0.7.1
94
+ description:
95
+ email: barousse.vivien@gmail.com
96
+ executables:
97
+ - baya
98
+ extensions: []
99
+ extra_rdoc_files:
100
+ - README.rdoc
101
+ files:
102
+ - README.rdoc
103
+ - bin/baya
104
+ - lib/baya.rb
105
+ - lib/baya/adapters/github.rb
106
+ - lib/baya/adapters/rsync.rb
107
+ - lib/baya/adapters/git.rb
108
+ - lib/baya/binaries/baya.rb
109
+ - lib/baya/runner.rb
110
+ - lib/baya/adapters.rb
111
+ - lib/baya/binaries.rb
112
+ - lib/baya/configuration/command_line.rb
113
+ - lib/baya/configuration/file.rb
114
+ homepage: http://github.com/VivienBarousse/baya
115
+ licenses: []
116
+ post_install_message:
117
+ rdoc_options:
118
+ - --main
119
+ - README.rdoc
120
+ require_paths:
121
+ - lib
122
+ required_ruby_version: !ruby/object:Gem::Requirement
123
+ none: false
124
+ requirements:
125
+ - - ! '>='
126
+ - !ruby/object:Gem::Version
127
+ version: '0'
128
+ required_rubygems_version: !ruby/object:Gem::Requirement
129
+ none: false
130
+ requirements:
131
+ - - ! '>='
132
+ - !ruby/object:Gem::Version
133
+ version: '0'
134
+ requirements: []
135
+ rubyforge_project:
136
+ rubygems_version: 1.8.23
137
+ signing_key:
138
+ specification_version: 3
139
+ summary: Simple backup and archive automation tool
140
+ test_files: []