tdl 0.0.1
Sign up to get free protection for your applications and to get access to all the features.
- data/LICENSE +21 -0
- data/README.md +72 -0
- data/Rakefile +21 -0
- data/bin/tdl-convert.rb +82 -0
- data/bin/tdl-create.rb +99 -0
- data/bin/tdl-launch.rb +51 -0
- data/bin/tdl-verify.rb +54 -0
- data/lib/cloud_inst.rb +81 -0
- data/lib/etdl.rb +125 -0
- data/lib/etdl_processor.rb +106 -0
- metadata +77 -0
data/LICENSE
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License
|
2
|
+
|
3
|
+
Copyright (c) 2010-2012 three.js authors
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,72 @@
|
|
1
|
+
# Background
|
2
|
+
|
3
|
+
The [Aeolus](http://aeolusproject.org) [Template Description Language](https://github.com/clalancette/oz/wiki/Oz-template-description-language) is used by [Oz](https://github.com/clalancette/oz), [Imagefactory](https://github.com/aeolusproject/imagefactory), and [Snap](https://github.com/movitto/snap) to describe operating systems and the components and configurations installed on them for useful purposes.
|
4
|
+
|
5
|
+
Oz and imagefactory use the TDL format in conjunction with native system tooling to bootstrap environments to be used as the basis of cloud images, while Snap uses native system tooling to deconstruct cloud instances into a TDLs which can be used on other cloud providers.
|
6
|
+
|
7
|
+
The problem at hand is that constructing these TDLs currently is a resource-intensive operation, as building these from scratch takes lots of time and bandwidth, a detriment to growing the Aeolus community. TDLs are built through a manual trial and error process, and even one small mistake in a TDL might not be discovered for hours and after alot of bandwidth is used.
|
8
|
+
|
9
|
+
tdl-tools aims to provide a few simple tools which makes TDL creation simple and easy. Furthermore these same tools can be used to facilitate systems and service orchestration using the Aeolus TDL format, which combined w/ the image creation capabilities provided by Oz/Imagefactory leads to a unique and easy-to-understand selling point for the Aeolus framework.
|
10
|
+
|
11
|
+
# Solution
|
12
|
+
|
13
|
+
We are proposing adding the following tools to the Aeolus suite to facilitate the creation and management of these TDLs:
|
14
|
+
|
15
|
+
* tdl-create: simple tool used to prompt the end user for a few optional fields and create a barebones TDL with blanks for the user to fill in
|
16
|
+
* tdl-verify: verified a tdl is in the correct format and is standards compliant
|
17
|
+
* tdl-launch: the main utility script which will launch a cloud instance using [Deltacloud](http://deltacloud.apache.org/) and execute the commands on the newly created instance. For the time being tdl-launch will only support rpm and deb based Linux instances, though other package systems and windows support may eventually be added
|
18
|
+
|
19
|
+
Furthermore, going forward we may add the following to the tdl-tools project:
|
20
|
+
|
21
|
+
* tdl-diff: a utility to compare tdls for differences
|
22
|
+
* web-tdl: A RESTful frontend to tdl-launch which will provide the capability to construct TDLs on the fly and automatically launch them on the cloud
|
23
|
+
* hostmytdl: a wrapper to the [Aeolus Templates Git Repository](https://github.com/aeolus-incubator/templates) which allows for the simple uploading and downloading of TDLs
|
24
|
+
* img2tdl: a utility similar to Snap but geared towards deconstructing a cloud image into a TDL
|
25
|
+
* tdl-update: a mechanism to update an instance launched by tdl-launch so as to incorporate new additions to a modified TDL
|
26
|
+
|
27
|
+
This set of utilities are still in the conceptual stage and will be flushed out further before proceeding. For now the first set (tdl-create, tdl-launch, and tdl-verify) are the main components that constitute the tdl-tools project.
|
28
|
+
|
29
|
+
|
30
|
+
# eTDLs
|
31
|
+
|
32
|
+
tdl-tools will require data pertaining to cloud provider credentials so as to launch new instances on that provider. To keep things simple, these will be added to the TDL files themselves as new fields under a 'cloud' tag so as to not overlap w/ the existing attributes. To prevent confusion, these TDLs will be known as 'extended TDLs', or eTDLs.
|
33
|
+
|
34
|
+
Besides these attributes, the TDL format will be left unchanged. The standard TDL fields will all be used to setup the cloud instance, with the exception of the 'os' fields which will have no material effect on the new instance, though will be used for comparison and reporting purposes (eg if set 'os' will be read and compared against the local cloud instance, with a warning being displayed if there is a mismatch).
|
35
|
+
|
36
|
+
At some point tdl-tools may also support reading cloud provider credentials from a file of the local filesystem (such as /etc/tdl-tools and/or ~/.tdl-tools) and/or from the command line so that standard TDLs can be launched on the cloud as is.
|
37
|
+
|
38
|
+
The eTDL fields will contain:
|
39
|
+
|
40
|
+
* type - the type of cloud provider to launch instance against
|
41
|
+
* provider - the deltacloud provider id to launch instance against
|
42
|
+
* username - the cloud username to use when launching the instance
|
43
|
+
* password - the cloud password to use when launching the instance
|
44
|
+
* image - the cloud image to base the instance off of
|
45
|
+
* keyname - the name of the key on the cloud provider to access the instance
|
46
|
+
* ssh_cmd - the command to run to use to ssh into the instance and run commands
|
47
|
+
* scp_cmd - the command to run to use to scp files to the instance
|
48
|
+
|
49
|
+
ssh_cmd and scp_cmd are templates with variables such as [address], [source] and [dst] which will be filled in on the fly with values representing the address of the cloud instance, and the source and destination filenames.
|
50
|
+
|
51
|
+
An example cloud section in an eTDL is as following
|
52
|
+
|
53
|
+
<cloud>
|
54
|
+
<type>ec2</type>
|
55
|
+
<provider>ec2-east-1</provider>
|
56
|
+
<username>ABCDEF</username>
|
57
|
+
<password>FEDCBA</password>
|
58
|
+
<image>ami-2ea50247</image>
|
59
|
+
<keyname>MYKEY</keyname>
|
60
|
+
<ssh_cmd>ssh -q -o StrictHostKeyChecking=no -t ec2-user@[address] -i ~/.ssh/my.key</ssh_cmd>
|
61
|
+
<scp_cmd>scp -o StrictHostKeyChecking=no -i ~/.ssh/my.key [source] ec2-user@[address]:[dst]</scp_cmd>
|
62
|
+
</cloud>
|
63
|
+
|
64
|
+
|
65
|
+
This of course relies on a few assumptions, namely the end user has access to the cloud provider they will be running instances on before hand, the specified image and key exists, and instances come up w/ a publically available ip address. Though these are the same assumptions that hold for various other Aeolus components.
|
66
|
+
|
67
|
+
|
68
|
+
# Going forward
|
69
|
+
|
70
|
+
The tdl-launch utility (in a previous incantation called 'mycloud') was already demonstrated in action to deploy various instances to facilitate a cross-cloud koji installation ( [screencast here](http://www.youtube.com/watch?v=qF2ctg7ItNc) ).
|
71
|
+
|
72
|
+
The tdl-launch utility needs to be cleaned up and optimized / abstracted a bit, and packaged into gem, rpm, and deb packages. Furthuremore the create-tdl utility needs to be written to facilitate setting up a tdl from scratch.
|
data/Rakefile
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
# tdl-tools project Rakefile
|
2
|
+
#
|
3
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
4
|
+
# Licensed under the MIT license
|
5
|
+
# Copyright (C) 2012 Red Hat, Inc.
|
6
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
7
|
+
#
|
8
|
+
###########################################################
|
9
|
+
|
10
|
+
require "rspec/core/rake_task"
|
11
|
+
|
12
|
+
desc "Run all specs"
|
13
|
+
RSpec::Core::RakeTask.new(:spec) do |spec|
|
14
|
+
spec.pattern = 'spec/**/*_spec.rb'
|
15
|
+
spec.rspec_opts = ['--backtrace']
|
16
|
+
end
|
17
|
+
|
18
|
+
desc "build the tdl gem"
|
19
|
+
task :build do
|
20
|
+
system "gem build tdl.gemspec"
|
21
|
+
end
|
data/bin/tdl-convert.rb
ADDED
@@ -0,0 +1,82 @@
|
|
1
|
+
#!/usr/bin/ruby
|
2
|
+
# Utility to convert tdls/etdls
|
3
|
+
#
|
4
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
5
|
+
# Licensed under the MIT license
|
6
|
+
# Copyright (C) 2012 Red Hat, Inc.
|
7
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
8
|
+
#
|
9
|
+
###########################################################
|
10
|
+
|
11
|
+
require 'thor'
|
12
|
+
require 'nokogiri'
|
13
|
+
|
14
|
+
VERSION = "0.1.0"
|
15
|
+
|
16
|
+
class TDLConvert < Thor
|
17
|
+
desc "convert", "convert tdl to/from etdl"
|
18
|
+
def convert(source)
|
19
|
+
extname = File.extname(source)
|
20
|
+
output = File.read(source)
|
21
|
+
if extname == ".etdl"
|
22
|
+
etdl2tdl(output)
|
23
|
+
else
|
24
|
+
tdl2etdl(output)
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
map "--version" => :__version
|
29
|
+
desc "--version", "Version information"
|
30
|
+
def __version
|
31
|
+
say "tdl-tools convert #{VERSION}"
|
32
|
+
end
|
33
|
+
|
34
|
+
desc "help", "Command usage and other information"
|
35
|
+
def help
|
36
|
+
say 'Convert a tdl to / from an etdl'
|
37
|
+
super
|
38
|
+
end
|
39
|
+
|
40
|
+
private
|
41
|
+
def etdl2tdl(output)
|
42
|
+
xml = Nokogiri::XML(output)
|
43
|
+
|
44
|
+
xml.xpath('/template/cloud').remove
|
45
|
+
xml.xpath('/template/verify').remove
|
46
|
+
|
47
|
+
puts xml.to_s
|
48
|
+
end
|
49
|
+
|
50
|
+
def tdl2etdl(output)
|
51
|
+
xml = Nokogiri::XML(output)
|
52
|
+
|
53
|
+
cloudn = Nokogiri::XML::Node.new('cloud', xml)
|
54
|
+
cloudn << "\n"
|
55
|
+
cloud_nodes = ['type', 'provider', 'username', 'password',
|
56
|
+
'image', 'keyname', 'ssh_cmd', 'scp_cmd'].each { |cn|
|
57
|
+
cnn = Nokogiri::XML::Node.new(cn, xml)
|
58
|
+
cnn << " "
|
59
|
+
cloudn << "\t\t"
|
60
|
+
cloudn << cnn
|
61
|
+
cloudn << "\n"
|
62
|
+
}
|
63
|
+
cloudn << "\t"
|
64
|
+
|
65
|
+
verifycmdn = Nokogiri::XML::Node.new('command', xml)
|
66
|
+
verifycmdn['name'] = ''
|
67
|
+
verifyn = Nokogiri::XML::Node.new('verify', xml)
|
68
|
+
verifyn << "\n\t\t"
|
69
|
+
verifyn << verifycmdn
|
70
|
+
verifyn << "\n\t"
|
71
|
+
|
72
|
+
xml.xpath('/template').first << "\t"
|
73
|
+
xml.xpath('/template').first << cloudn
|
74
|
+
xml.xpath('/template').first << "\n\n\t"
|
75
|
+
xml.xpath('/template').first << verifyn
|
76
|
+
xml.xpath('/template').first << "\n"
|
77
|
+
puts xml.to_s
|
78
|
+
end
|
79
|
+
|
80
|
+
end
|
81
|
+
|
82
|
+
TDLConvert.start(ARGV)
|
data/bin/tdl-create.rb
ADDED
@@ -0,0 +1,99 @@
|
|
1
|
+
#!/usr/bin/ruby
|
2
|
+
# Utility to create a barebones (e)TDL from the command line
|
3
|
+
#
|
4
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
5
|
+
# Licensed under the MIT license
|
6
|
+
# Copyright (C) 2012 Red Hat, Inc.
|
7
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
8
|
+
#
|
9
|
+
###########################################################
|
10
|
+
|
11
|
+
require 'thor'
|
12
|
+
require 'nokogiri'
|
13
|
+
|
14
|
+
# use fedora api to verify packages
|
15
|
+
require 'pkgwat'
|
16
|
+
|
17
|
+
VERSION = "0.1.0"
|
18
|
+
SAMPLE_TDL = 'data/sample.tdl'
|
19
|
+
SAMPLE_ETDL = 'data/sample.etdl'
|
20
|
+
|
21
|
+
class TDLCreate < Thor
|
22
|
+
desc "tdl", "create a new TDL"
|
23
|
+
method_options :interactive => :boolean
|
24
|
+
method_options :verify => :boolean
|
25
|
+
def tdl
|
26
|
+
output = File.read(SAMPLE_TDL)
|
27
|
+
output = run_interactive(output, options) if options[:interactive]
|
28
|
+
puts output
|
29
|
+
end
|
30
|
+
|
31
|
+
desc "etdl", "create a new eTDL"
|
32
|
+
method_options :interactive => :boolean
|
33
|
+
method_options :verify => :boolean
|
34
|
+
def etdl
|
35
|
+
output = File.read(SAMPLE_ETDL)
|
36
|
+
output = run_interactive(output, options) if options[:interactive]
|
37
|
+
puts output
|
38
|
+
end
|
39
|
+
|
40
|
+
map "--version" => :__version
|
41
|
+
desc "--version", "Version information"
|
42
|
+
def __version
|
43
|
+
say "tdl-tools create #{VERSION}"
|
44
|
+
end
|
45
|
+
|
46
|
+
desc "help", "Command usage and other information"
|
47
|
+
def help
|
48
|
+
say 'Create a new TDL from scratch'
|
49
|
+
super
|
50
|
+
end
|
51
|
+
|
52
|
+
private
|
53
|
+
def run_interactive(output, options = {})
|
54
|
+
xml = Nokogiri::XML(output)
|
55
|
+
say "CTRL-D at any point to terminate prompts and output (e)tdl"
|
56
|
+
|
57
|
+
begin
|
58
|
+
packages = []
|
59
|
+
name = ask("Template Name:")
|
60
|
+
description = ask("Template Description:")
|
61
|
+
os_name = ask("OS Name:")
|
62
|
+
os_version = ask("OS Version:")
|
63
|
+
while true
|
64
|
+
package = ask('Package:')
|
65
|
+
packages << package
|
66
|
+
end
|
67
|
+
# TODO also files and commands
|
68
|
+
# TODO also cloud and verify for etdls
|
69
|
+
rescue Exception => e
|
70
|
+
say "\n"
|
71
|
+
end
|
72
|
+
|
73
|
+
xml.xpath('/template/name').first.content = name
|
74
|
+
xml.xpath('/template/description').first.content = description
|
75
|
+
xml.xpath('/template/os/name').first.content = os_name
|
76
|
+
xml.xpath('/template/os/version').first.content = os_version
|
77
|
+
pkgnode = xml.xpath('/template/packages').first
|
78
|
+
pkgnode.children[0..1].remove
|
79
|
+
packages.each { |pkg|
|
80
|
+
# verify
|
81
|
+
if options[:verify]
|
82
|
+
fpkg = Pkgwat.get_package(pkg)
|
83
|
+
raise "Could not find package #{pkg} in Fedora" if fpkg.nil?
|
84
|
+
end
|
85
|
+
|
86
|
+
node = Nokogiri::XML::Node.new('package', xml)
|
87
|
+
node['name'] = pkg
|
88
|
+
pkgnode << node
|
89
|
+
|
90
|
+
node = pkgnode.children.first.clone
|
91
|
+
pkgnode << node
|
92
|
+
}
|
93
|
+
|
94
|
+
output = xml.to_s
|
95
|
+
end
|
96
|
+
|
97
|
+
end
|
98
|
+
|
99
|
+
TDLCreate.start(ARGV)
|
data/bin/tdl-launch.rb
ADDED
@@ -0,0 +1,51 @@
|
|
1
|
+
#!/usr/bin/ruby
|
2
|
+
# Utility to read in eTDLs and use Deltacloud to launch them
|
3
|
+
# Run with the path to the eTDL like so:
|
4
|
+
# ./tdl-launch.rb <path-to-etdl>
|
5
|
+
#
|
6
|
+
# The etdls are the same format as the TDLs supported by imagefactory/oz
|
7
|
+
# with additional ('extended') data pertaining to the cloud provider to
|
8
|
+
# deploy to and the environment to setup.
|
9
|
+
#
|
10
|
+
# Assumes instances are started on clouds w/ a public accessible
|
11
|
+
# address and with the ability to ssh in and run operations
|
12
|
+
# (eTDLs must specify ssh and scp commands to use)
|
13
|
+
#
|
14
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
15
|
+
# Licensed under the MIT license
|
16
|
+
# Copyright (C) 2012 Red Hat, Inc.
|
17
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
18
|
+
#
|
19
|
+
###########################################################
|
20
|
+
|
21
|
+
require 'pp'
|
22
|
+
require 'colored'
|
23
|
+
require 'nokogiri'
|
24
|
+
require 'tempfile'
|
25
|
+
|
26
|
+
require 'etdl'
|
27
|
+
require 'etdl_processor'
|
28
|
+
|
29
|
+
###############################################################
|
30
|
+
# Read in configuration from cmd line and xml file
|
31
|
+
|
32
|
+
if ARGV.size != 1
|
33
|
+
puts "Usage: tdl-launch.rb <path-to-tdl>".red
|
34
|
+
exit 1
|
35
|
+
end
|
36
|
+
|
37
|
+
ETDL = ARGV.first
|
38
|
+
|
39
|
+
unless File.readable?(ETDL)
|
40
|
+
puts "eTDL #{ETDL} is not readable".red
|
41
|
+
exit 1
|
42
|
+
end
|
43
|
+
|
44
|
+
doc = Nokogiri::XML(open(ARGV.first))
|
45
|
+
etdl = TDLTools::ETDL.parse(doc)
|
46
|
+
|
47
|
+
launch = ETDLProcessor.new
|
48
|
+
|
49
|
+
etdl.process(launch)
|
50
|
+
|
51
|
+
puts "Done!".bold
|
data/bin/tdl-verify.rb
ADDED
@@ -0,0 +1,54 @@
|
|
1
|
+
#!/usr/bin/ruby
|
2
|
+
# Utility to verify a specified (e)TDL
|
3
|
+
#
|
4
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
5
|
+
# Licensed under the MIT license
|
6
|
+
# Copyright (C) 2012 Red Hat, Inc.
|
7
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
8
|
+
#
|
9
|
+
###########################################################
|
10
|
+
|
11
|
+
require 'thor'
|
12
|
+
require 'nokogiri'
|
13
|
+
require 'colored'
|
14
|
+
|
15
|
+
VERSION = "0.1.0"
|
16
|
+
TDL_RNG = 'data/tdl.rng'
|
17
|
+
ETDL_RNG = 'data/etdl.rng'
|
18
|
+
|
19
|
+
class TDLVerify < Thor
|
20
|
+
desc "verify [path-to-tdl]", "Verify the tdl (or etdl) specified on the command line"
|
21
|
+
def verify(path)
|
22
|
+
say "Validating (e)TDL".bold
|
23
|
+
rng = (File.extname(path) == ".etdl" ? ETDL_RNG : TDL_RNG)
|
24
|
+
xsd = Nokogiri::XML::RelaxNG(File.read(rng))
|
25
|
+
doc = Nokogiri::XML(File.open(path))
|
26
|
+
errs = xsd.validate(doc)
|
27
|
+
if errs.empty?
|
28
|
+
say " TDL is valid".bold.green
|
29
|
+
exit 0
|
30
|
+
|
31
|
+
else
|
32
|
+
say " Errors in TDL".bold.red
|
33
|
+
errs.each { |err|
|
34
|
+
say " #{err.message}".red
|
35
|
+
}
|
36
|
+
exit 1
|
37
|
+
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
41
|
+
map "--version" => :__version
|
42
|
+
desc "--version", "Version information"
|
43
|
+
def __version
|
44
|
+
say "tdl-tools verify #{VERSION}"
|
45
|
+
end
|
46
|
+
|
47
|
+
desc "help", "Command usage and other information"
|
48
|
+
def help
|
49
|
+
say 'Verify a TDL'
|
50
|
+
super
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
TDLVerify.start(ARGV) if __FILE__ == $0
|
data/lib/cloud_inst.rb
ADDED
@@ -0,0 +1,81 @@
|
|
1
|
+
# tdl-tools cloud instance representation
|
2
|
+
#
|
3
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
4
|
+
# Licensed under the MIT license
|
5
|
+
# Copyright (C) 2013 Red Hat, Inc.
|
6
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
7
|
+
#
|
8
|
+
###########################################################
|
9
|
+
|
10
|
+
require 'deltacloud'
|
11
|
+
|
12
|
+
module TDLTools
|
13
|
+
class CloudInst
|
14
|
+
DC_URL = "http://localhost:3002/api"
|
15
|
+
|
16
|
+
NO_START = false # set to true to disable creation of instances
|
17
|
+
|
18
|
+
# ssh address of the instance
|
19
|
+
attr_reader :address
|
20
|
+
|
21
|
+
# ssh command to use
|
22
|
+
attr_reader :ssh
|
23
|
+
|
24
|
+
# scp command to use
|
25
|
+
attr_reader :scp
|
26
|
+
|
27
|
+
def initialize(cloud_attributes)
|
28
|
+
# Use deltacloud to launch and setup instance
|
29
|
+
@cloud_type = cloud_attributes[:type].intern
|
30
|
+
|
31
|
+
@image = cloud_attributes[:image]
|
32
|
+
@keyname = cloud_attributes[:keyname]
|
33
|
+
|
34
|
+
@dc = DeltaCloud.new(cloud_attributes[:username],
|
35
|
+
cloud_attributes[:password],
|
36
|
+
DC_URL)
|
37
|
+
@dc.use_driver @cloud_type
|
38
|
+
|
39
|
+
@dc.instance_variable_set :@api_provider,
|
40
|
+
cloud_attributes[:provider] if @cloud_type == :openstack # XXX needed for openstack
|
41
|
+
|
42
|
+
@ssh = cloud_attributes[:ssh_cmd]
|
43
|
+
@scp = cloud_attributes[:scp_cmd]
|
44
|
+
end
|
45
|
+
|
46
|
+
def launch
|
47
|
+
# startup cloud instance
|
48
|
+
unless NO_START
|
49
|
+
dc_inst = @dc.create_instance(@image, :keyname => @keyname)
|
50
|
+
sleep 300 # 5 minutes
|
51
|
+
end
|
52
|
+
|
53
|
+
# set various propertries from the instance
|
54
|
+
dc_inst = (@cloud_type == :openstack ? @dc.instances.first : @dc.instances.last) # XXX for openstack its first, for ec2 its last
|
55
|
+
address = dc_inst.public_addresses.first
|
56
|
+
address = dc_inst.private_addresses.first if address.nil?
|
57
|
+
@address = address[:address]
|
58
|
+
|
59
|
+
@ssh = @ssh.gsub(/\[address\]/, address)
|
60
|
+
@scp = @scp.gsub(/\[address\]/, address)
|
61
|
+
|
62
|
+
self
|
63
|
+
end
|
64
|
+
|
65
|
+
def exec(cmd)
|
66
|
+
`#{@ssh} #{cmd}`
|
67
|
+
end
|
68
|
+
|
69
|
+
def cp(from, to, append=false)
|
70
|
+
scpf = @scp.gsub(/\[source\]/, from).
|
71
|
+
gsub(/\[dst\]/, from)
|
72
|
+
`#{scpf}`
|
73
|
+
if append
|
74
|
+
`#{ssh} sudo 'cat #{to} #{from} > #{from}.new'`
|
75
|
+
`#{ssh} sudo mv #{from}.new #{to}`
|
76
|
+
else
|
77
|
+
`#{ssh} sudo mv #{from} #{to}`
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
data/lib/etdl.rb
ADDED
@@ -0,0 +1,125 @@
|
|
1
|
+
# tdl-tools etdl representation
|
2
|
+
#
|
3
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
4
|
+
# Licensed under the MIT license
|
5
|
+
# Copyright (C) 2013 Red Hat, Inc.
|
6
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
7
|
+
#
|
8
|
+
###########################################################
|
9
|
+
|
10
|
+
module TDLTools
|
11
|
+
class ETDL
|
12
|
+
# cloud attributes
|
13
|
+
#attr_reader :type, :provider, :username, :password, :keyname, :image, :ssh_cmd, :scp_cmd
|
14
|
+
attr_reader :cloud_attributes
|
15
|
+
|
16
|
+
# instance attributes
|
17
|
+
#attr_reader :name, :description, :hostname, :firewall, :packages, :services, :files, :dirs, :commands
|
18
|
+
attr_reader :instance_attributes
|
19
|
+
|
20
|
+
def initialize(args = {})
|
21
|
+
@cloud_attributes = args[:cloud_attributes] || {}
|
22
|
+
@instance_attributes = args[:instance_attributes] || {}
|
23
|
+
end
|
24
|
+
|
25
|
+
# Parse / return new eTDL instance from xml document
|
26
|
+
def self.parse(doc)
|
27
|
+
cloud_attributes = {}
|
28
|
+
instance_attributes = {}
|
29
|
+
|
30
|
+
doc.children.last.children.each { |c|
|
31
|
+
if c.name == 'name'
|
32
|
+
instance_attributes[:name] = c.text
|
33
|
+
|
34
|
+
elsif c.name == 'description'
|
35
|
+
instance_attributes[:description] = c.text
|
36
|
+
|
37
|
+
elsif c.name == 'hostname'
|
38
|
+
instance_attributes[:hostname] = c.text
|
39
|
+
|
40
|
+
elsif c.name == 'cloud'
|
41
|
+
c.children.each { |ca|
|
42
|
+
cloud_attributes[ca.name.intern] = ca.text
|
43
|
+
}
|
44
|
+
|
45
|
+
elsif c.name == 'firewall'
|
46
|
+
c.children.each { |rule|
|
47
|
+
if rule.name == "tcp" || rule.name == "udp"
|
48
|
+
instance_attributes[:firewall] ||= []
|
49
|
+
instance_attributes[:firewall] << {:proto => rule.name, :value => rule.text}
|
50
|
+
end
|
51
|
+
}
|
52
|
+
|
53
|
+
elsif c.name == 'packages'
|
54
|
+
c.children.each { |pkg|
|
55
|
+
unless pkg['name'].nil?
|
56
|
+
instance_attributes[:packages] ||= []
|
57
|
+
instance_attributes[:packages] << pkg['name']
|
58
|
+
end
|
59
|
+
}
|
60
|
+
|
61
|
+
elsif c.name == 'services'
|
62
|
+
c.children.each { |srv|
|
63
|
+
unless srv['name'].nil?
|
64
|
+
pre_cmds = []
|
65
|
+
post_cmds = []
|
66
|
+
srv.children.each { |s|
|
67
|
+
if s.name == "before"
|
68
|
+
pre_cmds << s.text
|
69
|
+
elsif s.name == "after"
|
70
|
+
post_cmds << s.text
|
71
|
+
end
|
72
|
+
}
|
73
|
+
instance_attributes[:services] ||= []
|
74
|
+
instance_attributes[:services] << {:name => srv['name'], :pre => pre_cmds, :post => post_cmds}
|
75
|
+
end
|
76
|
+
}
|
77
|
+
|
78
|
+
elsif c.name == 'dirs'
|
79
|
+
c.children.each { |dir|
|
80
|
+
owner = dir['owner'] || 'root'
|
81
|
+
group = dir['group'] || 'root'
|
82
|
+
remove = dir['remove'] || false
|
83
|
+
instance_attributes[:dirs] ||= []
|
84
|
+
instance_attributes[:dirs] << {:name => dir.text, :owner => owner, :group => group, :remove => remove} if dir.text.strip != ""
|
85
|
+
}
|
86
|
+
|
87
|
+
elsif c.name == 'files'
|
88
|
+
c.children.each { |file|
|
89
|
+
unless file['name'].nil?
|
90
|
+
mode = file['mode'] || '644'
|
91
|
+
owner = file['owner'] || 'root'
|
92
|
+
group = file['group'] || 'root'
|
93
|
+
instance_attributes[:files] ||= []
|
94
|
+
instance_attributes[:files] << {:name => file['name'], :mode => mode, :append => file['append'],
|
95
|
+
:owner => owner, :group => group,
|
96
|
+
:contents => file.text}
|
97
|
+
end
|
98
|
+
}
|
99
|
+
|
100
|
+
elsif c.name == 'commands'
|
101
|
+
c.children.each { |cmd|
|
102
|
+
unless cmd.name != "command"
|
103
|
+
user = cmd['user'] || 'root'
|
104
|
+
instance_attributes[:commands] ||= []
|
105
|
+
instance_attributes[:commands] << {:cmd => cmd.text, :user => user}
|
106
|
+
end
|
107
|
+
}
|
108
|
+
end
|
109
|
+
}
|
110
|
+
pp instance_attributes
|
111
|
+
#pp cloud_attributes
|
112
|
+
|
113
|
+
etdl = ETDL.new :cloud_attributes => cloud_attributes,
|
114
|
+
:instance_attributes => instance_attributes
|
115
|
+
end
|
116
|
+
|
117
|
+
def process(processor)
|
118
|
+
instance = processor.launch_instance(self)
|
119
|
+
processor.process(self, instance)
|
120
|
+
processor.verify(self, instance)
|
121
|
+
processor.terminate_instance(instance)
|
122
|
+
end
|
123
|
+
|
124
|
+
end
|
125
|
+
end
|
@@ -0,0 +1,106 @@
|
|
1
|
+
# tdl-tools etdl processor
|
2
|
+
#
|
3
|
+
# Part of the Aeolus suite http://aeolusproject.org
|
4
|
+
# Licensed under the MIT license
|
5
|
+
# Copyright (C) 2013 Red Hat, Inc.
|
6
|
+
# Written By Mo Morsi <mmorsi@redhat.com>
|
7
|
+
#
|
8
|
+
###########################################################
|
9
|
+
|
10
|
+
require 'cloud_inst'
|
11
|
+
|
12
|
+
###############################################################
|
13
|
+
# Use various subsystems to process an etdl
|
14
|
+
|
15
|
+
class ETDLProcessor
|
16
|
+
def launch_instance(etdl)
|
17
|
+
puts "Connecting to deltacloud instance at #{TDLTools::CloudInst::DC_URL}...".green
|
18
|
+
inst = TDLTools::CloudInst.new etdl.cloud_attributes
|
19
|
+
|
20
|
+
puts "Booting up cloud instance".green
|
21
|
+
puts " This will take a few minutes to become fully accessible".green
|
22
|
+
inst.launch
|
23
|
+
|
24
|
+
if inst.address.nil?
|
25
|
+
puts "could not find address to access instance with".red
|
26
|
+
exit 1
|
27
|
+
end
|
28
|
+
|
29
|
+
puts "using address #{inst.address}\nssh:#{inst.ssh}\nscp:#{inst.scp}".green
|
30
|
+
|
31
|
+
inst
|
32
|
+
end
|
33
|
+
|
34
|
+
def process(etdl, instance)
|
35
|
+
# TODO need to open firewall ports (we have to open them manually now)
|
36
|
+
|
37
|
+
# set hostname
|
38
|
+
hostname = etdl.instance_attributes[:hostname]
|
39
|
+
unless hostname.nil?
|
40
|
+
puts "Setting hostname to #{hostname}".green
|
41
|
+
instance.exec "sudo hostname #{hostname}"
|
42
|
+
|
43
|
+
# append hostname to /etc/hosts/
|
44
|
+
etdl.instance_attributes[:files] <<
|
45
|
+
{:name => '/etc/hosts', :append => true,
|
46
|
+
:owner => 'root', :group => 'root', :mode => 644,
|
47
|
+
:contents => "#{instance.address} #{hostname}"}
|
48
|
+
end
|
49
|
+
|
50
|
+
# yum install packages
|
51
|
+
packages = etdl.instance_attributes[:packages].join(" ")
|
52
|
+
puts "installing #{packages}".green
|
53
|
+
puts instance.exec("sudo yum install -y --nogpgcheck #{packages}").blue
|
54
|
+
|
55
|
+
# start services
|
56
|
+
services = etdl.instance_attributes[:services]
|
57
|
+
services.each { |s|
|
58
|
+
puts "starting/enabling service #{s[:name]}".green
|
59
|
+
s[:pre].each { |cmd|
|
60
|
+
puts " running precommand #{cmd}".green
|
61
|
+
puts instance.exec("sudo #{cmd}").blue
|
62
|
+
}
|
63
|
+
puts instance.exec("sudo service #{s[:name]} start").blue
|
64
|
+
puts instance.exec("sudo chkconfig --levels 35 #{s[:name]} on").blue
|
65
|
+
s[:post].each { |cmd|
|
66
|
+
puts " running postcommand #{cmd}".green
|
67
|
+
puts instance.exec("sudo #{cmd}").blue
|
68
|
+
}
|
69
|
+
}
|
70
|
+
|
71
|
+
# create dirs
|
72
|
+
dirs = etdl.instance_attributes[:dirs]
|
73
|
+
dirs.each { |d|
|
74
|
+
puts "creating dir #{d[:name]}".green
|
75
|
+
instance.exec("sudo rm -rf #{d[:name]}") if d[:remove]
|
76
|
+
instance.exec("sudo mkdir -p #{d[:name]}")
|
77
|
+
instance.exec("sudo chown #{d[:owner]}.#{d[:group]} #{d[:name]}")
|
78
|
+
}
|
79
|
+
|
80
|
+
# copy files over
|
81
|
+
files = etdl.instance_attributes[:files]
|
82
|
+
files.each { |f|
|
83
|
+
tf = Tempfile.new('tdl-launch')
|
84
|
+
tf.write f[:contents]
|
85
|
+
tf.close
|
86
|
+
|
87
|
+
instance.cp tf.path, f[:name]
|
88
|
+
instance.exec("sudo chown #{f[:owner]}.#{f[:group]} #{f[:name]}")
|
89
|
+
instance.exec("sudo chmod #{f[:mode]} #{f[:name]}")
|
90
|
+
}
|
91
|
+
|
92
|
+
cmds = etdl.instance_attributes[:commands]
|
93
|
+
cmds.each { |c|
|
94
|
+
puts "running command #{c[:cmd]} as #{c[:user]}"
|
95
|
+
puts instance.exec("sudo -u #{c[:user]} -i #{c[:cmd]}").blue
|
96
|
+
}
|
97
|
+
end
|
98
|
+
|
99
|
+
def verify(etdl, instance)
|
100
|
+
# TODO
|
101
|
+
end
|
102
|
+
|
103
|
+
def terminate_instance(instance)
|
104
|
+
# TODO
|
105
|
+
end
|
106
|
+
end
|
metadata
ADDED
@@ -0,0 +1,77 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: tdl
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.1
|
5
|
+
prerelease:
|
6
|
+
platform: ruby
|
7
|
+
authors:
|
8
|
+
- Mo Morsi
|
9
|
+
autorequire:
|
10
|
+
bindir: bin
|
11
|
+
cert_chain: []
|
12
|
+
date: 2013-02-26 00:00:00.000000000 Z
|
13
|
+
dependencies:
|
14
|
+
- !ruby/object:Gem::Dependency
|
15
|
+
name: rspec
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
17
|
+
none: false
|
18
|
+
requirements:
|
19
|
+
- - ~>
|
20
|
+
- !ruby/object:Gem::Version
|
21
|
+
version: 2.11.0
|
22
|
+
type: :development
|
23
|
+
prerelease: false
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
25
|
+
none: false
|
26
|
+
requirements:
|
27
|
+
- - ~>
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: 2.11.0
|
30
|
+
description: Aeolus Template Description Language Tools
|
31
|
+
email: mmorsi@redhat.com
|
32
|
+
executables:
|
33
|
+
- tdl-launch.rb
|
34
|
+
- tdl-convert.rb
|
35
|
+
- tdl-verify.rb
|
36
|
+
- tdl-create.rb
|
37
|
+
extensions: []
|
38
|
+
extra_rdoc_files: []
|
39
|
+
files:
|
40
|
+
- lib/etdl_processor.rb
|
41
|
+
- lib/etdl.rb
|
42
|
+
- lib/cloud_inst.rb
|
43
|
+
- LICENSE
|
44
|
+
- Rakefile
|
45
|
+
- README.md
|
46
|
+
- bin/tdl-launch.rb
|
47
|
+
- bin/tdl-convert.rb
|
48
|
+
- bin/tdl-verify.rb
|
49
|
+
- bin/tdl-create.rb
|
50
|
+
homepage: http://aeolusproject.org/
|
51
|
+
licenses: []
|
52
|
+
post_install_message:
|
53
|
+
rdoc_options: []
|
54
|
+
require_paths:
|
55
|
+
- lib
|
56
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
57
|
+
none: false
|
58
|
+
requirements:
|
59
|
+
- - ! '>='
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: 1.8.1
|
62
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
63
|
+
none: false
|
64
|
+
requirements:
|
65
|
+
- - ! '>='
|
66
|
+
- !ruby/object:Gem::Version
|
67
|
+
version: 1.3.3
|
68
|
+
requirements:
|
69
|
+
- deltacloud is required to launch etdls against cloud instances
|
70
|
+
- imagefactory is required to build images from tdls
|
71
|
+
rubyforge_project:
|
72
|
+
rubygems_version: 1.8.24
|
73
|
+
signing_key:
|
74
|
+
specification_version: 3
|
75
|
+
summary: Aeolus tdl-tools
|
76
|
+
test_files: []
|
77
|
+
has_rdoc:
|