gosen 0.1.7 → 0.2.0

data/README.md

@@ -5,13 +5,14 @@ It relies on the [Restfully library](http://github.com/crohr/restfully) for inte
 
 ## Features
 
-Currently, it allows to submit deployments that retry automatically when too many nodes failed, similarly to [Katapult](http://www.loria.fr/~lnussbau/katapult.html).
+Currently, this library allows to submit deployments that retry automatically when too many nodes fail.
+A clone of the [Katapult](http://www.loria.fr/~lnussbau/katapult.html) script, called Trebuchet, is also included and can be used on the command line.
 
 ## Installation
 
     $ gem install gosen
 
-## Usage
+## Library Usage
 
 The following example deploys the latest version of the Lenny-x64-big environment on the paramount-1 and paramount-2 nodes.
 If both nodes are not successfully deployed, Gosen retries again (in this case, at most 5 deployment are submitted).
@@ -43,6 +44,24 @@ The logger allows to print information about the deployment, in a style similar
     I, [2010-04-21T11:37:11.817323 #21673]  INFO -- : Nodes deployed: paramount-1.rennes.grid5000.fr paramount-2.rennes.grid5000.fr
     I, [2010-04-21T11:37:11.817440 #21673]  INFO -- : Had to run 1 kadeploy runs, deployed 2 nodes
 
+The Gosen specific options accepted by Gosen::Deployment.new() are:
+
+* :logger, a Ruby Logger object,
+* :min_deployed_nodes, the minimal number of successfully deployed nodes (defaults to 1),
+* :max_deploy_runs, the maximal number of deploy runs to perform (defaults to 1),
+* :continue_if_error, a boolean allowing to retry if a deployment returned with an error (defaults to false),
+* :ssh_public_key, an SSH public key to be installed in the deployed environment.
+
+It is also possible to pass options accepted by the [Deployments API](https://api.grid5000.fr/sid/deployments/help/index.html), such as version, block_device, partition_number, etc.
+
+## Script usage
+
+Trebuchet was designed to be used like Katapult:
+
+    $ trebuchet -e lenny-x64-base --env-version 3 --min-deployed-nodes 4 --max-deploy-runs 2 -c
+
+Run `trebuchet --help` to get usage information.
+
 ## Note on Patches/Pull Requests
 
 * Fork the project.

data/VERSION

@@ -1 +1 @@
-0.1.7
+0.2.0

data/bin/trebuchet

@@ -0,0 +1,123 @@
+#!/usr/bin/env ruby
+
+require 'gosen'
+require 'logger'
+require 'optparse'
+require 'ostruct'
+
+trap('INT') {
+  deployment_resource = @deployment.deployment_resource rescue nil
+  if deployment_resource && deployment_resource['status'] == 'processing'
+    deployment_resource.delete
+    puts "Cancelled deployment #{deployment_resource['uid']}"
+  end
+  exit
+}
+
+config = OpenStruct.new
+deployment_options = {}
+config.nodes = []
+
+logger = Logger.new(STDOUT)
+logger.level = Logger::INFO
+deployment_options[:logger] = logger
+
+options = OptionParser.new do |opts|
+  opts.banner = 'Usage: trebuchet [options]'
+  opts.separator ''
+  opts.separator 'Specific options:'
+  opts.on('-b', '--block-device BLOCKDEVICE', 'Specify the block device to use') do |p|
+    deployment_options[:block_device] = p
+  end
+  opts.on('-c', '--copy-ssh-key', 'Copy SSH keys to nodes') do
+    config.copy_ssh_key = true
+  end
+  opts.on('--disable_bootloader_install', 'Disable the automatic installation of a bootloader for a Linux based environnment') do
+    deployment_options[:disable_bootloader_install] = true
+  end
+  opts.on('--disable_disk_partitioning', 'Disable the disk partitioning') do
+    deployment_options[:disable_disk_partitioning] = true
+  end
+  opts.on('-e', '--deploy-env ENV', 'Environment to deploy') do |env|
+    config.deploy_env = env
+  end
+  opts.on('--env-version NUMBER', 'Number of version of the environment to deploy') do |v|
+    deployment_options[:version] = v
+  end
+  opts.on('-f', '--file MACHINELIST', 'Files containing the list of nodes') do |f|
+    config.nodefile = f
+  end
+  opts.on('-i', '--ssh-key-file FILE', "File containing keys to copy (defaults to #{config.ssh_keyfile})") do |f|
+    config.ssh_keyfile = f
+  end
+  opts.on('--ignore-nodes-deploying', 'Allow to deploy even on the nodes tagged as "currently deploying" (use this only if you know what you do)') do
+    deployment_options[:ignore_nodes_deploying] = true
+  end
+  opts.on('-l', '--deploy-user USER', 'User owning the deployment environment') do |u|
+    config.deploy_user = u
+  end
+  opts.on('-m', '--machine MACHINE', 'Node to run on') do |n|
+    config.nodes << n
+  end
+  opts.on('--max-deploy-runs NB', 'Maximum number of deployment runs before we admit we cannot get enough nodes deployed') do |n|
+    deployment_options[:max_deploy_runs] = n.to_i
+  end
+  opts.on('--min-deployed-nodes NB', 'Minimum number of nodes that must be correctly deployed before continuing') do |n|
+    deployment_options[:min_deployed_nodes] = n.to_i
+  end
+  opts.on('-p', '--partition-number NUMBER', 'Specify the partition number to use') do |p|
+    deployment_options[:partition_number] = p
+  end
+  opts.on('-r', '--reformat-tmp FSTYPE', 'Reformat the /tmp partition') do |fs|
+    fstypes = [ "ext2", "ext3", "ext4" ]
+    abort "FSTYPE must be one of #{fstypes.join(', ')}" unless fstypes.include?(fs)
+    deployment_options[:reformat_tmp] = fs
+  end
+end
+
+begin
+  options.parse!(ARGV)
+rescue OptionParser::ParseError => e
+  $stderr.puts e
+  exit 1
+end
+
+if config.nodes.empty?
+  config.nodefile ||= ENV['OAR_NODEFILE']
+  config.nodes = File.open(File.expand_path(config.nodefile)).readlines.collect { |l| l.chomp }.sort.uniq
+end
+
+if config.nodes.empty?
+  abort "No nodes specified, and no OAR_NODEFILE variable, exiting.\nRun trebuchet --help if you need help."
+end
+
+if config.deploy_env.nil?
+  abort "Error: an environment to deploy is required.\nRun trebuchet --help if you need help."
+end
+
+if config.deploy_user
+  config.deploy_env += "@#{config.deploy_user}"
+end
+
+if config.ssh_keyfile
+  abort 'Error: the --ssh-keyfile option only makes sense with --copy-ssh-key.\nRun trebuchet --help if you need help.' if config.copy_ssh_key.nil?
+end
+
+if config.copy_ssh_key
+  config.ssh_keyfile ||= '~/.ssh/authorized_keys'
+  deployment_options[:ssh_public_key] = File.open(File.expand_path(config.ssh_keyfile)).read
+end
+
+if ENV['RESTFULLY_CONFIG'].nil?
+  abort "No RESTFULLY_CONFIG environment variable, exiting.\nRun trebuchet --help if you need help."
+end
+
+Restfully::Session.new({ :configuration_file => ENV['RESTFULLY_CONFIG'] }) do |grid, session|
+  indexed_nodes = Gosen::index_nodes_by_site(config.nodes)
+  if indexed_nodes.keys.length > 1
+    raise StandardError.new("The node list needs to be specific to a single site\nRun trebuchet --help if you need help.")
+  end
+  site = grid.sites[indexed_nodes.keys.first.to_sym].load
+  @deployment = Gosen::Deployment.new(site, config.deploy_env, config.nodes, deployment_options)
+  @deployment.join
+end

data/lib/gosen.rb

@@ -4,7 +4,7 @@ require 'gosen/deployment'
 require 'gosen/error'
 
 module Gosen
-  VERSION = "0.1.7"
+  VERSION = "0.2.0"
 
   # Extracts the site part of a Grid'5000 node hostname, and returns a hash indexing the nodes by their site
   # Example:

data/lib/gosen/deployment.rb

@@ -80,6 +80,10 @@ module Gosen
       raise Gosen::Error.new("Not enough nodes deployed after #{@max_deploy_runs} deployment(s): needed #{@min_deployed_nodes} nodes, got only #{@good_nodes.length}")
     end
 
+    def deployment_resource
+      @deployment_run.deployment_resource rescue nil
+    end
+
     def no_more_required?
       @good_nodes.length >= @min_deployed_nodes
     end

data/test/gosen/test_deployment.rb

@@ -126,6 +126,23 @@ class TestDeployment < Test::Unit::TestCase
         @deployment_resource.stubs(:[]).with('status').returns('processing', 'processing', 'terminated')
       end
 
+      should 'have a reader on the deployment resource' do
+        @deployment_result = {
+          'paramount-1.rennes.grid5000.fr' => { 'state' => 'OK' },
+          'paramount-2.rennes.grid5000.fr' => { 'state' => 'OK' }
+        }
+        @deployment_resource.expects(:[]).with('result').returns(@deployment_result)
+        @site_deployments.expects(:submit).with({ :environment => @environment, :nodes => @nodes }).returns(@deployment_resource)
+        @min_deployed_nodes = 2
+        @logger.expects(:info).with("Kadeploy run 1 with #{@nodes.length} nodes (0 already deployed, need #{@min_deployed_nodes} more)")
+        @logger.expects(:info).with("Nodes deployed: paramount-1.rennes.grid5000.fr paramount-2.rennes.grid5000.fr")
+        @logger.expects(:info).with("Had to run 1 kadeploy runs, deployed #{@deployment_result.length} nodes")
+
+        @deployment = Gosen::Deployment.new(@site, @environment, @nodes, { :logger => @logger, :min_deployed_nodes => @min_deployed_nodes })
+        @deployment.join
+        assert_equal(@deployment_resource, @deployment.deployment_resource)
+      end
+
       should 'submit a deployment run and wait for the result' do
         @deployment_result = {
           'paramount-1.rennes.grid5000.fr' => { 'state' => 'OK' },

data/test/gosen/test_deployment_run.rb

@@ -81,6 +81,11 @@ class TestDeploymentRun < Test::Unit::TestCase
       @deployment = Gosen::DeploymentRun.new(@site, @environment, @nodes, { :ssh_public_key => @ssh_public_key })
     end
 
+    should 'have a reader on the deployment resource' do
+       @deployment.join
+       assert_equal(@resource, @deployment.deployment_resource)
+    end
+
     should 'wait for deployment completion and give access to the results' do
       assert_nothing_raised(Exception) {
         @deployment.join

metadata

@@ -4,9 +4,9 @@ version: !ruby/object:Gem::Version
   prerelease: false
   segments: 
   - 0
-  - 1
-  - 7
-  version: 0.1.7
+  - 2
+  - 0
+  version: 0.2.0
 platform: ruby
 authors: 
 - Pierre Riteau
@@ -14,8 +14,8 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-10-26 00:00:00 +02:00
-default_executable: 
+date: 2010-10-28 00:00:00 +02:00
+default_executable: trebuchet
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: restfully
@@ -64,8 +64,8 @@ dependencies:
   version_requirements: *id003
 description: Gosen is a Ruby library providing high-level operations using the Grid'5000 RESTful API, such as Kadeploy deployments.
 email: priteau@gmail.com
-executables: []
-
+executables: 
+- trebuchet
 extensions: []
 
 extra_rdoc_files: 
@@ -78,6 +78,7 @@ files:
 - README.md
 - Rakefile
 - VERSION
+- bin/trebuchet
 - lib/gosen.rb
 - lib/gosen/deployment.rb
 - lib/gosen/deployment_run.rb