lab 0.0.3 → 0.0.4

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in lab.gemspec
+gemspec

data/Gemfile.lock

@@ -0,0 +1,16 @@
+PATH
+  remote: .
+  specs:
+    lab (0.0.1)
+      nokogiri
+
+GEM
+  remote: http://rubygems.org/
+  specs:
+    nokogiri (1.5.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  lab!

data/README

@@ -0,0 +1,77 @@
+This folder contains the libraries necessary to run the lab plugin, and can also be used in a standalone way to automate virtual machines.
+
+==CONCEPTS:
+
+The lab provides a clean interface to common vm functions such as start / stop / snapshot / revert and even running system commands or higher-level functions like opening a browser to a specified URL. It's designed so the different VM technologies have a similiar interface, and you can ignore the specifics of the VM tech. The majority of the functionality is implemented in the form of drivers and controllers. Drivers implement the underlying command for each vm software (such as start/stop/revert), and controllers implement the commands which apply to all vms (such as listing all running vms, or cloning a vm). 
+
+If you're interested in porting a vm software (see below), please take a look at the workstation_driver.rb and the workstation_controller.rb -- This is a simple driver / controller in the lab, and you can simply copy / modify this to implement a new driver & controller for the software. 
+
+==SUPPORTED VM TECHNOLOGIES:
+
+NOTE: The lab libraries have only been tested on linux, porting to windows is not planned at this time.
+
+Implemented:
+ - workstation (Tested against 7.x)
+ - remote_workstation (Tested against 7.x)
+ - virtualbox (Tested against 4.x)
+ - remote_esx (VMware ESX Host Agent 4.1.0 build-348481)
+
+Partially Implemented:
+ - amazon_ec2 (via fog gem)
+ - dynagen
+
+Need Implementation:
+ - qemu
+ - qemudo
+ - others?
+
+==PLATFORM SUPPORT:
+
+You will need to have this code running on a linux box, Currently this has only been run / tested on Ubuntu 9.04 -> 10.04, though it should run on any linux with an ssh client and the dependencies below. Remote VM Hosts will need to be linux as well, though other platforms may work (untested). If you're interested in porting it to windows, please contact me (jcran). 
+
+Platform Dependencies:
+  - whatever vm software is necessary for the driver you're using (see SUPPORTED VM TECHNOLOGIES above)
+  - net/scp - the gem (net-scp). Required to copy files to/from the devices in the case that tools are not installed. Not necessary if tools are installed. 
+  - fog - require to use the amazon_ec2 driver
+
+==STANDALONE API:
+
+BACKGROUND:
+
+The lab libraries add tons of useful functionality that isn't exposed through the lab plugin, such as the ability to run commands on hosts. This library can serve as an excellent base for more complex operations on a remote host as well. 
+
+USAGE:
+
+You must first create a yaml file which describes your vm. See data/lab/test_targets.yml for an example.  
+
+  require 'vm_controller'
+  vm_controller = ::Lab::Controllers::VmController.new(YAML.load_file(lab_def)) 
+  vm_controller['vm1'].start
+  vm_controller['vm1'].snapshot("clean") 
+  vm_controller['vm1'].run_command("rm /etc/resolv.conf")
+  vm_controller['vm1'].open_uri("http://autopwn:8080")
+  vm_controller['vm1'].revert("clean")
+  vm_controller['vm1'].revert("clean")
+
+==METASPLOIT MSFCONSOLE LAB PLUGIN:
+
+BACKGROUND:
+
+The lab plugin for msfconsole adds a number of commands which may be useful if you're interested in automating remote hosts with rc scripts, or if you need to control targets / support systems while utilizing the metasploit console. A potential use case is testing an IPS / IDS, and resetting the target after running each exploit. 
+
+USAGE:
+
+Here's some example usage for the lab plugin. 
+
+  msf> load lab						// Loads the lab plugin
+  msf> lab_load <path_to_lab_file> 			// Loads from a lab configuration file. See data/lab/test_targets.yml for an example
+  msf> lab_load_dir workstation /path/to/vmx/files	// Loads from a local directory.
+  msf> lab_load_running remote_esx root esx_server	// Loads all running vms. 
+  msf> lab_start vm1					// Start a vm which was loaded above
+  msf> lab_snapshot vm1 snapshot_1			// Snapshot a vm as 'snapshot_1'
+  msf> lab_run_command ("rm -rf /")			// oops!
+  msf> lab_show 					// Show all vms that we're aware of
+  msf> lab_show_running					// Show only running vms
+  msf> lab_start vm2					// Start another vm
+  msf> lab_suspend  vm1					// Suspend a vm
+  msf> lab_revert all snapshot_1			// Revert all vms back to 'snapshot_1'

data/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

data/TODO

@@ -0,0 +1,15 @@
+This is a list of basic priorities for the lab code...
+
+* Implement more technologies
+
+ a) finish amazon ec2 (via fog)
+ b) qemu
+ c) qemudo
+ d) kvm
+ e) other cloud technologies (newservers, slicehost/rackspace,etc)
+
+* Implement a cloning function on each controller
+
+* Support Windows as a host platform. Currently all the code assumes a linux host is running it. The same applies for the remote_* drivers -- they've not been tested on windows.
+
+* Consolidate the remote_system_command code & provide a filter. Create an unsafe_system_command and unsafe_remote_system_command function call for when we control the entire string.

data/lab.gemspec

@@ -0,0 +1,24 @@
+# -*- encoding: utf-8 -*-
+$:.push File.expand_path("../lib", __FILE__)
+require "lab/version"
+
+Gem::Specification.new do |s|
+  s.name        = "lab"
+  s.version     = Lab::VERSION
+  s.authors     = ["Jonathan Cran"]
+  s.email       = ["jcran@pentestify.com"]
+  s.homepage    = "http://www.pentestify.com/projects/lab"
+  s.summary     = %q{Manage vms like a boss.}
+  s.description = %q{Start/Stop/Revert and do other cool stuff w/ Vmware, Virtualbox, and ESXi vms}
+
+  s.rubyforge_project = "lab"
+
+  s.files         = `git ls-files`.split("\n")
+  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
+  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.require_paths = ["lib"]
+
+  # specify any dependencies here; for example:
+  # s.add_development_dependency "rspec"
+   s.add_runtime_dependency "nokogiri"
+end

data/lib/lab.rb

@@ -0,0 +1,4 @@
+# TODO - use autoload 
+
+require 'lab/vm_controller'
+require 'lab/version'

data/lib/lab/controller/dynagen_controller.rb

@@ -0,0 +1,14 @@
+module Lab
+module Controllers
+module DynagenController
+
+	def self.running_list
+		raise "Unsupported"
+	end
+
+	def self.dir_list(basepath=nil)
+		raise "Unsupported"
+	end
+end
+end
+end

data/lib/lab/controller/fog_controller.rb

@@ -0,0 +1,6 @@
+module Lab
+module Controllers
+module FogController
+end
+end
+end

data/lib/lab/controller/remote_esx_controller.rb

@@ -0,0 +1,62 @@
+# This controller was built against: 
+# VMware ESX Host Agent 4.1.0 build-348481
+
+module Lab
+module Controllers
+module RemoteEsxController
+		
+	# Note that 3.5 was different (vmware-vim-cmd)
+	VIM_CMD = 'vim-cmd'.freeze
+
+	def self.dir_list(basepath=nil)
+		# Does this method really even make sense for esx?
+		return "Unsupported :("
+	end
+
+	def self.running_list(user, host)
+		user.gsub!(/(\W)*/, '')
+		host.gsub!(/(\W)*/, '')
+
+		# first get all registered vms
+		registered_vms = self.get_vms(user, host) || []
+		running_vms = []
+
+		# now let's see which ones are running
+		# TODO:  this is ghetto, would be better not to connect repeatedly
+		registered_vms.each do |vm|
+			remote_cmd = "ssh #{user}@#{host} \"#{VIM_CMD} vmsvc/power.getstate #{vm[:id]}\""
+			raw = `#{remote_cmd}`
+			running_vms << vm if raw =~ /Powered on/			
+		end
+
+		return running_vms
+	end
+
+private 
+
+	def self.get_vms(user, host)
+		user.gsub!(/(\W)*/, '')
+		host.gsub!(/(\W)*/, '')
+		
+		vms = [] # array of VM hashes
+		remote_cmd = "ssh #{user}@#{host} \"#{VIM_CMD} vmsvc/getallvms | grep ^[0-9] | sed 's/[[:blank:]]\\{3,\\}/ /g'\""
+		raw = `#{remote_cmd}`.split("\n")
+
+		raw.each do |line|
+			# So effing ghetto
+			id_and_name = line.split('[datastore').first
+			id = id_and_name.split(' ').first
+	
+			## TODO - there's surely a better way to do this.
+			name_array = id_and_name.split(' ')
+			name_array.shift
+			name = name_array.join(' ')
+			vms << {:id => id, :name => name}
+		end
+		
+		return vms
+	end
+	
+end
+end
+end

data/lib/lab/controller/remote_workstation_controller.rb

@@ -0,0 +1,22 @@
+module Lab
+module Controllers
+module RemoteWorkstationController
+
+	def self.running_list(user, host)
+		user.gsub!(/(\W)*/, '')
+		host.gsub!(/(\W)*/, '')
+
+		remote_cmd = "ssh #{user}@#{host} \"vmrun list nogui\""
+		vm_list = `#{remote_cmd}`.split("\n")
+		vm_list.shift
+
+		return vm_list
+	end
+
+	def self.dir_list(basepath=nil)
+		vm_list = Find.find(basepath).select { |f| f =~ /\.vmx$/ }
+		return vm_list
+	end
+end
+end
+end

data/lib/lab/controller/virtualbox_controller.rb

@@ -0,0 +1,25 @@
+module Lab
+module Controllers
+module VirtualBoxController
+
+	def self.running_list
+		vm_names_and_uuids = `VBoxManage list runningvms`
+		return vm_names_and_uuids.scan(/\"(.*)\" {.*}/).flatten
+	end
+
+	def self.config_list
+		vm_names_and_uuids = `VBoxManage list vms`
+		return vm_names_and_uuids.scan(/\"(.*)\" {.*}/).flatten
+	end
+
+	def self.config_list_uuid
+		vm_names_and_uuids = `VBoxManage list vms`
+		return vm_names_and_uuids.scan(/\".*\" {(.*)}/).flatten
+	end
+		
+	def self.dir_list(basepath=nil)
+		vm_list = Find.find(basepath).select { |f| f =~ /\.xml$/ }
+	end
+end
+end
+end

data/lib/lab/controller/workstation_controller.rb

@@ -0,0 +1,17 @@
+module Lab
+module Controllers
+module WorkstationController
+
+	def self.running_list
+		vm_list = `vmrun list`.split("\n")
+		vm_list.shift
+		return vm_list
+	end
+
+	def self.dir_list(basepath=nil)
+		vm_list = Find.find(basepath).select { |f| f =~ /\.vmx$/ }
+		return vm_list
+	end
+end
+end
+end

data/lib/lab/controller/workstation_vixr_controller.rb

@@ -0,0 +1,19 @@
+module Lab
+module Controllers
+module WorkstationVixrController
+
+	def self.running_list
+		vm_list = `vmrun list`.split("\n")
+		vm_list.shift
+
+		return vm_list
+	end
+
+	def self.dir_list(basepath=nil)
+		vm_list = Find.find(basepath).select { |f| f =~ /\.vmx$/ }
+
+		return vm_list
+	end
+end
+end
+end

data/lib/lab/controllers.rb

@@ -0,0 +1,10 @@
+require 'controller/workstation_controller'
+require 'controller/virtualbox_controller'
+require 'controller/fog_controller'
+require 'controller/dynagen_controller'
+require 'controller/remote_workstation_controller'
+require 'controller/remote_esx_controller'
+#require 'controller/qemu_controller'
+#require 'controller/qemudo_controller'
+
+

data/lib/lab/driver/dynagen_driver.rb

@@ -0,0 +1,47 @@
+require 'vm_driver'
+
+#
+# $Id$
+#
+
+#
+# To use this driver, you have to have a lab which is preconfigured. The best / easiest
+# way i've found to to set up a lab is GNS3
+# 
+
+module Lab
+module Drivers
+	class DynagenDriver < VmDriver
+		def initialize(config,dynagen_config)
+			super(config)
+			@running = false
+			@dynagen_platform = filter_command(dynagen_config['dynagen_platform'])
+		end
+
+		def start
+			# TODO - write the location-file to a temp-file 
+			#        and set the autostart property 
+
+			## start background dynamips process
+			system_command("dynamips -H #{@dynagen_platform} &")
+			system_command("dynagen	#{@location}")
+			@running = true
+		end
+
+		def stop
+			system_command("killall dynagen")
+			@running = false
+		end
+
+		def cleanup
+			`killall dynagen`
+			`killall dynamips`
+			@running = false
+		end
+
+		def running?
+			return @running
+		end
+	end
+end
+end

data/lib/lab/driver/fog_driver.rb

@@ -0,0 +1,164 @@
+require 'vm_driver'
+
+##
+## $Id$
+##
+
+module Lab
+module Drivers
+class FogDriver < VmDriver
+
+	def initialize(config,fog_config)
+		
+		super(config)
+		@fog_config = fog_config
+
+		# Soft dependency
+		begin
+			require 'fog'
+		rescue LoadError
+			raise "WARNING: Library fog not found. Could Not Create Driver"
+		end
+
+		if @fog_config['fog_type'] == "ec2"
+
+			# AWS / EC2 Base Credential Configuration
+			@aws_cert_file = IO.read(fog_config['fog_aws_cert_file']).chomp if fog_config['fog_aws_cert_file']
+			@aws_private_key_file = IO.read(fog_config['fog_aws_private_key_file']).chomp if fog_config['fog_aws_private_key_file']
+			@ec2_access_key_file = IO.read(fog_config['fog_ec2_access_key_file']).chomp if fog_config['fog_ec2_access_key_file']
+			@ec2_secret_access_key_file = IO.read(fog_config['fog_ec2_secret_access_key_file']).chomp if fog_config['fog_ec2_secret_access_key_file']
+			
+			# Instance Keys
+			@ec2_instance_public_key_file = IO.read(fog_config['fog_ec2_instance_public_key_file']).chomp if fog_config['fog_ec2_instance_public_key_file']
+			@ec2_instance_private_key_file = IO.read(fog_config['fog_ec2_instance_private_key_file']).chomp if fog_config['fog_ec2_instance_private_key_file']
+			
+			# Instance Details
+			@ec2_base_ami = fog_config['fog_ec2_base_ami']
+			@ec2_flavor = fog_config['fog_ec2_flavor']
+			@ec2_user = fog_config['fog_ec2_user']
+			@ec2_region = fog_config['fog_ec2_region']
+					
+			# Set up a connection
+			@compute = Fog::Compute.new(
+				:provider => "Aws",
+				:aws_access_key_id => @aws_access_key_file,
+				:aws_secret_access_key => @aws_secret_access_key_file )
+		else
+			raise "Unsupported Fog Type"
+		end
+	end
+
+	def start
+		ec2_settings = {
+			:image_id => @ec2_base_ami, 
+			:flavor_id =>  @ec2_flavor,
+			:public_key_path => @ec2_instance_public_key_file,
+			:private_key_path => @ec2_instance_private_key_file,
+			:username => @ec2_user}
+			
+		begin
+			@fog_server = @compute.servers.bootstrap(ec2_settings)
+		rescue Fog::Compute::AWS::Error => e
+			raise "Couldn't authenticate to AWS - did you place keys in the creds/ directory?"
+			exit
+		end
+	end
+
+	def stop
+		@fog_server.destroy
+	end
+
+	def suspend
+		raise "unimplemented"
+	end
+
+	def pause
+		raise "unimplemented"
+	end
+
+	def reset
+		raise "unimplemented"
+	end
+
+	def create_snapshot(snapshot)
+		raise "unimplemented"
+	end
+
+	def revert_snapshot(snapshot)
+		raise "unimplemented"
+	end
+
+	def delete_snapshot(snapshot)
+		raise "unimplemented"
+	end
+
+=begin
+
+	def run_command(command)
+		## vm_driver will need a little patching for this to work, as 
+		## amis use keys for auth. i think it's just a matter of not passing the 
+		## password to ssh_exec. So maybe the thing to do is have a ssh_key_exec
+		## function in vm_driver.rb that does the right thing. 
+
+		script_rand_name = rand(10000)
+
+		if @os == "windows"
+			local_tempfile_path = "/tmp/lab_script_#{script_rand_name}.bat"
+			remote_tempfile_path = "C:\\\\lab_script_#{script_rand_name}.bat"
+			remote_run_command = remote_tempfile_path
+		else
+			local_tempfile_path = "/tmp/lab_script_#{script_rand_name}.sh"
+			remote_tempfile_path = "/tmp/lab_script_#{script_rand_name}.sh"
+			remote_run_command = "/bin/sh #{remote_tempfile_path}"
+		end
+
+		# write out our script locally
+		File.open(local_tempfile_path, 'w') {|f| f.write(command) }
+
+		# since we can't copy easily w/o tools, let's just run it directly :/
+		if @os == "linux"
+			output_file = "/tmp/lab_command_output_#{rand(1000000)}"
+			
+			scp_to(local_tempfile_path, remote_tempfile_path)
+			ssh_exec(remote_run_command + "> #{output_file}")
+			scp_from(output_file, output_file)
+			ssh_exec("rm #{output_file}")
+			ssh_exec("rm #{remote_tempfile_path}")
+			
+			# Ghettohack!
+			string = File.open(output_file,"r").read
+			`rm #{output_file}`
+			
+		else
+			raise "zomgwtfbbqnotools"
+		end
+	end
+	
+	def copy_from(from, to)
+		raise "unimplemented"
+	end
+
+	def copy_to(from, to)
+		raise "unimplemented"
+	end
+
+	def check_file_exists(file)
+		raise "unimplemented"
+	end
+
+	def create_directory(directory)
+		raise "unimplemented"
+	end
+=end
+
+	def cleanup
+		@fog_server.destroy
+	end
+
+	def running?
+		return true #TODO
+	end
+
+end
+end 
+end