aws-must-templates 0.1.6 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: a561a1094145391bcc56b1b4c91d4dd2daded2c8
-  data.tar.gz: 972049e8dd61b986b738c61533be83d1bb3cc2f2
+  metadata.gz: f7ef597a5063b288cc1c4fba6f4a2119cf008391
+  data.tar.gz: b89652f4001fea744e1be3e6d42233ef154bc15f
 SHA512:
-  metadata.gz: 46193b67910910d382a950cf47d773c3568fe7c8b5ac20e71bbd19d7423db8318aebbe77deaf8b3c693d3d576147a294548274cd264911974cb71f399aa4444c
-  data.tar.gz: d2a0e4a27b32d51393f9569b935723efe1035f192572e659fa17b5c97f1ddfe69d3c029df5b4b0c9055765038e1b9bcb52a94e3decd391a11244e54cd1a5ac91
+  metadata.gz: 0f17a8a069e5249a231ec574e3b801e5149a2a7c54aef1f5e98dc5d0cbbacbbc76280568280aaa9fee9d85ea332656ed4fa9048fa7ab58cc2c88a6cd47cbb2ec
+  data.tar.gz: 024b79c1a947156e3ea98347a5312c832188bbb7ace1b09cd37fd54292051a4a1817e45e5c5023b1571ef0cdfeeb8d1a3e1c8f2e12c094e56e80c766a53e8a61

data/README.md

@@ -1,11 +1,13 @@
-# aws-must-templates - cloudformation templates for aws-must - $Release:0.1.6$
+# aws-must-templates - cloudformation templates for aws-must - $Release:0.2.1$
 
 Set of [extensible](#OVERRIDE)
 [templates](https://rawgit.com/jarjuk/aws-must-templates/master/generated-docs/aws-must-templates.html)
 for [aws-must](https://github.com/jarjuk/aws-must) tool to generate
 CloudFormation JSON from a YAML configuration, and a
-[Test Runner](#TESTING) for [validating correctness](#TEST_CASES) of
-CloudFormation stacks provisioned.
+[Test Runner](#TESTING) with
+[reusable tests](https://rawgit.com/jarjuk/aws-must-templates/master/generated-docs/aws-must-templates-spec.html)
+for [validating correctness](#TEST_CASES) of CloudFormation stacks
+provisioned.
 
 See blog posts
 [part1](https://jarjuk.wordpress.com/2015/08/18/announcing-aws-must-templates-part1)
@@ -50,8 +52,8 @@ When using code generators, consider
    the threshold to write tests
 
 3. by allowing users [to reuse](#TEST_CASES)
-   [test cases](generated-docs/test-suites.md) used
-   in **aws-must-templates** quality assurance
+   [test cases](https://rawgit.com/jarjuk/aws-must-templates/master/generated-docs/aws-must-templates-spec.html)
+   from **aws-must-templates** quality assurance
 
 4. by having the possibility to [include own test](#TEST_CASES)
    cases to test suites
@@ -90,8 +92,12 @@ used to validate **aws-must-templates** implementation. For, example
   bucket, one of the instances (`myInstance`) which is granted a read
   access to the S3 bucket
   
-See [test report](generated-docs/test-suites.md) of
-**aws-must-templates** for more information.
+* [suite2.yaml](suite2.yaml): Creates VPC with Public and Private
+         Subnets (NAT) similar to
+         [scenario 2](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html)
+  
+See [test report](https://gist.github.com/jarjuk/9ab1c25d436c4e468f5e)
+of **aws-must-templates** for more information.
 
 ### Generate CloudFormation JSON templates
 
@@ -189,18 +195,29 @@ To use the **aws-must-templates** Test Runner
 
 Add following lines to `Rakefile`
 
+    # name of configuration file
+    suite_runner_configs= "suite-runner-configs.yaml"
+
+    # Override configuration in 'suite.rake' 
+    $suite_runner_configs = File.exist?(suite_runner_configs) ? YAML.load_file( suite_runner_configs ) : {}
+
 	spec = Gem::Specification.find_by_name 'aws-must-templates'
 	load "#{spec.gem_dir}/lib/tasks/suite.rake"
 	
-create an empty `test-suites.yaml`	 -file
+Optionally, copy
+[default Test Runner Configurations](https://rawgit.com/jarjuk/aws-must-templates/master/suite-runner-configs.yaml)
+to `suite-runner-configs.yaml` file in current working directory, and
+modify settings in the file to override default values.
+	
+Create an empty `test-suites.yaml`	 -file
 
 	touch test-suites.yaml
 
 and run 	
 
-	bundle exec rake -T suite
+	rake -T suite
 	
-For an empty test-suites.yaml the result shows
+For an empty `test-suites.yaml`, the result shows
 
 	rake suite:all[gen_opts]  # Run all suites
 
@@ -220,31 +237,60 @@ pair
 
 **Prepare ssh-connection configuration**
 
-Test Runner uses SSH connection to access an EC2 instance, and needs
-an entry in `ssh/config` file
-[defining](http://linux.die.net/man/5/ssh_config) user name, and
-ssh-key used in authentication, etc.
+Test Runner uses
+[SSH Client Configuration](http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man5/ssh_config.5?query=ssh_config&sec=5)
+in `ssh/config.aws` file. Configuration file `ssh/config.aws` is
+created automatically, using `ssh/config.init`, if it does not exist.
 
-For example, the SSH configuration below for an EC2 instance
-`myInstance` defines `IdentityFile` parameter matching `demo-key` EC2
-key used in **aws-must-templates** test suites.
+For example, the following configuration in `ssh/config.init`
+instructs OpenSSH 
 
-     host myInstance
-         StrictHostKeyChecking no
-         UserKnownHostsFile=/dev/null
-         user ubuntu
-         IdentityFile ~/.ssh/demo-key/demo-key
+    Host *.internal
+        ProxyCommand ssh myFront1 -F ssh/config.aws nc -q0 %h 22
+
+
+    # using RHEL Amazon vpc-nat instance --> ec2-user
+    Host myNat
+         user ec2-user
+
+    Host *
+        user ubuntu
+        StrictHostKeyChecking no
+        UserKnownHostsFile=/dev/null
+        IdentityFile ~/.ssh/demo-key/demo-key
+		
+		
+* to use user name `ubuntu` and SSH private key in
+  `~/.ssh/demo-key/demo-key` for all hosts, expect for instance
+  `myNat` user name is `ec2-user`. 
+  
+* to proxy connections to host names ending `.internal` to proxy
+  connections over instance `myFront1`
 
-Parameters `UserKnownHostsFile` and `StrictHostKeyChecking` prevent
-ssh from updating your default `.ssh/known_hosts` file with the
-fingerprint of the (temporary) instance used in testing.
+* to prevent OpenSSH from updating your default `.ssh/known_hosts`
+  file with the fingerprint of the (temporary) instance used in
+  testing.
+
+Once configuration in `ssh/config.init` is in place, running
+
+	rake suite:ec2-sync
+
+updates EC2 instance metadata in `ssh/config.aws`.
+	
+**Notice** You may need to change `aws_region` setting values in
+[suite-runner-configs.yaml](#SETUP-TEST-RUNNER).
+
+See
+[blog post](https://jarjuk.wordpress.com/2015/09/08/using-openssh-on-aws-platform/#more-273https://jarjuk.wordpress.com/2015/09/08/using-openssh-on-aws-platform)
+for more information on using OpenSSH on AWS platform.
 
 ### Implement Test Cases<a id="TEST_CASES"/>
 
-Test Runner searches Test Cases first from sub-directories under
-`spec/aws-must-templates` directory, and if not found, tries to locate
-Test Cases in Gem **aws-must-templates**. This allows user to reuse
-existing test in **aws-must-templates** and implement own Test Cases.
+Test Runner allows user to implement own Test Cases, or to
+[reuse](https://rawgit.com/jarjuk/aws-must-templates/master/generated-docs/aws-must-templates-spec.html)
+existing tests in **aws-must-templates**.  It first searches Test
+Cases locally under sub-directory `spec/aws-must-templates`, and if
+not found, tries to locate Test Cases in Gem **aws-must-templates**.
 
 Test runner uses Rspec with [serverspec](http://serverspec.org/)
 library.  For example, `ValidOSVersion` Test Case in
@@ -269,9 +315,9 @@ with the code shown:
 
 
 For more information on Test Cases, see
-[test report](generated-docs/test-suites.md) created, when running
-[test-suites](test-suites.yaml) in **aws-must-templates** development,
-and a [diagram](generated-docs/xref_suite_X_test.pdf) for an overview.
+[test report](https://gist.github.com/jarjuk/9ab1c25d436c4e468f5e)
+generated, when running [test-suites](test-suites.yaml) in
+**aws-must-templates** development.
 
 
 ### Configure Test Suites <a id="TEST-SUITES"/>
@@ -284,12 +330,12 @@ The picture below present main elements used in `test-suites.yaml`.
 ![test-suites.yaml elements](./pics/test-suites.jpg)
 
 A Test Suite validates correctness of a CloudFormation Stack. One Test
-Suite defines tests for multiple EC2 Instances. Each EC2 Instance must
-have a corresponding SSH Connection prepared in
-[ssh/config](#TEST-CONTEXT) -file. An EC2 Instance acts in many
-Roles. A Role maps to a [Test Case](#TEST_CASES), and and defines
-values for the Test Case Parameters. The parameter may be a constant,
-or a reference to Stack Parameter, or to Stack Output.
+Suite defines tests for multiple EC2 Instances. SSH Client
+Configuration for a EC2 Instance is looked up in
+[ssh/config.aws](#TEST-CONTEXT) -file. An EC2 Instance acts in many
+Roles. A Role maps to a [Test Case](#TEST_CASES), and defines values
+for the Test Case Parameters. The parameter may be a constant, or a
+reference to Stack Parameter, or to Stack Output.
 
 As an example, the Test Suite for `mystack` is 
 
@@ -327,16 +373,23 @@ a more detailed explanation, and for more examples.
 
 **NOTICE** It advisable to check on AWS console that all stack
   resources are deleted successfully after running test suites.
+  
+**Notice** Some of tests included in **aws-must-templates** use AWS
+  [SDK for Ruby - Version 2](http://docs.aws.amazon.com/sdkforruby/api/index.html). The
+  SDK searches `ENV['AWS_REGION']` for a region information. You have
+  the option to set the environment variable, or configure property
+  `aws_region` in [suite-runner-configs.yaml](#SETUP-TEST-RUNNER).
+  
 
 To run test suite `mystack` defined in in `test-suites.yml` using
 default templates in **aws-must-templates** use the command
 
-    bundle exec rake suite:mystack
+    rake suite:mystack
 	
 To [override](#OVERRIDE) default implementation with templates in
 directory `myextensions`, use the command
 
-	bundle exec rake suite:mystack['-m myextensions/ aws-must-templates']
+	rake suite:mystack['-m myextensions/ aws-must-templates']
 
 
 For a Test Suite, Test Runner
@@ -345,17 +398,17 @@ For a Test Suite, Test Runner
 * uses the JSON template to provision a stack on Amazon platform, 
 * and, once the `StackStatus` is `CREATE_COMPLETE`, 
 * iterates EC2 Instances and runs Test Cases in the EC2 Instance Role
-* creates a test report to `generated-doc/suites` directory
+* creates test reports to `generated-doc/suites` directory
 * finally, after the test execution, the stack is deleted from Amazon platform.
 
 
 To run all test suites defined in `test-suites.yaml`, use the command
 
-	bundle exec rake suite:all
+	rake suite:all
 	
 or to override the default implementation
 
-	bundle exec rake suite:all['-m myextensions/ aws-must-templates']
+	rake suite:all['-m myextensions/ aws-must-templates']
 
 Command
 
@@ -375,7 +428,7 @@ See [RELEASES](RELEASES.md)
 
 Add more tests, e.g.
 
-* VPC and subnets
+* SNS
 * install Chef
 
 Add more template support

data/lib/tasks/cross-ref.rb

@@ -49,6 +49,12 @@ def xref_to_dot( xref_suite_X_test, xref_test_X_suite )
   puts <<-EOS
           digraph {
              rankdir=TB
+             
+             // options for neato 
+             // http://stackoverflow.com/questions/7670304/how-to-deal-with-densely-connected-graphs-with-neato
+             overlap=false;
+             splines=true;
+
 
              node      [     fontname = "Courier"
                              fontsize = 8

data/lib/tasks/suite.rake

@@ -8,6 +8,18 @@ require 'json'
 require 'rake'
 require 'rspec/core/rake_task'
 
+# ------------------------------------------------------------------
+# configuration extesions (see task 'suite-runner-configs' for documentation)
+
+default_suite_runner_configs = {
+  'ssh_config_file' => "ssh/config.aws",
+  'ssh_config_init' => "ssh/config.init",
+  'aws_region'      => nil,
+}
+
+suite_runner_configs = 
+  default_suite_runner_configs.merge( $suite_runner_configs ? $suite_runner_configs : {} )
+
 # ------------------------------------------------------------------
 # test-suites.yaml
 
@@ -33,6 +45,45 @@ END_STATES     = SUCESS_STATES + FAILURE_STATES
 
 namespace :suite do
 
+  # **********
+  # desc "Outpu configuration"
+  task "suite-runner-configs" do
+
+    # puts( default_suite_runner_configs.to_yaml)
+    doc = <<EOS
+---
+# Test Runner suite-runner-default
+#  ------------------------------------------------------------------
+# 
+# copy this file to cwd with the name 'suite-runner-configs.yaml' and
+# add following lines to Rakefile
+#
+#    suite_runner_configs= "suite-runner-configs.yaml"
+#    $suite_runner_configs = File.exist?(suite_runner_configs) ? YAML.load_file( suite_runner_configs ) : {}
+#
+#    spec = Gem::Specification.find_by_name 'aws-must-templates'
+#    load "\#{spec.gem_dir}/lib/tasks/suite.rake"
+#
+#  ------------------------------------------------------------------
+
+# SSH Client Configuration file where EC2 Instance Tag Name/DNS Name
+# mapping is synchronized
+ssh_config_file: #{default_suite_runner_configs['ssh_config_file']}
+
+# Name of file, which is used to seed 'ssh_config_file', if
+# 'ssh_config_file' file does not exist
+ssh_config_init: #{default_suite_runner_configs['ssh_config_init']}
+
+# http://docs.aws.amazon.com/sdkforruby/api/index.html#Configuration
+# "The SDK searches the following locations for a region: ENV['AWS_REGION']"
+# 
+# Test runner sets ENV['AWS_REGION'] if following property is set
+#
+aws_region: #{default_suite_runner_configs['aws_region']}
+EOS
+    puts doc
+  end
+
   # suite_properties.each{  |a| a.keys.first }
 
   # **********
@@ -51,6 +102,14 @@ namespace :suite do
 
   end
 
+  # **********
+  desc "Syncrronize ec2 instance metadata to "
+  task "ec2-sync" do
+    aws_ssh_resolver="aws-ssh-resolver.rb"
+    sh "#{aws_ssh_resolver} aws --ssh-config-file #{suite_runner_configs['ssh_config_file']} --ssh-config-init #{suite_runner_configs['ssh_config_init']}"
+  end
+
+
   # **********
   all_suites = test_suites.suite_ids.map{ |id| "suite:" + id }
   task :suites, :gen_opts  do |t,args|
@@ -91,16 +150,36 @@ namespace :suite do
     # find the stack name for suite
     stack = test_suites.get_suite_stack_id( suite_id )
 
+    desc "#{suite_id} - syncrhornize #{suite_runner_configs['ssh_config_file']}"
+    task "#{suite_id}-sync", :gen_opts  do |t,args|
+      ssh_client_config_synchronize( suite_runner_configs )
+    end
+
+    # json
+    task "#{suite_id}-json", :gen_opts  do |t,args|
+      args.with_defaults( :gen_opts => "-m aws-must-templates" )
+      json_template="#{aws_must} gen #{stack}.yaml #{args.gen_opts}"  
+      sh "#{json_template}"
+    end
+
+
+    desc "#{suite_id} - create stack '#{stack}', wait to complete, sync #{suite_runner_configs['ssh_config_file']}"
+    task "#{suite_id}-stack-start", :gen_opts  do |t,args|
+      Rake::Task["suite:#{suite_id}-stack-create"].invoke(args.gen_opts)
+      Rake::Task["suite:#{suite_id}-stack-wait"].invoke()
+      Rake::Task["suite:#{suite_id}-sync"].invoke()
+    end
+
     # **********
     # Create stack for a suite
-    desc "Create stack #{stack} for suite #{suite_id}"
+    desc "#{suite_id} - create stack '#{stack}'"
     task "#{suite_id}-stack-create", :gen_opts  do |t,args|
       args.with_defaults( :gen_opts => "-m aws-must-templates" )
       json_template="#{aws_must} gen #{stack}.yaml #{args.gen_opts}"  
       sh "aws cloudformation create-stack --stack-name #{stack} --capabilities CAPABILITY_IAM  --template-body \"$(#{json_template})\"  --disable-rollback"
     end
 
-    desc "Create stack #{stack} for suite #{suite_id}"
+    desc "#{suite_id} - wait stack #{stack} #{END_STATES.join(', ')} "
     task "#{suite_id}-stack-wait" do
 
       while true
@@ -123,7 +202,7 @@ namespace :suite do
 
     # **********
     # delete stack for a suite
-    desc "Delete stack #{stack} for suite #{suite_id}"
+    desc "#{suite_id} - delete stack #{stack}"
     task "#{suite_id}-stack-delete" do
       sh "aws cloudformation delete-stack --stack-name #{stack}"
     end
@@ -134,25 +213,28 @@ namespace :suite do
     end
 
     # **********
-    desc "Show status for stack #{stack}"
+    desc "#{suite_id} - show status for stack #{stack}"
     task "#{suite_id}-stack-status" do
       sh "aws cloudformation describe-stacks --stack-name #{stack}"
     end
 
     # **********
     # Test common roles for suite
-    desc "Suite #{suite_id} - common roles"
+    desc "#{suite_id} - common roles"
     RSpec::Core::RakeTask.new( "#{suite_id}-common" ) do |t|
-      puts "------------------------------------------------------------------"
-      puts "suite=#{suite_id }"
 
-      # see spec/spec_helper.rb
-      ENV['TARGET_SUITE_ID'] = suite_id
+      set_rspec_task( t, suite_runner_configs, test_suites, suite_id )
+
+      # puts "------------------------------------------------------------------"
+      # puts "suite=#{suite_id }"
+
+      # # see spec/spec_helper.rb
+      # ENV['TARGET_SUITE_ID'] = suite_id
 
       # test all roles for the instance
-      t.rspec_opts = rspec_opts( suite_id )
-      t.fail_on_error = false       
-      t.ruby_opts= rspec_ruby_opts
+      # t.rspec_opts = rspec_opts( suite_id )
+      # t.fail_on_error = false       
+      # t.ruby_opts= rspec_ruby_opts
       # t.pattern = suite["roles"].map {  |r|  spec_pattern( r ) }.join(",")
       t.pattern = test_suites.suite_role_ids( suite_id ).map{ |r| spec_pattern( r ) }.join(",")
 
@@ -161,18 +243,18 @@ namespace :suite do
 
 
     # **********
-    # Run tasks for suite suite_id
+    # Run tasks for 'suite_id'
 
-    desc "Run all takss for suite '#{suite_id}' - {suite['desc']"
+    desc "#{suite_id} - #{test_suites.get_suite(suite_id)['desc']}"
     suite_tasks = 
       [ 
        "suite:report_dir", 
        [ "suite:#{suite_id}-stack-create", "gen_opts" ],
-        "suite:#{suite_id}-stack-wait", 
+       "suite:#{suite_id}-stack-wait", 
+       "suite:#{suite_id}-sync", 
       ] + 
       ( Rake::Task.task_defined?(  "suite:#{suite_id}-common"  ) ? [  "suite:#{suite_id}-common"  ] : [] )  + 
-      # ( suite["instances"] ? suite["instances"].each.map{ |a| "suite:#{suite_id}:" + a.keys.first } : []  ) + 
-      ( test_suites.suite_instance_ids( suite_id ).each.map{ |instance_id| "suite:#{suite_id}:" + instance_id }  ) + 
+      ( test_suites.suite_instance_names( suite_id ).each.map{ |instance_name| "suite:#{suite_id}:" + instance_name }  ) + 
       [ "suite:#{suite_id}-stack-delete" ] 
 
     task suite_id, :gen_opts do |ta,args|
@@ -185,6 +267,7 @@ namespace :suite do
             taskname = task.shift
             Rake::Task[taskname].invoke( *(task.select{ |arg_name| args[arg_name]}.map{ |arg_name| args[arg_name] }) )
           else
+            taskname = task
             Rake::Task[task].invoke( args )
           end
           failed_tasks << task unless ( $?.nil? || $?.success? )
@@ -192,7 +275,7 @@ namespace :suite do
           # sh "rake #{t}; true"
         rescue => e
           puts "#{e.class}: #{e.message}"
-          failed_tasks << task
+          failed_tasks << taskname
           # puts e.backtrace
           puts "continue with next task"
         end
@@ -213,37 +296,19 @@ namespace :suite do
     namespace suite_id do
 
       # suite["instances"].each do |instance_map|
-      test_suites.suite_instance_ids( suite_id ).each do |instance_id|
+      test_suites.suite_instance_names( suite_id ).each do |instance_name|
 
-        # instance_id = instance_map.keys.first
-        # instance = instance_map[instance_id]
+        # instance_name = instance_map.keys.first
+        # instance = instance_map[instance_name]
 
         # **********
-        desc "Test roles for instance '#{instance_id}' in suite '#{suite_id}'"
-        RSpec::Core::RakeTask.new( instance_id ) do |t|
-
-          puts "------------------------------------------------------------------"
-          puts "suite=#{suite_id }, instance=#{instance_id}"
-
-          # see spec/spec_helper.rb
-          ENV['TARGET_SUITE_ID'] = suite_id
-          ENV['TARGET_INSTANCE_ID'] = instance_id
-
-          t.rspec_opts = rspec_opts( suite_id, instance_id )
-          t.fail_on_error = false
-          t.ruby_opts= rspec_ruby_opts
-
-          # test all roles for the instance
-          pattern = test_suites.suite_instance_role_ids( suite_id, instance_id ).map{ |r| spec_pattern( r ) }.join(",")
-          raise <<-EOS if pattern.nil? ||  pattern.empty?
+        desc "#{suite_id} - test instance '#{instance_name}'"
+        RSpec::Core::RakeTask.new( instance_name ) do |t|
 
-              No tests defined for an instance in suite=#{suite_id }, instance=#{instance_id}
-
-          EOS
-          t.pattern = pattern
+          set_rspec_task( t, suite_runner_configs, test_suites, suite_id, instance_name )
 
         end
-      end # instance_ids
+      end # instance_names
     end # ns suite_id
 
   end # suite_properties.each
@@ -278,20 +343,57 @@ namespace :suite do
   end
 
   # to pass to rpsec
-  def rspec_opts( suite_id, instance_id=nil )
+  def rspec_opts( suite_id, instance_name=nil )
     # "--format documentation"
-    # "--format progress --format documentation --out generated-docs/suites/#{suite_id}#{ instance_id ? '-'+instance_id : ""}.txt"
-    "--format progress --format documentation --out #{suite_test_report_filepath( suite_id, instance_id )}"
+    # "--format progress --format documentation --out generated-docs/suites/#{suite_id}#{ instance_name ? '-'+instance_name : ""}.txt"
+    "--format progress --format documentation --out #{suite_test_report_filepath( suite_id, instance_name )}"
   end
 
-  def suite_test_report_filepath( suite_id, instance_id=nil )
-    "#{suite_test_report_dirpath()}/#{suite_id}#{ instance_id ? '-' + instance_id : ""}.txt"
+  def suite_test_report_filepath( suite_id, instance_name=nil )
+    "#{suite_test_report_dirpath()}/#{suite_id}#{ instance_name ? '-' + instance_name : ""}.txt"
   end
 
   def suite_test_report_dirpath
     "generated-docs/suites"
   end
 
+  # read ec2 instance metadata && update 'ssh_config_file'
+  def ssh_client_config_synchronize( suite_runner_configs )
+    aws_ssh_resolver="aws-ssh-resolver.rb"
+    sh "#{aws_ssh_resolver} aws --ssh-config-file #{suite_runner_configs['ssh_config_file']} --ssh-config-init #{suite_runner_configs['ssh_config_init']}"
+  end
+
+  def set_rspec_task( t, suite_runner_configs, test_suites, suite_id, instance_name=nil  )
+
+    puts "------------------------------------------------------------------"
+    puts "suite=#{suite_id } #{instance_name ? ' instance:  ' + instance_name : ''}"
+
+    # see spec/spec_helper.rb
+    ENV['TARGET_SUITE_ID'] = suite_id
+    if instance_name then
+      ENV['TARGET_INSTANCE_NAME'] = instance_name 
+    else
+      ENV.delete('TARGET_INSTANCE_NAME')
+    end
+    ENV['AWS_REGION'] = suite_runner_configs["aws_region"] if suite_runner_configs["aws_region"]
+
+    t.rspec_opts = rspec_opts( suite_id, instance_name )
+    t.fail_on_error = false
+    t.ruby_opts= rspec_ruby_opts
+
+    # test all roles for the instance
+    pattern = (instance_name ? 
+               test_suites.suite_instance_role_ids( suite_id, instance_name ).map{ |r| spec_pattern( r ) }.join(",") :
+               test_suites.suite_role_ids( suite_id ).map{ |r| spec_pattern( r ) }.join(",") )
+
+    raise <<-EOS if pattern.nil? ||  pattern.empty?
+
+              No tests defined for an instance in suite=#{suite_id } #{instance_name ? ' for instance ' + instance_name : ''}
+
+          EOS
+    t.pattern = pattern
+
+  end
 
   
 end # ns suite

data/lib/test-suites/test_suites.rb

@@ -53,7 +53,7 @@ module AwsMustTemplates
 
       # ------------------------------------------------------------------
       # return list of instanceid for a suite
-      def suite_instance_ids( suite_id ) 
+      def suite_instance_names( suite_id ) 
         suite = get_suite( suite_id )
         return nil unless suite
         return suite["instances"] ? suite["instances"].map{ |instance_hash| instance_hash.keys.first } : []
@@ -75,21 +75,21 @@ module AwsMustTemplates
         return suite_id
       end
 
-      # return array of role_ids for 'instance_id' in 'suite_id'
-      def suite_instance_role_ids( suite_id, instance_id )
-        roles = suite_instance_roles( suite_id, instance_id )
+      # return array of role_ids for 'instance_name' in 'suite_id'
+      def suite_instance_role_ids( suite_id, instance_name )
+        roles = suite_instance_roles( suite_id, instance_name )
         return roles unless roles
         return roles.map{ |r| r.is_a?( Hash ) ? r.keys.first : r }
       end
 
-      # return roles  for 'instance_id' in 'suite_id'
-      def suite_instance_roles( suite_id, instance_id )
-        instance = suite_instance( suite_id, instance_id )
+      # return roles  for 'instance_name' in 'suite_id'
+      def suite_instance_roles( suite_id, instance_name )
+        instance = suite_instance( suite_id, instance_name )
         return nil unless instance
-        return [] unless instance[instance_id]
-        return [] unless instance[instance_id]["roles"]
+        return [] unless instance[instance_name]
+        return [] unless instance[instance_name]["roles"]
         # roles may be a hash or a string
-        return instance[instance_id]["roles"]
+        return instance[instance_name]["roles"]
       end
 
 
@@ -98,11 +98,11 @@ module AwsMustTemplates
 
       private
 
-      # instance sub document for `instance_id` in `suite_id`
-      def suite_instance( suite_id, instance_id )
+      # instance sub document for `instance_name` in `suite_id`
+      def suite_instance( suite_id, instance_name )
         suite = get_suite( suite_id )
         return nil unless suite && suite["instances"]
-        return suite["instances"].select { |i| i.keys.first == instance_id }.first
+        return suite["instances"].select { |i| i.keys.first == instance_name }.first
       end