sonic-screwdriver 2.0.0 → 2.2.2

data/docs/_reference/sonic-list.md

@@ -11,16 +11,16 @@ reference: true
 
 Lists ec2 instances.
 
-List ec2 servers. A filter must be provided.  The filter can be a mix of instance ids and ec2 tags. sonic list will auto-detect the what type of filter it is filter appropriately.  The filter for listing is optional.
+A filter must be provided.  The filter can be a mix of instance ids and ec2 tags. sonic list will auto-detect the what type of filter it is.  The filter is optional.
 
-Examples:
+## Examples
 
     $ sonic list
     $ sonic list hi-web-prod
     $ sonic list hi-web-prod,hi-clock-prod
     $ sonic list i-09482b1a6e330fbf7
 
-Example Output:
+## Example Output
 
     $ sonic list --header i-09482b1a6e330fbf7
     Instance Id Public IP Private IP  Type

data/docs/bin/web

@@ -1,4 +1,4 @@
 #!/bin/bash -ex
 
 bundle exec jekyll clean
-exec bundle exec jekyll serve
+exec bundle exec jekyll serve --host 0.0.0.0 "$@"

data/docs/quick-start.md

@@ -6,35 +6,31 @@ In a hurry? No sweat! Here's a quick overview of how to use sonic.
 
 ### Install
 
-```sh
-gem install sonic-screwdriver
-```
+    gem install sonic-screwdriver
 
 ### Usage
 
-```sh
-# ssh into an instance
-sonic ssh i-0f7f833131a51ce35
-sonic ssh hi-web # ec2 tag
-sonic ssh hi-web --cluster staging # ecs service name
-sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474 --cluster staging # ECS container id
-sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170 --cluster staging # ECS task id
+    # ssh into an instance
+    sonic ssh i-0f7f833131a51ce35
+    sonic ssh demo-web # ec2 tag
+    sonic ssh demo-web --cluster staging # ecs service name
+    sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474 --cluster staging # ECS container id
+    sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170 --cluster staging # ECS task id
 
-# docker exec to a running ECS docker container
-sonic ecs exec hi-web
+    # docker exec to a running ECS docker container
+    sonic ecs exec demo-web
 
-# docker run with same environment as the ECS docker running containers
-sonic ecs sh hi-web
+    # docker run with same environment as the ECS docker running containers
+    sonic ecs sh demo-web
 
-# run command on 1 instance
-sonic execute i-0f7f833131a51ce35 uptime
+    # run command on 1 instance
+    sonic execute --instance-ids i-0f7f833131a51ce35 uptime
 
-# run command on all instances tagged with hi-web and worker
-sonic execute hi-web,hi-worker uptime
+    # run command on all instances tagged with demo-web and worker
+    sonic execute --tags Name=demo-web,demo-worker uptime
 
-# list ec2 instances
-sonic list hi-web
-```
+    # list ec2 instances
+    sonic list demo-web
 
 Congratulations! You now know the basic sonic screwdriver commands now.
 

data/lib/sonic.rb

@@ -1,6 +1,6 @@
 $:.unshift(File.expand_path("../", __FILE__))
+require "rainbow/ext/string"
 require "sonic/version"
-require "colorize"
 
 module Sonic
   autoload :Core, 'sonic/core'

data/lib/sonic/checks.rb

@@ -3,7 +3,7 @@ module Sonic
     def check_cluster_exists!
       cluster = ecs.describe_clusters(clusters: [@cluster]).clusters.first
       unless cluster
-        UI.error "The #{@cluster.green} cluster does not exist.  Are you sure you specified the right cluster?"
+        UI.error "The #{@cluster.color(:green)} cluster does not exist.  Are you sure you specified the right cluster?"
         exit 1
       end
     end
@@ -20,7 +20,7 @@ module Sonic
 
       service = resp.services.first
       unless service
-        UI.error "The #{@service.green} service does not exist in #{@cluster.green} cluster.  Are you sure you specified the right service and cluster?"
+        UI.error "The #{@service.color(:green)} service does not exist in #{@cluster.color(:green)} cluster.  Are you sure you specified the right service and cluster?"
         exit 1
       end
     end

data/lib/sonic/cli.rb

@@ -27,8 +27,10 @@ module Sonic
     long_desc Help.text("execute")
     option :zero_warn, type: :boolean, default: true, desc: "Warns user when no instances found"
     # filter - Filter ec2 instances by tag name or instance_ids separated by commas
-    def execute(filter, *command)
-      Execute.new(command, options.merge(filter: filter)).execute
+    option :instance_ids, desc: %Q|Instance ids to execute command on. Format: --instance-ids "i-111,i-222"|
+    option :tags, desc: %Q|Tags used to determine what instances to execute command on. Format: --tags "Key1=v1,v2;Key2=v3"|
+    def execute(*command)
+      Execute.new(command, options).execute
     end
 
     desc "list [FILTER]", "Lists ec2 instances."

data/lib/sonic/default/settings.yml

@@ -1,5 +1,8 @@
 bastion:
   host_key_check: false
+  # host:
+
+ssh:
   user: ec2-user
 
 ecs_service_cluster_map:

data/lib/sonic/docker.rb

@@ -30,7 +30,7 @@ module Sonic
 
     # command is an Array
     def execute(*command)
-      UI.say "=> #{command.join(' ')}".colorize(:green)
+      UI.say "=> #{command.join(' ')}".color(:green)
       success = system(*command)
       unless success
         UI.error(command.join(' '))
@@ -59,7 +59,7 @@ module Sonic
 
       ssh = ["ssh", ssh_options, "-At", host, "uptime", "2>&1"]
       command = ssh.join(' ')
-      puts "=> #{command}".colorize(:green)
+      puts "=> #{command}".color(:green)
       output = `#{command}`
       if output.include?("Permission denied")
         puts output

data/lib/sonic/execute.rb

@@ -1,4 +1,3 @@
-require 'colorize'
 require 'yaml'
 require 'active_support/core_ext/hash'
 
@@ -9,7 +8,8 @@ module Sonic
     def initialize(command, options)
       @command = command
       @options = options
-      @filter = @options[:filter].split(',').map{|s| s.strip}
+      @tags = @options[:tags]
+      @instance_ids = @options[:instance_ids]
     end
 
     # aws ssm send-command \
@@ -19,6 +19,7 @@ module Sonic
     #   --parameters '{"commands":["#!/usr/bin/python","print \"Hello world from python\""]}' \
     #   --query "Command.CommandId"
     def execute
+      check_filter_options!
       ssm_options = build_ssm_options
       if @options[:noop]
         UI.noop = true
@@ -34,6 +35,7 @@ module Sonic
         puts
         begin
           resp = send_command(ssm_options)
+
           command_id = resp.command.command_id
           success = true
         rescue Aws::SSM::Errors::InvalidInstanceId => e
@@ -50,9 +52,31 @@ module Sonic
       display_ssm_commands(command_id, ssm_options)
       puts
       return if @options[:noop]
-      wait(command_id)
-      display_ssm_output(command_id, ssm_options)
+      status = wait(command_id)
+      instances_found = display_ssm_output(command_id)
       display_console_url(command_id)
+
+      if status == "Success"
+        puts "Command successful: #{status}".color(:green)
+        exit_status(0)
+      else
+        puts "Command unsuccessful: #{status}".color(:red)
+        exit_status(1)
+      end
+    end
+
+    def exit_status(code)
+      exit(code) unless cli?
+
+      if code == 0
+        true
+      else
+        raise "Error running command"
+      end
+    end
+
+    def cli?
+      $0.include?('sonic')
     end
 
     def wait(command_id)
@@ -69,15 +93,21 @@ module Sonic
       end
       puts "\nCommand finished."
       puts
+      status
     end
 
-    def display_ssm_output(command_id, ssm_options)
-      instance_ids = ssm_options[:instance_ids]
-      return unless instance_ids && instance_ids.size > 0
+    def display_ssm_output(command_id)
+      resp = ssm.list_command_invocations(command_id: command_id)
+      command_invocations = resp.command_invocations
+      command_invocation = command_invocations.first
+      unless command_invocation
+        puts "WARN: No instances found that matches the --tags or --instance-ids option".color(:yellow)
+        return false # instances_found
+      end
+      instance_id = command_invocation.instance_id
 
-      instance_id = instance_ids.first
-      if ssm_options[:instance_ids].size > 1
-        puts "Multiple instance targets. Only displaying output for #{instance_id}."
+      if command_invocations.size > 1
+        puts "Multiple instance targets. Total targets: #{command_invocations.size}. Only displaying output for #{instance_id}."
       else
         puts "Displaying output for #{instance_id}."
       end
@@ -89,6 +119,7 @@ module Sonic
       ssm_output(resp, "output")
       ssm_output(resp, "error")
       puts
+      true # instances_found
     end
 
     def display_console_url(command_id)
@@ -97,16 +128,15 @@ module Sonic
       puts "To see the more output details visit:"
       puts "  #{console_url}"
       puts
-      copy_paste_clipboard(console_url)
-      UI.say "Pro tip: the console url is already in your copy/paste clipboard."
+      copy_paste_clipboard(console_url, "Pro tip: the console url is already in your copy/paste clipboard.")
     end
 
     def colorized_status(status)
       case status
       when "Success"
-        status.colorize(:green)
+        status.color(:green)
       when "Failed"
-        status.colorize(:red)
+        status.color(:red)
       else
         status
       end
@@ -121,7 +151,7 @@ module Sonic
       return if content.empty?
 
       puts "Command standard #{type}:"
-      # "https://s3.amazonaws.com/lr-infrastructure-prod/ssm/commands/sonic/0a4f4bef-8f63-4235-8b30-ae296477261a/i-0b2e6e187a3f9ada9/awsrunPowerShellScript/0.awsrunPowerShellScript/stderr">
+      # "https://s3.amazonaws.com/infra-prod/ssm/commands/sonic/0a4f4bef-8f63-4235-8b30-ae296477261a/i-0b2e6e187a3f9ada9/awsrunPowerShellScript/0.awsrunPowerShellScript/stderr">
       if content.include?("--output truncated--") && !resp[s3_key].empty?
         s3_url = resp[s3_key]
         info = s3_url.sub('https://s3.amazonaws.com/', '').split('/')
@@ -147,7 +177,6 @@ module Sonic
 
       begin
         resp = ssm.send_command(options)
-        # puts "NOOP FOR NOW"
       rescue Aws::SSM::Errors::UnsupportedPlatformType
         retries += 1
         # toggle AWS-RunShellScript / AWS-RunPowerShellScript
@@ -166,12 +195,18 @@ module Sonic
     end
 
     def build_ssm_options
-      criteria = transform_filter(@filter)
+      criteria = transform_filter_option
       command = build_command(@command)
       options = criteria.merge(
         document_name: "AWS-RunShellScript", # default
         comment: "sonic #{ARGV.join(' ')}"[0..99], # comment has a max of 100 chars
-        parameters: { "commands" => command }
+        parameters: { "commands" => command },
+        # Default CloudWatchLog settings. Can be overwritten with settings.yml send_command
+        # IMPORTANT: make sure the EC2 instance the command runs on has access to write to CloudWatch Logs.
+        cloud_watch_output_config: {
+          # cloud_watch_log_group_name: "ssm", # Defaults to /aws/ssm/AWS-RunShellScript (aws/ssm/SystemsManagerDocumentName https://amzn.to/38TKVse)
+          cloud_watch_output_enabled: true,
+        },
       )
       settings_options = settings["send_command"] || {}
       options.merge(settings_options.deep_symbolize_keys)
@@ -181,6 +216,12 @@ module Sonic
       @settings ||= Setting.new.data
     end
 
+    def check_filter_options!
+      return if @tags || @instance_ids
+      puts "ERROR: Please provide --tags or --instance-ids option".color(:red)
+      exit 1
+    end
+
     #
     # Public: Transform the filter to the ssm send_command equivalent options
     #
@@ -188,41 +229,31 @@ module Sonic
     #
     # Examples
     #
-    #   transform_filter(["hi-web-prod", "hi-worker-prod", "i-006a097bb10643e20"])
+    #   transform_filter_option
     #   # => {
     #      instance_ids: ["i-006a097bb10643e20"],
     #      targets: [{key: "Name", values: "hi-web-prod,hi-worker-prod"}]
     #     }
     #
     # Returns the duplicated String.
-    def transform_filter(filter)
-      valid = validate_filter(filter)
-      unless valid
-        UI.error("The filter you provided '#{filter.join(',')}' is not valid.")
-        UI.say("The filter must either be all instance ids or just a list of tag names.")
-        exit 1
-      end
-
-      if filter.detect { |i| instance_id?(i) }
-        instance_ids = filter
-        {instance_ids: instance_ids}
-      else
-        tags = filter
-        targets = [{
-          key: "tag:#{tag_name}",
-          values: tags
-        }]
+    def transform_filter_option
+      if @tags
+        list = @tags.split(';')
+        targets = list.inject([]) do |final,item|
+          tag_name,value_list = item.split('=')
+          values = value_list.split(',').map(&:strip)
+          # structure expected by ssm send_command
+          option = {
+            key: "tag:#{tag_name}",
+            values: values
+          }
+          final << option
+          final
+        end
         {targets: targets}
-      end
-    end
-
-    # Either all instance ids are no instance ids is a valid filter
-    def validate_filter(filter)
-      if filter.detect { |i| instance_id?(i) }
-        instance_ids = filter.select { |i| instance_id?(i) }
-        instance_ids.size == filter.size
-      else
-        true
+      else # @instance_ids
+        instance_ids = @instance_ids.split(',')
+        {instance_ids: instance_ids}
       end
     end
 
@@ -239,7 +270,7 @@ module Sonic
         # The script is being feed inline so just join the command together into one script.
         # Still keep in an array form because that's how ssn.send_command works with AWS-RunShellScript
         # usually reads the command.
-        [command.join(" ")]
+        command.is_a?(Array) ? command : [command]
       end
     end
 
@@ -254,8 +285,7 @@ You can use the following command to check registered instances to SSM.
 #{ssm_describe_command}
       EOS
       UI.warn(message)
-      copy_paste_clipboard(ssm_describe_command)
-      UI.say "Pro tip: ssm describe-instance-information already in your copy/paste clipboard."
+      copy_paste_clipboard(ssm_describe_command, "Pro tip: ssm describe-instance-information already in your copy/paste clipboard.")
     end
 
     def file_path?(command)
@@ -312,9 +342,10 @@ EOL
       end
     end
 
-    def copy_paste_clipboard(command)
+    def copy_paste_clipboard(command, text)
       return unless RUBY_PLATFORM =~ /darwin/
       system("echo '#{command}' | pbcopy")
+      UI.say text
     end
   end
 end

data/lib/sonic/help/execute.md

@@ -1,11 +1,12 @@
-Run as a command across a list of servers. A filter must be provided.  The filter can be a mix of instance ids and ec2 tags. This command can also take a path to a file. To specify a path to a file use file:// at the beginning of your file.
+* A filter must be provided.  The filter can be a mix of instance ids and ec2 tags.
+* The command can be provided inline or as a file. To a file use `file://` at the beginning of your file.
 
 ## Examples Summary
 
-    $ sonic execute hi-web-prod uptime
-    $ sonic execute hi-web-prod,hi-worker-prod,hi-clock-prod uptime
-    $ sonic execute i-030033c20c54bf149,i-030033c20c54bf150 uname -a
-    $ sonic execute i-030033c20c54bf149 file://hello.sh
+    sonic execute --tags Name=demo-web uptime
+    sonic execute --tags Name=demo-web,demo-worker uptime # multiple values
+    sonic execute --instance-ids i-030033c20c54bf149,i-030033c20c54bf150 uname -a
+    sonic execute --instance-ids i-030033c20c54bf149 file://hello.sh # script from file
 
 You cannot mix instance ids and tag names in the filter.
 
@@ -13,39 +14,37 @@ You cannot mix instance ids and tag names in the filter.
 
 Here's a command example output in detailed:
 
-```sh
-$ sonic execute i-0bf51a000ab4e73a8 uptime
-Sending command to SSM with options:
----
-instance_ids:
-- i-0bf51a000ab4e73a8
-document_name: AWS-RunShellScript
-comment: sonic execute i-0bf51a000ab4e73a8 uptime
-parameters:
-  commands:
-  - uptime
-output_s3_region: us-east-1
-output_s3_bucket_name: [reacted]
-output_s3_key_prefix: ssm/commands/sonic
-
-Command sent to AWS SSM. To check the details of the command:
-  aws ssm list-commands --command-id 0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2
-  aws ssm get-command-invocation --command-id 0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2 --instance-id i-0bf51a000ab4e73a8
-
-Waiting for ssm command to finish.....
-Command finished.
-
-Displaying output for i-0bf51a000ab4e73a8.
-Command status: Success
-Command standard output:
- 01:08:10 up 8 days,  6:41,  0 users,  load average: 0.00, 0.00, 0.00
-
-To see the more output details visit:
-  https://us-west-2.console.aws.amazon.com/systems-manager/run-command/0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2
-
-Pro tip: the console url is already in your copy/paste clipboard.
-$
-```
+    $ sonic execute --instance-ids i-0bf51a000ab4e73a8 uptime
+    Sending command to SSM with options:
+    ---
+    instance_ids:
+    - i-0bf51a000ab4e73a8
+    document_name: AWS-RunShellScript
+    comment: sonic execute --instance-ids i-0bf51a000ab4e73a8 uptime
+    parameters:
+      commands:
+      - uptime
+    output_s3_region: us-east-1
+    output_s3_bucket_name: [reacted]
+    output_s3_key_prefix: ssm/commands/sonic
+
+    Command sent to AWS SSM. To check the details of the command:
+      aws ssm list-commands --command-id 0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2
+      aws ssm get-command-invocation --command-id 0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2 --instance-id i-0bf51a000ab4e73a8
+
+    Waiting for ssm command to finish.....
+    Command finished.
+
+    Displaying output for i-0bf51a000ab4e73a8.
+    Command status: Success
+    Command standard output:
+     01:08:10 up 8 days,  6:41,  0 users,  load average: 0.00, 0.00, 0.00
+
+    To see the more output details visit:
+      https://us-west-2.console.aws.amazon.com/systems-manager/run-command/0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2
+
+    Pro tip: the console url is already in your copy/paste clipboard.
+    $
 
 Notice the conveniences of `sonic execute`, it: