sonic-screwdriver 1.4.0 → 2.2.1

data/docs/_reference/sonic-list.md

@@ -0,0 +1,40 @@
+---
+title: sonic list
+reference: true
+---
+
+## Usage
+
+    sonic list [FILTER]
+
+## Description
+
+Lists ec2 instances.
+
+A filter must be provided.  The filter can be a mix of instance ids and ec2 tags. sonic list will auto-detect the what type of filter it is.  The filter is optional.
+
+## Examples
+
+    $ sonic list
+    $ sonic list hi-web-prod
+    $ sonic list hi-web-prod,hi-clock-prod
+    $ sonic list i-09482b1a6e330fbf7
+
+## Example Output
+
+    $ sonic list --header i-09482b1a6e330fbf7
+    Instance Id Public IP Private IP  Type
+    i-09482b1a6e330fbf7 54.202.152.168  172.31.21.108 t2.small
+    $
+
+You cannot mix instance ids and tag names in the filter.
+
+
+## Options
+
+```
+[--header], [--no-header]    # Displays header
+[--verbose], [--no-verbose]  
+[--noop], [--no-noop]        
+```
+

data/docs/_reference/sonic-ssh.md

@@ -0,0 +1,86 @@
+---
+title: sonic ssh
+reference: true
+---
+
+## Usage
+
+    sonic ssh [IDENTIFER]
+
+## Description
+
+Ssh into a instance using identifier. identifer can be several things: instance id, ec2 tag, ECS service name, etc.
+
+* EC2 instance id. Example: i-067c5e3f026c1e801
+* EC2 tag value. Any tag value is search, the tag key does not matter only the tag value matters. Example: hi-web
+* ECS service. Example: my-ecs-service
+* ECS container instance id. Example: 7fbc8c75-4675-4d39-a5a4-0395ff8cd474
+* ECS task id. Example: 1ed12abd-645c-4a05-9acf-739b9d790170
+
+When using ecs identifiers the `--cluster` option is required or can be set in ~/.sonic/settings.yml.
+
+Examples:
+
+    $ sonic ssh i-067c5e3f026c1e801
+    $ sonic ssh hi-web
+    $ sonic ssh --cluster my-cluster my-ecs-service
+    $ sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474
+    $ sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170
+
+Sonic ssh builds up the ssh command and shells out to it. For example, the following commands:
+
+    sonic ssh i-027363802c6ff314f
+
+Translates to:
+
+    ssh ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com
+
+You can also tack on any command to be run at the end of the command. Example:
+
+    $ sonic ssh i-027363802c6ff314f uptime
+    => ssh ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com uptime
+ 15:57:02 up 18:21,  0 users,  load average: 0.00, 0.01, 0.00
+
+## Specifying pem keys
+
+The recommended way to specify custom private keys is to use ssh-agent as covered here: https://blog.boltops.com/2017/09/21/3-ssh-tips-ssh-agent-tunnel-and-escaping-from-the-dead
+
+But you can also specify the pem key to use with the -i option.  Example:
+
+    $ sonic ssh -i ~/.ssh/id_rsa-custom ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com
+
+## Retry option
+
+For newly launched instances, the instance's ssh access might not be quite ready. Typically, you must press up enter repeatedly until the instance is ready.  Sonic ssh has a retry option that automates this. Example:
+
+    $ sonic ssh -r i-027363802c6ff314f
+
+Bastion Host Support
+
+Sonic ssh also supports a bastion host.
+
+    $ sonic ssh --bastion bastion.domain.com i-027363802c6ff314f
+    $ sonic ssh --bastion user@bastion.domain.com i-027363802c6ff314f
+
+Here's what the output looks like:
+
+    $ sonic ssh --bastion 34.211.223.3 i-0f7f833131a51ce35 uptime
+    => ssh -At ec2-user@34.211.223.3 ssh ec2-user@10.10.110.135 uptime
+     17:57:59 up 37 min,  0 users,  load average: 0.00, 0.02, 0.00
+    Connection to 34.211.223.3 closed.
+    $
+
+You can also set the bastion host and other options with a [settings file](http://sonic-screwdriver.cloud/docs/settings/).
+
+
+## Options
+
+```
+-i, [--keys=KEYS]                # comma separated list of ssh private key paths
+-r, [--retry], [--no-retry]      # keep retrying the server login until successful. Useful when on newly launched instances.
+    [--bastion=BASTION]          # Bastion jump host to use.  Defaults to no bastion server.
+    [--cluster=CLUSTER]          # ECS Cluster to use.  Default cluster is default
+    [--verbose], [--no-verbose]  
+    [--noop], [--no-noop]        
+```
+

data/docs/_reference/sonic-version.md

@@ -0,0 +1,21 @@
+---
+title: sonic version
+reference: true
+---
+
+## Usage
+
+    sonic version
+
+## Description
+
+prints version
+
+
+## Options
+
+```
+[--verbose], [--no-verbose]  
+[--noop], [--no-noop]        
+```
+

data/docs/bin/web

@@ -1,4 +1,4 @@
 #!/bin/bash -ex
 
 bundle exec jekyll clean
-exec bundle exec jekyll serve
+exec bundle exec jekyll serve --host 0.0.0.0 "$@"

data/docs/quick-start.md

@@ -6,36 +6,31 @@ In a hurry? No sweat! Here's a quick overview of how to use sonic.
 
 ### Install
 
-```sh
-gem install sonic-screwdriver
-```
+    gem install sonic-screwdriver
 
 ### Usage
 
-```sh
-# ssh into an instance
-sonic ssh i-0f7f833131a51ce35
-sonic ssh hi-web-stag # ec2 tag
-sonic ssh hi-web-stag --cluster stag # ecs service name
-sonic ssh hi-web-stag --cluster stag  # ecs service name
-sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474 --cluster stag  # ECS container id
-sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170 --cluster stag  # ECS task id
+    # ssh into an instance
+    sonic ssh i-0f7f833131a51ce35
+    sonic ssh demo-web # ec2 tag
+    sonic ssh demo-web --cluster staging # ecs service name
+    sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474 --cluster staging # ECS container id
+    sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170 --cluster staging # ECS task id
 
-# docker exec to a running ECS docker container
-sonic ecs-exec hi-web-stag
+    # docker exec to a running ECS docker container
+    sonic ecs exec demo-web
 
-# docker run with same environment as the ECS docker running containers
-sonic ecs-run hi-web-stag
+    # docker run with same environment as the ECS docker running containers
+    sonic ecs sh demo-web
 
-# run command on 1 instance
-sonic execute i-0f7f833131a51ce35 uptime
+    # run command on 1 instance
+    sonic execute --instance-ids i-0f7f833131a51ce35 uptime
 
-# run command on all instances tagged with hi-web-stag and worker
-sonic execute hi-web-stag,hi-worker-stag uptime
+    # run command on all instances tagged with demo-web and worker
+    sonic execute --tags Name=demo-web,demo-worker uptime
 
-# list ec2 instances
-sonic list hi-web-stag
-```
+    # list ec2 instances
+    sonic list demo-web
 
 Congratulations! You now know the basic sonic screwdriver commands now.
 

data/docs/reference.md

@@ -0,0 +1,12 @@
+---
+title: CLI Reference
+---
+{% include reference.md %}
+
+* [sonic completion]({% link _reference/sonic-completion.md %})
+* [sonic completion_script]({% link _reference/sonic-completion_script.md %})
+* [sonic ecs]({% link _reference/sonic-ecs.md %})
+* [sonic execute]({% link _reference/sonic-execute.md %})
+* [sonic list]({% link _reference/sonic-list.md %})
+* [sonic ssh]({% link _reference/sonic-ssh.md %})
+* [sonic version]({% link _reference/sonic-version.md %})

data/{bin → exe}/sonic

@@ -7,8 +7,8 @@ Signal.trap("INT") {
   exit
 }
 
-$:.unshift(File.expand_path('../../lib', __FILE__))
-require 'sonic'
-require 'sonic/cli'
+$:.unshift(File.expand_path("../../lib", __FILE__))
+require "sonic"
+require "sonic/cli"
 
 Sonic::CLI.start(ARGV)

data/lib/bash_scripts/docker-exec.sh

@@ -12,4 +12,5 @@ else
   COMMAND=bash
 fi
 
+set -x
 exec docker exec -ti $CONTAINER_ID $COMMAND

data/lib/bash_scripts/docker-run.sh

@@ -12,4 +12,11 @@ CONTAINER_IMAGE=$(cat /tmp/sonic/docker-image.txt)
 cp /tmp/sonic/env-file.txt ~/
 rm -rf /tmp/sonic
 
-exec docker run -ti --env-file ~/env-file.txt $CONTAINER_IMAGE "$@"
+if [ $# -eq 0 ]; then
+  COMMAND=bash
+else
+  COMMAND=$@
+fi
+
+set -x
+exec docker run -ti --env-file ~/env-file.txt "$CONTAINER_IMAGE" $COMMAND

data/lib/sonic.rb

@@ -1,16 +1,24 @@
 $:.unshift(File.expand_path("../", __FILE__))
+require "rainbow/ext/string"
 require "sonic/version"
-require "colorize"
 
 module Sonic
-  autoload :AwsServices, 'sonic/aws_services'
+  autoload :Core, 'sonic/core'
+  autoload :Help, 'sonic/help'
+  autoload :AwsService, 'sonic/aws_service'
   autoload :CLI, 'sonic/cli'
+  autoload :BaseCommand, 'sonic/base_command'
   autoload :Command, 'sonic/command'
   autoload :Docker, 'sonic/docker'
   autoload :Execute, 'sonic/execute'
   autoload :List, 'sonic/list'
-  autoload :Settings, 'sonic/settings'
+  autoload :Setting, 'sonic/setting'
   autoload :Ssh, 'sonic/ssh'
   autoload :UI, 'sonic/ui'
   autoload :Checks, 'sonic/checks'
+  autoload :Completion, "sonic/completion"
+  autoload :Completer, "sonic/completer"
+  autoload :Ecs, "sonic/ecs"
+
+  extend Core
 end

data/lib/sonic/{aws_services.rb → aws_service.rb}

@@ -1,9 +1,10 @@
 require "aws-sdk-ec2"
 require "aws-sdk-ecs"
 require "aws-sdk-ssm"
+require "aws-sdk-s3"
 
 module Sonic
-  module AwsServices
+  module AwsService
     def ecs
       @ecs ||= Aws::ECS::Client.new
     end
@@ -19,5 +20,9 @@ module Sonic
     def ssm
       @ssm ||= Aws::SSM::Client.new
     end
+
+    def s3
+      @s3 ||= Aws::S3::Client.new
+    end
   end
 end

data/lib/sonic/base_command.rb

@@ -0,0 +1,82 @@
+require "thor"
+
+# Override thor's long_desc identation behavior
+# https://github.com/erikhuda/thor/issues/398
+class Thor
+  module Shell
+    class Basic
+      def print_wrapped(message, options = {})
+        message = "\n#{message}" unless message[0] == "\n"
+        stdout.puts message
+      end
+    end
+  end
+end
+
+module Sonic
+  class BaseCommand < Thor
+    class << self
+      def dispatch(m, args, options, config)
+        # Allow calling for help via:
+        #   sonic command help
+        #   sonic command -h
+        #   sonic command --help
+        #   sonic command -D
+        #
+        # as well thor's normal way:
+        #
+        #   sonic help command
+        help_flags = Thor::HELP_MAPPINGS + ["help"]
+        if args.length > 1 && !(args & help_flags).empty?
+          args -= help_flags
+          args.insert(-2, "help")
+        end
+
+        #   sonic version
+        #   sonic --version
+        #   sonic -v
+        version_flags = ["--version", "-v"]
+        if args.length == 1 && !(args & version_flags).empty?
+          args = ["version"]
+        end
+
+        super
+      end
+
+      # Override command_help to include the description at the top of the
+      # long_description.
+      def command_help(shell, command_name)
+        meth = normalize_command_name(command_name)
+        command = all_commands[meth]
+        alter_command_description(command)
+        super
+      end
+
+      def alter_command_description(command)
+        return unless command
+
+        # Add description to beginning of long_description
+        long_desc = if command.long_description
+            "#{command.description}\n\n#{command.long_description}"
+          else
+            command.description
+          end
+
+        # add reference url to end of the long_description
+        unless website.empty?
+          full_command = [command.ancestor_name, command.name].compact.join('-')
+          url = "#{website}/reference/sonic-#{full_command}"
+          long_desc += "\n\nHelp also available at: #{url}"
+        end
+
+        command.long_description = long_desc
+      end
+      private :alter_command_description
+
+      # meant to be overriden
+      def website
+        "http://sonic-screwdriver.cloud"
+      end
+    end
+  end
+end

data/lib/sonic/checks.rb

@@ -3,7 +3,7 @@ module Sonic
     def check_cluster_exists!
       cluster = ecs.describe_clusters(clusters: [@cluster]).clusters.first
       unless cluster
-        UI.error "The #{@cluster.green} cluster does not exist.  Are you sure you specified the right cluster?"
+        UI.error "The #{@cluster.color(:green)} cluster does not exist.  Are you sure you specified the right cluster?"
         exit 1
       end
     end
@@ -20,7 +20,7 @@ module Sonic
 
       service = resp.services.first
       unless service
-        UI.error "The #{@service.green} service does not exist in #{@cluster.green} cluster.  Are you sure you specified the right service and cluster?"
+        UI.error "The #{@service.color(:green)} service does not exist in #{@cluster.color(:green)} cluster.  Are you sure you specified the right service and cluster?"
         exit 1
       end
     end

data/lib/sonic/cli.rb

@@ -1,49 +1,61 @@
 require 'thor'
-require 'sonic/cli/help'
 
 module Sonic
-  class CLI < Command
+  class CLI < BaseCommand
+    desc "ecs SUBCOMMAND", "ecs subcommands"
+    long_desc Help.text(:ecs)
+    subcommand "ecs", Ecs
+
+    # desc "command SUBCOMMAND", "command subcommands"
+    # long_desc Help.text(:command)
+    # subcommand "command", Command
+
     class_option :verbose, type: :boolean
     class_option :noop, type: :boolean
-    class_option :cluster, desc: "ECS Cluster to use.  Default cluster is default"
-    class_option :bastion, desc: "Bastion jump host to use.  Defaults to no bastion server."
-    class_option :project_root, desc: "Project root. Useful for testing.", hide: true
-
-    desc "ssh [IDENTIFER]", "ssh into a instance using identifier. identifer can be several things: instance id, ec2 tag, ECS service name, etc"
-    long_desc Help.ssh
-    method_option :keys, :aliases => '-i', :desc => "comma separated list of ssh private key paths"
-    method_option :retry, :aliases => '-r', :type => :boolean, :desc => "keep retrying the server login until successful. Useful when on newly launched instances."
+
+    desc "ssh [IDENTIFER]", "Ssh into a instance using identifier. identifer can be several things: instance id, ec2 tag, ECS service name, etc."
+    long_desc Help.text(:ssh)
+    option :keys, :aliases => '-i', :desc => "comma separated list of ssh private key paths"
+    option :retry, :aliases => '-r', :type => :boolean, :desc => "keep retrying the server login until successful. Useful when on newly launched instances."
+    option :bastion, desc: "Bastion jump host to use.  Defaults to no bastion server."
+    option :cluster, desc: "ECS Cluster to use.  Default cluster is default"
     def ssh(identifier, *command)
       Ssh.new(identifier, options.merge(command: command)).run
     end
 
-    desc "ecs-exec [ECS_SERVICE]", "docker exec into running docker container associated with the service on a container instance"
-    long_desc Help.ecs_exec
-    def ecs_exec(service, *command)
-      Docker.new(service, options.merge(command: command)).exec
-    end
-
-    # Cannot name the command run because that is a reserved Thor keyword :(
-    desc "ecs-run [ECS_SERVICE]", "docker run with the service on a container instance"
-    long_desc Help.ecs_run
-    def ecs_run(service, *command)
-      Docker.new(service, options.merge(command: command)).run
-    end
-
-    desc "execute [FILTER] [COMMAND]", "runs command across fleet of servers via AWS Run Command"
-    long_desc Help.execute
+    desc "execute [FILTER] [COMMAND]", "Runs command across fleet of servers via AWS Run Command."
+    long_desc Help.text("execute")
     option :zero_warn, type: :boolean, default: true, desc: "Warns user when no instances found"
     # filter - Filter ec2 instances by tag name or instance_ids separated by commas
-    def execute(filter, *command)
-      Execute.new(command, options.merge(filter: filter)).execute
+    option :instance_ids, desc: %Q|Instance ids to execute command on. Format: --instance-ids "i-111,i-222"|
+    option :tags, desc: %Q|Tags used to determine what instances to execute command on. Format: --tags "Key1=v1,v2;Key2=v3"|
+    def execute(*command)
+      Execute.new(command, options).execute
     end
 
-    desc "list [FILTER]", "lists ec2 instances"
-    long_desc Help.list
+    desc "list [FILTER]", "Lists ec2 instances."
+    long_desc Help.text(:list)
     option :header, type: :boolean, desc: "Displays header"
     # filter - Filter ec2 instances by tag name or instance_ids separated by commas
     def list(filter)
       List.new(options.merge(filter: filter)).run
     end
+
+    desc "completion *PARAMS", "Prints words for auto-completion."
+    long_desc Help.text("completion")
+    def completion(*params)
+      Completer.new(CLI, *params).run
+    end
+
+    desc "completion_script", "Generates a script that can be eval to setup auto-completion."
+    long_desc Help.text("completion_script")
+    def completion_script
+      Completer::Script.generate
+    end
+
+    desc "version", "prints version"
+    def version
+      puts VERSION
+    end
   end
 end