sonic-screwdriver 1.4.0 → 2.2.1

data/docs/_docs/tutorial-ecs-run.md

@@ -1,100 +0,0 @@
----
-title: ECS Run
----
-
-The nice thing about the previous `ecs-exec` command we covered is that it allows you to get into the actual running container and debug with the exact environment that is on production.  The cavaet with doing this is that we are affecting a live process in actual use. If you do something inadvertently wrong on the server it could affect users.  Sometimes it is nice to start up a new container with the exact same environment as the other running containers but be isolated so you cannot affect live requests.
-
-The `sonic ecs-run` command is similar to the `sonic ecs-exec` command except it'll run a brand new container with the same environment variables as the task associated with the service. This allows you to debug in a container with the exact environment variables as the running tasks/containers without affecting the live service. So this is safer since you will not be able to mess up a live container that is in service.
-
-### sonic ecs-run
-
-```sh
-sonic ecs-run [ECS_SERVICE] --cluster [ECS_CLUSTER]
-```
-
-Here's an example:
-
-```sh
-sonic ecs-run hi-web-stag
-```
-
-You see something like this:
-
-```sh
-$ sonic ecs-run hi-web-stag
-Running: scp -r /tmp/sonic ec2-user@34.211.195.71:/tmp/sonic  > /dev/null
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-=> ssh -t ec2-user@34.211.195.71 bash /tmp/sonic/bash_scripts/docker-run.sh
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-+ exec bundle exec rails server -b 0.0.0.0
-=> Booting WEBrick
-=> Rails 4.2.3 application starting in development on http://0.0.0.0:3000
-=> Run `rails server -h` for more startup options
-=> Ctrl-C to shutdown server
-[2017-06-14 19:01:30] INFO  WEBrick 1.3.1
-[2017-06-14 19:01:30] INFO  ruby 2.3.3 (2016-11-21) [x86_64-linux]
-[2017-06-14 19:01:30] INFO  WEBrick::HTTPServer#start: pid=1 port=3000
-^C[2017-06-14 19:01:34] INFO  going to shutdown ...
-[2017-06-14 19:01:34] INFO  WEBrick::HTTPServer#start done.
-Exiting
-Connection to 34.211.195.71 closed.
-$
-```
-
-In the above output a WEBrick server gets started.  The reason this happens is because the Dockerfile default `CMD` in this project happens to start a webserver.  Most of the time you probably want to start shell for debugging.  To start a bash shell just tack the bash command at the end.
-
-```sh
-$ sonic ecs-run hi-web-stag bash
-Running: scp -r /tmp/sonic ec2-user@34.211.195.71:/tmp/sonic  > /dev/null
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-=> ssh -t ec2-user@34.211.195.71 bash /tmp/sonic/bash_scripts/docker-run.sh bash
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-root@56a495dbd5cd:/app#
-```
-
-You are now in a docker container running exactly the same environment as the other running containers with the `hi-web-stag` service. While this looks similiar to the `ecs-exec` command this container is a brand new process and is isolated from any live request. You can do whatever you want in this container and experiment to your heart's content.
-
-We can prove that this is a brand new docker container that is outside of ECS' knowledge. Let's ssh into the same instance and take a look at all the running docker containers in another terminal.
-
-```sh
-$ sonic ssh hi-web-stag docker ps
-=> ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ec2-user@34.211.195.71 docker ps
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-CONTAINER ID        IMAGE                                          COMMAND             CREATED             STATUS              PORTS                     NAMES
-29e7c1253c46        tongueroo/hi:ufo-2017-06-13T14-48-08-0a9eea5   "bash"              54 seconds ago      Up 53 seconds       3000/tcp                  cocky_goldstine
-fc4035f90bdc        tongueroo/hi:ufo-2017-06-13T14-48-08-0a9eea5   "bin/web"           About an hour ago   Up About an hour    0.0.0.0:32768->3000/tcp   ecs-hi-web-stag-11-web-9eb081978abad89a9701
-bf646ae7789a        amazon/amazon-ecs-agent:latest                 "/agent"            About an hour ago   Up About an hour                              ecs-agent
-$
-```
-
-The output shows that there is this extra runnning container called `cocky_goldstine`.  This name does not look like the typical ECS managed running docker container: `ecs-hi-web-stag-11-web-9eb081978abad89a9701`.  This is how we know that this is a container outside of ECS control.
-
-```sh
-$ sonic ecs-run hi-web-stag bash
-Running: scp -r /tmp/sonic ec2-user@34.211.195.71:/tmp/sonic  > /dev/null
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-=> ssh -t ec2-user@34.211.195.71 bash /tmp/sonic/bash_scripts/docker-run.sh bash
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-root@29e7c1253c46:/app# exit
-exit
-Connection to 34.211.195.71 closed.
-$
-```
-
-Let's exit out of the first terminal where you ran the original `sonic ecs-run` command and then list the running containers again.
-
-```sh
-$ sonic ssh hi-web-stag docker ps
-=> ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ec2-user@34.211.195.71 docker ps
-Warning: Permanently added '34.211.195.71' (ECDSA) to the list of known hosts.
-CONTAINER ID        IMAGE                                          COMMAND             CREATED             STATUS              PORTS                     NAMES
-fc4035f90bdc        tongueroo/hi:ufo-2017-06-13T14-48-08-0a9eea5   "bin/web"           About an hour ago   Up About an hour    0.0.0.0:32768->3000/tcp   ecs-hi-web-stag-11-web-9eb081978abad89a9701
-bf646ae7789a        amazon/amazon-ecs-agent:latest                 "/agent"            About an hour ago   Up About an hour                              ecs-agent
-$
-```
-
-Zapped!  The `cocky_goldstine` container that was created with `sonic ecs-run` is no more.
-
-<a id="prev" class="btn btn-basic" href="{% link _docs/tutorial-ecs-exec.md %}">Back</a>
-<a id="next" class="btn btn-primary" href="{% link _docs/tutorial-execute.md %}">Next Step</a>
-<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>

data/lib/sonic/cli/help.rb

@@ -1,152 +0,0 @@
-# https://github.com/erikhuda/thor/issues/398
-class Thor
-  module Shell
-    class Basic
-      def print_wrapped(message, options = {})
-        pretty_message = message.split("\n").map{|l| "  #{l}"}.join("\n")
-        stdout.puts pretty_message
-      end
-    end
-  end
-end
-
-module Sonic
-  class CLI < Command
-    class Help
-      class << self
-        def ssh
-<<-EOL
-Ssh quicky into an EC2 instance using an identifier. The identifier can be many things. Examples of valid identifiers:
-
-* EC2 instance id. Example: i-067c5e3f026c1e801
-* EC2 tag value. Any tag value is search, the tag key does not matter only the tag value matters. Example: hi-web-prod
-* ECS service. Example: my-ecs-service
-* ECS container instance id. Example: 7fbc8c75-4675-4d39-a5a4-0395ff8cd474
-* ECS task id. Example: 1ed12abd-645c-4a05-9acf-739b9d790170
-
-When using ecs identifiers the `--cluster` option is required or can be set in ~/.sonic/settings.yml.
-
-Examples:
-
-$ sonic ssh i-067c5e3f026c1e801
-$ sonic ssh hi-web-prod
-$ sonic ssh --cluster my-cluster my-ecs-service
-$ sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474
-$ sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170
-
-Sonic ssh builds up the ssh command and calls it. For example, the following commands:
-
-sonic ssh i-027363802c6ff314f
-
-Translates to:
-
-ssh ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com
-
-You can also tack on any command to be run at the end of the command. Example:
-
-$ sonic ssh i-027363802c6ff314f uptime
-=> ssh ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com uptime
- 15:57:02 up 18:21,  0 users,  load average: 0.00, 0.01, 0.00
-
-Specifying pem keys:
-
-The recommended way to specify custom private keys is to use ssh-agent as covered here: https://blog.boltops.com/2017/09/21/3-ssh-tips-ssh-agent-tunnel-and-escaping-from-the-dead
-
-But you can also specify the pem key to use with the -i option.  Example:
-
-$ sonic ssh -i ~/.ssh/id_rsa-custom ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com
-
-Retry option:
-
-For newly launched instances, the instance's ssh access might not be quite ready. Typically, you must press up enter repeatedly until the instance is ready.  Sonic ssh has a retry option that automates this. Example:
-
-$ sonic ssh -r i-027363802c6ff314f
-
-Bastion Host Support
-
-Sonic ssh also supports a bastion host.
-
-$ sonic ssh --bastion bastion.domain.com i-027363802c6ff314f
-$ sonic ssh --bastion user@bastion.domain.com i-027363802c6ff314f
-
-Here's what the output looks like:
-
-$ sonic ssh --bastion 34.211.223.3 i-0f7f833131a51ce35 uptime
-=> ssh -At ec2-user@34.211.223.3 ssh ec2-user@10.10.110.135 uptime
- 17:57:59 up 37 min,  0 users,  load average: 0.00, 0.02, 0.00
-Connection to 34.211.223.3 closed.
-$
-EOL
-        end
-
-        def ecs_exec
-<<-EOL
-Ssh into an ECS container instance, finds a running docker container associated
-with the service and docker exec's into it.
-
-Examples:
-
-$ sonic ecs-exec my-service --cluster my-cluster
-
-You can use a variety of identifiers.  These include the ECS service name and task id.
-EOL
-        end
-
-        def ecs_run
-<<-EOL
-Ssh into an ECS container instance and runs a docker container using the same
-environment and image as the specified running service.
-
-Examples:
-
-$ sonic ecs-run --cluster my-cluster my-service
-$ sonic ecs-run --cluster my-cluster my-service
-
-# If there are flags in the command that you want to pass down to the docker
-run command you will need to put the command in single quotes.  This is due to
-the way Thor (what this tool uses) parses options.
-
-$ sonic ecs-run --cluster prod-hi hi-web-prod 'rake -T'
-EOL
-        end
-
-        def execute
-<<-EOL
-Run as a command across a list of servers. A filter must be provided.  The filter can be a mix of instance ids and ec2 tags. This command can also take a path to a file. To specify a path to a file use file:// at the beginning of your file.
-
-Examples:
-
-$ sonic execute hi-web-prod uptime
-$ sonic execute hi-web-prod,hi-worker-prod,hi-clock-prod uptime
-$ sonic execute i-030033c20c54bf149,i-030033c20c54bf150 uname -a
-$ sonic execute i-030033c20c54bf149 file://hello.sh
-
-You cannot mix instance ids and tag names in the filter.
-EOL
-        end
-
-        def list
-<<-EOL
-List ec2 servers. A filter must be provided.  The filter can be a mix of instance ids and ec2 tags. sonic list will auto-detect the what type of filter it is filter appropriately.  The filter for listing is optional.
-
-Examples:
-
-$ sonic list
-$ sonic list hi-web-prod
-$ sonic list hi-web-prod,hi-clock-prod
-$ sonic list i-09482b1a6e330fbf7
-
-Example Output:
-
-$ sonic list --header i-09482b1a6e330fbf7
-Instance Id Public IP Private IP  Type
-i-09482b1a6e330fbf7 54.202.152.168  172.31.21.108 t2.small
-$
-
-You cannot mix instance ids and tag names in the filter.
-EOL
-        end
-      end
-    end
-  end
-end

data/lib/sonic/settings.rb

@@ -1,115 +0,0 @@
-require 'yaml'
-
-module Sonic
-  class Settings
-    def initialize(project_root=nil)
-      @project_root = project_root || '.'
-    end
-
-    def data
-      return @data if @data
-
-      project_file = "#{@project_root}/.sonic/settings.yml"
-      project = File.exist?(project_file) ? YAML.load_file(project_file) : {}
-
-      user_file = "#{home}/.sonic/settings.yml"
-      user = File.exist?(user_file) ? YAML.load_file(user_file) : {}
-
-      default_file = File.expand_path("../default/settings.yml", __FILE__)
-      default = YAML.load_file(default_file)
-
-      @data = default.merge(user.merge(project))
-      ensure_default_cluster!(@data)
-      ensure_default_bastion!(data)
-      @data
-    end
-
-    # Public: Returns default cluster based on the ECS service name.
-    #
-    # service  - ECS service
-    # count - The Integer number of times to duplicate the text.
-    #
-    # The settings.yml format:
-    #
-    # service_cluster:
-    #   default: stag
-    #   hi-web-prod: prod
-    #   hi-clock-prod: prod
-    #   hi-worker-prod: prod
-    #   hi-web-stag: stag
-    #   hi-clock-stag: stag
-    #   hi-worker-stag: stag
-    #
-    # Examples
-    #
-    #   default_cluster('hi-web-prod')
-    #   # => 'prod'
-    #   default_cluster('whatever')
-    #   # => 'stag'
-    #
-    # Returns the ECS cluster name.
-    def default_cluster(service)
-      service_cluster = data["service_cluster"]
-      service_cluster[service] || service_cluster["default"]
-    end
-
-    # When user's .sonic/settings.yml lack the default cluster, we add it on.
-    # Otherwise the user get confusing and scary aws-sdk-core/param_validator errors:
-    # Example: https://gist.github.com/sonic/67b9a68a77363b908d1c36047bc2709a
-    def ensure_default_cluster!(data)
-      unless data["service_cluster"]["default"]
-        data["service_cluster"]["default"] = "default"
-      end
-      data
-    end
-
-    # Public: Returns default bastion host.
-    #
-    # cluster  - cluster provided by user
-    #
-    # The settings.yml format:
-    #
-    # bastion:
-    #   default: bastion.mydomain.com
-    #   prod: bastion.mydomain.com
-    #   stag: bastion-stag.mydomain.com
-    #
-    # Examples
-    #
-    #   default_bastion('stag')
-    #   # => 'bastion-stag.mydomain.com'
-    #   default_bastion('whatever')
-    #   # => 'bastion.mydomain.com'
-    #
-    # Returns the bastion host that is mapped to the cluster
-    def default_bastion(cluster)
-      bastion = data["bastion"]
-      bastion[cluster] || bastion["default"]
-    end
-
-    # When user's .sonic/settings.yml lack the default cluster, we add it on.
-    def ensure_default_bastion!(data)
-      unless data["bastion"] && data["bastion"].has_key?("default")
-        data["bastion"] ||= {}
-        data["bastion"]["default"] = nil
-      end
-      data
-    end
-
-    # By default bypass strict host key checking for convenience.
-    # But user can overrride this.
-    def host_key_check_options
-      if data["host_key_check"] == true
-        []
-      else
-        # disables host key checking
-        %w[-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null]
-      end
-    end
-
-    def home
-      # hack but fast
-      ENV['TEST'] ? "spec/fixtures/home" : ENV['HOME']
-    end
-  end
-end