sonic-screwdriver 1.4.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. data/.circleci/bin/commit_docs.sh +26 -0
  3. data/.circleci/config.yml +70 -0
  4. data/.gitignore +1 -1
  5. data/.ruby-version +1 -0
  6. data/CHANGELOG.md +13 -2
  7. data/Gemfile +3 -3
  8. data/Gemfile.lock +43 -14
  9. data/Guardfile +17 -10
  10. data/LICENSE.txt +2 -2
  11. data/README.md +10 -10
  12. data/Rakefile +9 -2
  13. data/docs/_config.yml +3 -0
  14. data/docs/_docs/help.md +1 -1
  15. data/docs/_docs/install-bastion.md +5 -15
  16. data/docs/_docs/install.md +3 -3
  17. data/docs/_docs/settings.md +40 -56
  18. data/docs/_docs/tutorial-ecs-exec.md +16 -20
  19. data/docs/_docs/tutorial-ecs-sh.md +73 -0
  20. data/docs/_docs/tutorial-execute.md +93 -17
  21. data/docs/_docs/tutorial-ssh.md +13 -18
  22. data/docs/_docs/why-ec2-run-command.md +1 -1
  23. data/docs/_includes/commands.html +5 -5
  24. data/docs/_includes/content.html +5 -0
  25. data/docs/_includes/css/main.css +15 -9
  26. data/docs/_includes/css/sonic.css +7 -5
  27. data/docs/_includes/example.html +4 -4
  28. data/docs/_includes/reference.md +1 -0
  29. data/docs/_includes/subnav.html +2 -1
  30. data/docs/_reference/sonic-completion.md +44 -0
  31. data/docs/_reference/sonic-completion_script.md +25 -0
  32. data/docs/_reference/sonic-ecs-exec.md +30 -0
  33. data/docs/_reference/sonic-ecs-help.md +21 -0
  34. data/docs/_reference/sonic-ecs-sh.md +35 -0
  35. data/docs/_reference/sonic-ecs.md +25 -0
  36. data/docs/_reference/sonic-execute.md +84 -0
  37. data/docs/_reference/sonic-list.md +40 -0
  38. data/docs/_reference/sonic-ssh.md +86 -0
  39. data/docs/_reference/sonic-version.md +21 -0
  40. data/docs/img/tutorials/ec2-console-run-command.png +0 -0
  41. data/docs/quick-start.md +9 -10
  42. data/docs/reference.md +12 -0
  43. data/{bin → exe}/sonic +3 -3
  44. data/lib/bash_scripts/docker-exec.sh +1 -0
  45. data/lib/bash_scripts/docker-run.sh +8 -1
  46. data/lib/sonic.rb +10 -2
  47. data/lib/sonic/{aws_services.rb → aws_service.rb} +6 -1
  48. data/lib/sonic/base_command.rb +82 -0
  49. data/lib/sonic/cli.rb +37 -27
  50. data/lib/sonic/command.rb +8 -22
  51. data/lib/sonic/completer.rb +161 -0
  52. data/lib/sonic/completer/script.rb +6 -0
  53. data/lib/sonic/completer/script.sh +10 -0
  54. data/lib/sonic/core.rb +15 -0
  55. data/lib/sonic/default/settings.yml +6 -16
  56. data/lib/sonic/docker.rb +29 -1
  57. data/lib/sonic/ecs.rb +22 -0
  58. data/lib/sonic/execute.rb +153 -18
  59. data/lib/sonic/help.rb +9 -0
  60. data/lib/sonic/help/command/send.md +10 -0
  61. data/lib/sonic/help/completion.md +22 -0
  62. data/lib/sonic/help/completion_script.md +3 -0
  63. data/lib/sonic/help/ecs/exec.md +8 -0
  64. data/lib/sonic/help/ecs/sh.md +13 -0
  65. data/lib/sonic/help/execute.md +60 -0
  66. data/lib/sonic/help/list.md +17 -0
  67. data/lib/sonic/help/ssh.md +60 -0
  68. data/lib/sonic/list.rb +4 -1
  69. data/lib/sonic/setting.rb +47 -0
  70. data/lib/sonic/ssh.rb +41 -20
  71. data/lib/sonic/ssh/identifier_detector.rb +6 -2
  72. data/lib/sonic/version.rb +1 -1
  73. data/sonic.gemspec +14 -9
  74. data/spec/lib/cli_spec.rb +5 -10
  75. data/spec/lib/sonic/execute_spec.rb +0 -1
  76. data/spec/spec_helper.rb +18 -10
  77. metadata +115 -16
  78. data/docs/_docs/tutorial-ecs-run.md +0 -100
  79. data/lib/sonic/cli/help.rb +0 -152
  80. data/lib/sonic/settings.rb +0 -115
@@ -0,0 +1,84 @@
1
+ ---
2
+ title: sonic execute
3
+ reference: true
4
+ ---
5
+
6
+ ## Usage
7
+
8
+ sonic execute [FILTER] [COMMAND]
9
+
10
+ ## Description
11
+
12
+ Runs command across fleet of servers via AWS Run Command.
13
+
14
+ Run as a command across a list of servers. A filter must be provided. The filter can be a mix of instance ids and ec2 tags. This command can also take a path to a file. To specify a path to a file use file:// at the beginning of your file.
15
+
16
+ ## Examples Summary
17
+
18
+ $ sonic execute hi-web-prod uptime
19
+ $ sonic execute hi-web-prod,hi-worker-prod,hi-clock-prod uptime
20
+ $ sonic execute i-030033c20c54bf149,i-030033c20c54bf150 uname -a
21
+ $ sonic execute i-030033c20c54bf149 file://hello.sh
22
+
23
+ You cannot mix instance ids and tag names in the filter.
24
+
25
+ ## Example Detailed
26
+
27
+ Here's a command example output in detailed:
28
+
29
+ ```sh
30
+ $ sonic execute i-0bf51a000ab4e73a8 uptime
31
+ Sending command to SSM with options:
32
+ ---
33
+ instance_ids:
34
+ - i-0bf51a000ab4e73a8
35
+ document_name: AWS-RunShellScript
36
+ comment: sonic execute i-0bf51a000ab4e73a8 uptime
37
+ parameters:
38
+ commands:
39
+ - uptime
40
+ output_s3_region: us-east-1
41
+ output_s3_bucket_name: [reacted]
42
+ output_s3_key_prefix: ssm/commands/sonic
43
+
44
+ Command sent to AWS SSM. To check the details of the command:
45
+ aws ssm list-commands --command-id 0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2
46
+ aws ssm get-command-invocation --command-id 0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2 --instance-id i-0bf51a000ab4e73a8
47
+
48
+ Waiting for ssm command to finish.....
49
+ Command finished.
50
+
51
+ Displaying output for i-0bf51a000ab4e73a8.
52
+ Command status: Success
53
+ Command standard output:
54
+ 01:08:10 up 8 days, 6:41, 0 users, load average: 0.00, 0.00, 0.00
55
+
56
+ To see the more output details visit:
57
+ https://us-west-2.console.aws.amazon.com/systems-manager/run-command/0bb18d58-6436-49fd-9bfd-0c4b6c51c7a2
58
+
59
+ Pro tip: the console url is already in your copy/paste clipboard.
60
+ $
61
+ ```
62
+
63
+ Notice the conveniences of `sonic execute`, it:
64
+
65
+ 1. Showed the parameters that will be sent as part of the send_command call to SSM.
66
+ 2. Sent the command to SSM.
67
+ 3. Waited for the command to finish.
68
+ 4. Displayed the output of the command.
69
+ 5. Provided the console url that visit to view more details about the SSM command.
70
+
71
+ The AWS SSM console looks like this:
72
+
73
+ <img src="/img/tutorials/ec2-console-run-command.png" class="doc-photo" />
74
+
75
+
76
+ ## Options
77
+
78
+ ```
79
+ [--zero-warn], [--no-zero-warn] # Warns user when no instances found
80
+ # Default: true
81
+ [--verbose], [--no-verbose]
82
+ [--noop], [--no-noop]
83
+ ```
84
+
@@ -0,0 +1,40 @@
1
+ ---
2
+ title: sonic list
3
+ reference: true
4
+ ---
5
+
6
+ ## Usage
7
+
8
+ sonic list [FILTER]
9
+
10
+ ## Description
11
+
12
+ Lists ec2 instances.
13
+
14
+ List ec2 servers. A filter must be provided. The filter can be a mix of instance ids and ec2 tags. sonic list will auto-detect the what type of filter it is filter appropriately. The filter for listing is optional.
15
+
16
+ Examples:
17
+
18
+ $ sonic list
19
+ $ sonic list hi-web-prod
20
+ $ sonic list hi-web-prod,hi-clock-prod
21
+ $ sonic list i-09482b1a6e330fbf7
22
+
23
+ Example Output:
24
+
25
+ $ sonic list --header i-09482b1a6e330fbf7
26
+ Instance Id Public IP Private IP Type
27
+ i-09482b1a6e330fbf7 54.202.152.168 172.31.21.108 t2.small
28
+ $
29
+
30
+ You cannot mix instance ids and tag names in the filter.
31
+
32
+
33
+ ## Options
34
+
35
+ ```
36
+ [--header], [--no-header] # Displays header
37
+ [--verbose], [--no-verbose]
38
+ [--noop], [--no-noop]
39
+ ```
40
+
@@ -0,0 +1,86 @@
1
+ ---
2
+ title: sonic ssh
3
+ reference: true
4
+ ---
5
+
6
+ ## Usage
7
+
8
+ sonic ssh [IDENTIFER]
9
+
10
+ ## Description
11
+
12
+ Ssh into a instance using identifier. identifer can be several things: instance id, ec2 tag, ECS service name, etc.
13
+
14
+ * EC2 instance id. Example: i-067c5e3f026c1e801
15
+ * EC2 tag value. Any tag value is search, the tag key does not matter only the tag value matters. Example: hi-web
16
+ * ECS service. Example: my-ecs-service
17
+ * ECS container instance id. Example: 7fbc8c75-4675-4d39-a5a4-0395ff8cd474
18
+ * ECS task id. Example: 1ed12abd-645c-4a05-9acf-739b9d790170
19
+
20
+ When using ecs identifiers the `--cluster` option is required or can be set in ~/.sonic/settings.yml.
21
+
22
+ Examples:
23
+
24
+ $ sonic ssh i-067c5e3f026c1e801
25
+ $ sonic ssh hi-web
26
+ $ sonic ssh --cluster my-cluster my-ecs-service
27
+ $ sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474
28
+ $ sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170
29
+
30
+ Sonic ssh builds up the ssh command and shells out to it. For example, the following commands:
31
+
32
+ sonic ssh i-027363802c6ff314f
33
+
34
+ Translates to:
35
+
36
+ ssh ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com
37
+
38
+ You can also tack on any command to be run at the end of the command. Example:
39
+
40
+ $ sonic ssh i-027363802c6ff314f uptime
41
+ => ssh ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com uptime
42
+ 15:57:02 up 18:21, 0 users, load average: 0.00, 0.01, 0.00
43
+
44
+ ## Specifying pem keys
45
+
46
+ The recommended way to specify custom private keys is to use ssh-agent as covered here: https://blog.boltops.com/2017/09/21/3-ssh-tips-ssh-agent-tunnel-and-escaping-from-the-dead
47
+
48
+ But you can also specify the pem key to use with the -i option. Example:
49
+
50
+ $ sonic ssh -i ~/.ssh/id_rsa-custom ec2-user@ec2-52-24-216-170.us-west-2.compute.amazonaws.com
51
+
52
+ ## Retry option
53
+
54
+ For newly launched instances, the instance's ssh access might not be quite ready. Typically, you must press up enter repeatedly until the instance is ready. Sonic ssh has a retry option that automates this. Example:
55
+
56
+ $ sonic ssh -r i-027363802c6ff314f
57
+
58
+ Bastion Host Support
59
+
60
+ Sonic ssh also supports a bastion host.
61
+
62
+ $ sonic ssh --bastion bastion.domain.com i-027363802c6ff314f
63
+ $ sonic ssh --bastion user@bastion.domain.com i-027363802c6ff314f
64
+
65
+ Here's what the output looks like:
66
+
67
+ $ sonic ssh --bastion 34.211.223.3 i-0f7f833131a51ce35 uptime
68
+ => ssh -At ec2-user@34.211.223.3 ssh ec2-user@10.10.110.135 uptime
69
+ 17:57:59 up 37 min, 0 users, load average: 0.00, 0.02, 0.00
70
+ Connection to 34.211.223.3 closed.
71
+ $
72
+
73
+ You can also set the bastion host and other options with a [settings file](http://sonic-screwdriver.cloud/docs/settings/).
74
+
75
+
76
+ ## Options
77
+
78
+ ```
79
+ -i, [--keys=KEYS] # comma separated list of ssh private key paths
80
+ -r, [--retry], [--no-retry] # keep retrying the server login until successful. Useful when on newly launched instances.
81
+ [--bastion=BASTION] # Bastion jump host to use. Defaults to no bastion server.
82
+ [--cluster=CLUSTER] # ECS Cluster to use. Default cluster is default
83
+ [--verbose], [--no-verbose]
84
+ [--noop], [--no-noop]
85
+ ```
86
+
@@ -0,0 +1,21 @@
1
+ ---
2
+ title: sonic version
3
+ reference: true
4
+ ---
5
+
6
+ ## Usage
7
+
8
+ sonic version
9
+
10
+ ## Description
11
+
12
+ prints version
13
+
14
+
15
+ ## Options
16
+
17
+ ```
18
+ [--verbose], [--no-verbose]
19
+ [--noop], [--no-noop]
20
+ ```
21
+
data/docs/quick-start.md CHANGED
@@ -15,26 +15,25 @@ gem install sonic-screwdriver
15
15
  ```sh
16
16
  # ssh into an instance
17
17
  sonic ssh i-0f7f833131a51ce35
18
- sonic ssh hi-web-stag # ec2 tag
19
- sonic ssh hi-web-stag --cluster stag # ecs service name
20
- sonic ssh hi-web-stag --cluster stag # ecs service name
21
- sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474 --cluster stag # ECS container id
22
- sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170 --cluster stag # ECS task id
18
+ sonic ssh hi-web # ec2 tag
19
+ sonic ssh hi-web --cluster staging # ecs service name
20
+ sonic ssh 7fbc8c75-4675-4d39-a5a4-0395ff8cd474 --cluster staging # ECS container id
21
+ sonic ssh 1ed12abd-645c-4a05-9acf-739b9d790170 --cluster staging # ECS task id
23
22
 
24
23
  # docker exec to a running ECS docker container
25
- sonic ecs-exec hi-web-stag
24
+ sonic ecs exec hi-web
26
25
 
27
26
  # docker run with same environment as the ECS docker running containers
28
- sonic ecs-run hi-web-stag
27
+ sonic ecs sh hi-web
29
28
 
30
29
  # run command on 1 instance
31
30
  sonic execute i-0f7f833131a51ce35 uptime
32
31
 
33
- # run command on all instances tagged with hi-web-stag and worker
34
- sonic execute hi-web-stag,hi-worker-stag uptime
32
+ # run command on all instances tagged with hi-web and worker
33
+ sonic execute hi-web,hi-worker uptime
35
34
 
36
35
  # list ec2 instances
37
- sonic list hi-web-stag
36
+ sonic list hi-web
38
37
  ```
39
38
 
40
39
  Congratulations! You now know the basic sonic screwdriver commands now.
data/docs/reference.md ADDED
@@ -0,0 +1,12 @@
1
+ ---
2
+ title: CLI Reference
3
+ ---
4
+ {% include reference.md %}
5
+
6
+ * [sonic completion]({% link _reference/sonic-completion.md %})
7
+ * [sonic completion_script]({% link _reference/sonic-completion_script.md %})
8
+ * [sonic ecs]({% link _reference/sonic-ecs.md %})
9
+ * [sonic execute]({% link _reference/sonic-execute.md %})
10
+ * [sonic list]({% link _reference/sonic-list.md %})
11
+ * [sonic ssh]({% link _reference/sonic-ssh.md %})
12
+ * [sonic version]({% link _reference/sonic-version.md %})
data/{bin → exe}/sonic RENAMED
@@ -7,8 +7,8 @@ Signal.trap("INT") {
7
7
  exit
8
8
  }
9
9
 
10
- $:.unshift(File.expand_path('../../lib', __FILE__))
11
- require 'sonic'
12
- require 'sonic/cli'
10
+ $:.unshift(File.expand_path("../../lib", __FILE__))
11
+ require "sonic"
12
+ require "sonic/cli"
13
13
 
14
14
  Sonic::CLI.start(ARGV)
@@ -12,4 +12,5 @@ else
12
12
  COMMAND=bash
13
13
  fi
14
14
 
15
+ set -x
15
16
  exec docker exec -ti $CONTAINER_ID $COMMAND
@@ -12,4 +12,11 @@ CONTAINER_IMAGE=$(cat /tmp/sonic/docker-image.txt)
12
12
  cp /tmp/sonic/env-file.txt ~/
13
13
  rm -rf /tmp/sonic
14
14
 
15
- exec docker run -ti --env-file ~/env-file.txt $CONTAINER_IMAGE "$@"
15
+ if [ $# -eq 0 ]; then
16
+ COMMAND=bash
17
+ else
18
+ COMMAND=$@
19
+ fi
20
+
21
+ set -x
22
+ exec docker run -ti --env-file ~/env-file.txt "$CONTAINER_IMAGE" $COMMAND
data/lib/sonic.rb CHANGED
@@ -3,14 +3,22 @@ require "sonic/version"
3
3
  require "colorize"
4
4
 
5
5
  module Sonic
6
- autoload :AwsServices, 'sonic/aws_services'
6
+ autoload :Core, 'sonic/core'
7
+ autoload :Help, 'sonic/help'
8
+ autoload :AwsService, 'sonic/aws_service'
7
9
  autoload :CLI, 'sonic/cli'
10
+ autoload :BaseCommand, 'sonic/base_command'
8
11
  autoload :Command, 'sonic/command'
9
12
  autoload :Docker, 'sonic/docker'
10
13
  autoload :Execute, 'sonic/execute'
11
14
  autoload :List, 'sonic/list'
12
- autoload :Settings, 'sonic/settings'
15
+ autoload :Setting, 'sonic/setting'
13
16
  autoload :Ssh, 'sonic/ssh'
14
17
  autoload :UI, 'sonic/ui'
15
18
  autoload :Checks, 'sonic/checks'
19
+ autoload :Completion, "sonic/completion"
20
+ autoload :Completer, "sonic/completer"
21
+ autoload :Ecs, "sonic/ecs"
22
+
23
+ extend Core
16
24
  end
@@ -1,9 +1,10 @@
1
1
  require "aws-sdk-ec2"
2
2
  require "aws-sdk-ecs"
3
3
  require "aws-sdk-ssm"
4
+ require "aws-sdk-s3"
4
5
 
5
6
  module Sonic
6
- module AwsServices
7
+ module AwsService
7
8
  def ecs
8
9
  @ecs ||= Aws::ECS::Client.new
9
10
  end
@@ -19,5 +20,9 @@ module Sonic
19
20
  def ssm
20
21
  @ssm ||= Aws::SSM::Client.new
21
22
  end
23
+
24
+ def s3
25
+ @s3 ||= Aws::S3::Client.new
26
+ end
22
27
  end
23
28
  end
@@ -0,0 +1,82 @@
1
+ require "thor"
2
+
3
+ # Override thor's long_desc identation behavior
4
+ # https://github.com/erikhuda/thor/issues/398
5
+ class Thor
6
+ module Shell
7
+ class Basic
8
+ def print_wrapped(message, options = {})
9
+ message = "\n#{message}" unless message[0] == "\n"
10
+ stdout.puts message
11
+ end
12
+ end
13
+ end
14
+ end
15
+
16
+ module Sonic
17
+ class BaseCommand < Thor
18
+ class << self
19
+ def dispatch(m, args, options, config)
20
+ # Allow calling for help via:
21
+ # sonic command help
22
+ # sonic command -h
23
+ # sonic command --help
24
+ # sonic command -D
25
+ #
26
+ # as well thor's normal way:
27
+ #
28
+ # sonic help command
29
+ help_flags = Thor::HELP_MAPPINGS + ["help"]
30
+ if args.length > 1 && !(args & help_flags).empty?
31
+ args -= help_flags
32
+ args.insert(-2, "help")
33
+ end
34
+
35
+ # sonic version
36
+ # sonic --version
37
+ # sonic -v
38
+ version_flags = ["--version", "-v"]
39
+ if args.length == 1 && !(args & version_flags).empty?
40
+ args = ["version"]
41
+ end
42
+
43
+ super
44
+ end
45
+
46
+ # Override command_help to include the description at the top of the
47
+ # long_description.
48
+ def command_help(shell, command_name)
49
+ meth = normalize_command_name(command_name)
50
+ command = all_commands[meth]
51
+ alter_command_description(command)
52
+ super
53
+ end
54
+
55
+ def alter_command_description(command)
56
+ return unless command
57
+
58
+ # Add description to beginning of long_description
59
+ long_desc = if command.long_description
60
+ "#{command.description}\n\n#{command.long_description}"
61
+ else
62
+ command.description
63
+ end
64
+
65
+ # add reference url to end of the long_description
66
+ unless website.empty?
67
+ full_command = [command.ancestor_name, command.name].compact.join('-')
68
+ url = "#{website}/reference/sonic-#{full_command}"
69
+ long_desc += "\n\nHelp also available at: #{url}"
70
+ end
71
+
72
+ command.long_description = long_desc
73
+ end
74
+ private :alter_command_description
75
+
76
+ # meant to be overriden
77
+ def website
78
+ "http://sonic-screwdriver.cloud"
79
+ end
80
+ end
81
+ end
82
+ end