knuckle_cluster 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d1938a27084e6da4bd72ce501a8e373a5a919b18
+  data.tar.gz: e27ce437a1519ce998b79429d00ca18c793a9fae
+SHA512:
+  metadata.gz: 3b470a48b031d80f4761aa538243862beb946e46825b37fbdd361db52216452d21bca92ae02641e62804303b50823590edf2a5eea91ff081d6f26c94a685872c
+  data.tar.gz: 56ef57f3f84bc4cc707296d67592b5ac8e5867a94663b0f2fe21eda609c154fbc70c9f94feb6f9e231a05e09c6dd9f247a789bebe40151d528ef1f10cbd46ad7

data/.gitignore

@@ -0,0 +1,13 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+
+# rspec failure tracking
+.rspec_status
+*.gem

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.1
+before_install: gem install bundler -v 1.14.6

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,13 @@
+# Contributor Code of Conduct
+
+As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
+
+Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
+
+This Code of Conduct is adapted from the [Contributor Covenant](http:contributor-covenant.org), version 1.0.0, available at [http://contributor-covenant.org/version/1/0/0/](http://contributor-covenant.org/version/1/0/0/)

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in knuckle_cluster.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 Peter Hofmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,227 @@
+# KnuckleCluster
+
+Have you ever wanted to shuck away the hard, rough exterior of an ECS cluster and get to the soft, chewy innards? Sounds like you need KnuckleCluster!
+This tool provides scripts, invoked via cli or rakefile, to list, connect to and/or run commands on ecs agents and containers via ssh.  This makes it very easy to interrogate ECS agents and containers without having to go digging for IP addresses and things.
+
+## Features
+* See what agents in your ECS cluster are doing
+* Easily connect to running agents
+* Easily connect and get a console inside running containers
+* Create shortcuts to oft-used commands and run them easily
+* Optionally integrates with [aws-vault](https://github.com/99designs/aws-vault) for AWS authentication
+
+## Development Status
+Is being used in production for various projects and is considered stable. Any new features/bug fixes etc are most welcome!  Test coverage is minimal...
+
+## Installation
+
+KnuckleCluster can be used in two ways:
+1.  As a system-wide executable with a config file (preferred)
+1.  As a rake task in your project
+
+Install the executable with:
+
+    $ gem install knuckle_cluster
+
+OR
+
+Add this line to your application's Gemfile:
+
+```ruby
+source 'https://rubygems.envato.com/' do
+  gem 'knuckle_cluster'
+end
+```
+
+And then execute:
+
+    $ bundle
+
+
+## Usage
+
+Knuckle Cluster can be used either as a part of an application via a rakefile, or at a system level.
+
+You'll need to execute knuckle_cluster with appropriate AWS permissions on the cluster in question, stored in your ENV. I like to use `aws-vault` to handle this for me.
+
+### System usage
+
+Create a file: `~/.ssh/knuckle_cluster`.  This is the config file that will be used to make connections from the command line.  It is a yaml file.  The connection name is the key, and all parameters are below it. EG:
+```
+platform:
+  cluster_name: platform-ecs-cluster-ABC123
+  region: us-east-1
+  bastion: platform_bastion
+  rsa_key_location: ~/.ssh/platform_rsa_key
+  ssh_username: ubuntu
+  sudo: true
+  aws_vault_profile: platform_super_user
+  shortcuts:
+    console:
+      container: web
+      command: bundle exec rails console
+    db:
+      container: worker
+      command: script/db_console
+  tunnels:
+    db:
+      local_port: 54321
+      remote_host: postgres-db.yourcompany.com
+      remote_port: 5432
+```
+
+You can also use inheritance to simplify the inclusion of multiple similar targets:
+```
+super_platform:
+  <<: *default_platform
+  cluster_name: super-platform-ecs-cluster-ABC123
+
+ultra_platform:
+  <<: *default_platform
+  cluster_name: ultra-platform-ecs-cluster-DEF987
+  sudo: false
+
+default_platform: &default_platform
+  region: us-east-1
+  bastion: platform_bastion
+  rsa_key_location: ~/.ssh/platform_rsa_key
+  ssh_username: ubuntu
+  sudo: true
+  aws_vault_profile: platform_super_user
+```
+
+See [Options for Knuckle Cluster](#options-for-knuckle-cluster) below for a list of what each option does.
+
+Command line options:
+
+```
+knuckle_cluster CLUSTER_PROFILE agents - list all agents and select one to start a shell
+knuckle_cluster CLUSTER_PROFILE containers - list all containers and select one to start a shell
+knuckle_cluster CLUSTER_PROFILE logs CONTAINER_NAME - tail the logs for a container
+knuckle_cluster CLUSTER_PROFILE CONTAINER_NAME [OPTIONAL COMMANDS] - connect to a container and start a shell or run a command
+knuckle_cluster CLUSTER_PROFILE SHORTCUT_NAME - run a shortcut defined in your knuckle_cluster configuration
+knuckle_cluster CLUSTER_PROFILE tunnel TUNNEL_NAME - open a tunnel defined in your knuckle_cluster configuration
+```
+
+### Rakefile usage
+
+It takes one argument at minimum: `cluster_name` .  A region is likely also required as it will default to `us-east-1`.
+Eg:
+```
+kc = KnuckleCluster.new(
+  cluster_name: 'platform-ecs-cluster-ABC123',
+  region: 'us-east-1',
+  bastion: 'platform_bastion',
+  rsa_key_location: "~/.ssh/platform_rsa_key",
+  ssh_username: "ubuntu",
+  sudo: true
+)
+task :agents do
+  kc.connect_to_agents
+end
+
+task :containers do
+  kc.connect_to_containers
+end
+```
+
+invoke with `rake agents` or `rake containers`
+
+
+Once you have an instance of KnuckleCluster, you can now do things!
+```
+$ knuckle_cluster super_platform containers
+```
+Which will give you the output and run bash for you on the actual docker container:
+```
+Listing Containers
+TASK              | AGENT               | INDEX | CONTAINER
+------------------|---------------------|-------|--------------------
+task-one          | i-123abc123abc123ab | 1     | t1-container-one
+task-two          | i-123abc123abc123ab | 2     | t2-container-one
+                  |                     | 3     | t2-container-two
+task-three        | i-456def456def456de | 4     | t3-container-one
+                  |                     | 5     | t3-container-two
+                  |                     | 6     | t3-container-three
+
+Connect to which container?
+```
+
+Same with connecting directly to agents
+```
+$ knuckle_cluster super_platform agents
+```
+```
+Listing Agents
+INDEX | INSTANCE_ID         | TASK       | CONTAINER
+------|---------------------|------------|--------------------
+1     | i-123abc123abc123ab | task-one   | t1-container-one
+      |                     | task-two   | t2-container-one
+      |                     |            | t2-container-two
+2     | i-456def456def456de | task-three | t3-container-one
+      |                     |            | t3-container-two
+      |                     |            | t3-container-three
+
+Connect to which agent?
+```
+
+Both `connect_to_containers` and `connect_to_agents` can have the following optional arguments:
+
+Argument | Description
+-------- | -----------
+command | Runs a command on the specified container/agent. When connecting to containers, this defaults to `bash`
+auto | Automatically connects to the first container/agent it can find. Handy when used in conjunction with `command`.
+
+
+
+Eg: To connect to a container, echo something and then immediately disconnect you could use:
+```
+kc.connect_to_containers(auto: true, command: "echo I love KnuckleCluster!")
+```
+
+
+## Options for Knuckle Cluster
+Possible options are below. If left blank, they will be ignored and defaults used where available.:
+
+Argument | Description
+-------- | -----------
+cluster_name | The name of the cluster (not the ARN). eg 'my-super-cluster'. Required
+region | The AWS region you would like to use. Defaults to `us-east-1`
+bastion | if you have a bastion to proxy to your ecs cluster via ssh, put the name of it here as defined in your `~/.ssh/config` file.
+rsa_key_location | The RSA key needed to connect to an ecs agent eg `~/.ssh/id_rsa`.
+ssh_username | The username to conncet. Will default to `ec2-user`
+sudo | true or false - will sudo the `docker` command on the target machine. Usually not needed unless the user is not a part of the `docker` group.
+aws_vault_profile | If you use the `aws-vault` tool to manage your AWS credentials, you can specify a profile here that will be automatically used to connect to this cluster.
+profile | Another profile to inherit settings from. Settings from lower profiles can be overridden in higher ones.
+
+
+## Maintainer
+[Envato](https://github.com/envato)
+
+## Contributors
+- [Peter Hofmann](https://github.com/envatopoho)
+- [Giancarlo Salamanca](https://github.com/salamagd)
+- [Jiexin Huang](https://github.com/jiexinhuang)
+
+## License
+
+`KnuckleCluster` uses MIT license. See
+[`LICENSE.txt`](https://github.com/envato/knuckle_cluster/blob/master/LICENSE.txt) for
+details.
+
+## Code of conduct
+
+We welcome contribution from everyone. Read more about it in
+[`CODE_OF_CONDUCT.md`](https://github.com/envato/knuckle_cluster/blob/master/CODE_OF_CONDUCT.md)
+
+## Contributing
+
+For bug fixes, documentation changes, and small features:
+
+1. Fork it ( https://github.com/[my-github-username]/knuckle_cluster/fork )
+2. Create your feature branch (git checkout -b my-new-feature)
+3. Commit your changes (git commit -am 'Add some feature')
+4. Push to the branch (git push origin my-new-feature)
+5. Create a new Pull Request
+
+For larger new features: Do everything as above, but first also make contact with the project maintainers to be sure your change fits with the project direction and you won't be wasting effort going in the wrong direction.

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "knuckle_cluster"
+
+require "irb"
+IRB.start(__FILE__)

data/bin/knuckle_cluster

@@ -0,0 +1,45 @@
+#!/usr/bin/env bundle exec ruby
+require "knuckle_cluster"
+require "yaml"
+
+profile = ARGV[0]
+
+if ARGV.count < 2
+  puts <<~USAGE
+    knuckle_cluster CLUSTER_PROFILE agents - list all agents and select one to start a shell
+    knuckle_cluster CLUSTER_PROFILE containers - list all containers and select one to start a shell
+    knuckle_cluster CLUSTER_PROFILE logs CONTAINER_NAME - tail the logs for a container
+    knuckle_cluster CLUSTER_PROFILE CONTAINER_NAME [OPTIONAL COMMANDS] - connect to a container and start a shell or run a command
+    knuckle_cluster CLUSTER_PROFILE SHORTCUT_NAME - run a shortcut defined in your knuckle_cluster configuration
+    knuckle_cluster CLUSTER_PROFILE tunnel TUNNEL_NAME - open a tunnel defined in your knuckle_cluster configuration
+  USAGE
+  exit
+end
+
+begin
+  config = KnuckleCluster::Configuration.load_parameters(profile: profile)
+rescue => e
+  puts "ERROR: There was a problem loading your configuration: #{e}"
+  exit
+end
+
+kc = KnuckleCluster.new(
+  config
+)
+
+if ARGV[1] == 'agents'
+  kc.connect_to_agents
+elsif ARGV[1] == 'containers'
+  kc.connect_to_containers
+elsif ARGV[1] == 'logs'
+  kc.container_logs(name: ARGV[2])
+elsif ARGV[1] == 'tunnel'
+  kc.open_tunnel(name: ARGV[2])
+else
+  command = ARGV.drop(2)
+  if command.any?
+    kc.connect_to_container(name: ARGV[1], command: command.join(' '))
+  else
+    kc.connect_to_container(name: ARGV[1])
+  end
+end

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/knuckle_cluster.gemspec

@@ -0,0 +1,31 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'knuckle_cluster/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "knuckle_cluster"
+  spec.version       = KnuckleCluster::VERSION
+  spec.authors       = ["Envato"]
+  spec.email         = ["rubygems@envato.com"]
+
+  spec.summary       = %q{Handy cluster tool}
+  spec.description   = %q{Interrogation of AWS ECS clusters, with the ability to directly connect to hosts and containers.}
+  spec.homepage      = "https://github.com/envato/knuckle_cluster"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.14"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+
+  spec.add_dependency 'aws-sdk', '~> 2.8'
+  spec.add_dependency 'table_print', '~> 1.5'
+
+  spec.bindir      = "bin"
+  spec.executables = 'knuckle_cluster'
+end

data/lib/knuckle_cluster/agent.rb

@@ -0,0 +1,28 @@
+module KnuckleCluster
+  class Agent
+    def initialize(
+      index:,
+      instance_id:,
+      public_ip:,
+      private_ip:,
+      availability_zone:,
+      container_instance_arn:,
+      task_registry:
+    )
+      @index = index
+      @instance_id = instance_id
+      @public_ip = public_ip
+      @private_ip = private_ip
+      @availability_zone = availability_zone
+      @container_instance_arn = container_instance_arn
+      @task_registry = task_registry
+    end
+
+    attr_reader :index, :instance_id, :public_ip, :private_ip,
+                :availability_zone, :container_instance_arn, :task_registry
+
+    def tasks
+      task_registry.where(container_instance_arn: container_instance_arn)
+    end
+  end
+end

data/lib/knuckle_cluster/agent_registry.rb

@@ -0,0 +1,75 @@
+require 'knuckle_cluster/agent'
+require 'knuckle_cluster/task_registry'
+
+require 'forwardable'
+
+module KnuckleCluster
+  class AgentRegistry
+    extend Forwardable
+
+    def initialize(aws_client_config:, cluster_name:)
+      @aws_client_config = aws_client_config
+      @cluster_name = cluster_name
+    end
+
+    def agents
+      @agents ||= load_agents
+    end
+
+    def find_by(container_instance_arn:)
+      agents_by_container_instance_arn[container_instance_arn]&.first
+    end
+
+    def_delegators :task_registry, :tasks, :containers
+
+    private
+
+    attr_reader :aws_client_config, :cluster_name
+
+    def load_agents
+      container_instance_arns = ecs_client.list_container_instances(cluster: cluster_name)
+                                   .container_instance_arns
+      return [] if container_instance_arns.empty?
+
+      ecs_instances_by_id = ecs_client.describe_container_instances(
+        cluster:             cluster_name,
+        container_instances: container_instance_arns,
+      ).container_instances.group_by(&:ec2_instance_id)
+
+      ec2_instance_reservations = ec2_client.describe_instances(instance_ids: ecs_instances_by_id.keys)
+                                            .reservations
+
+      ec2_instance_reservations.map(&:instances).flatten.map.with_index do |instance, index|
+        Agent.new(
+          index:                  index + 1,
+          instance_id:            instance[:instance_id],
+          public_ip:              instance[:public_ip_address],
+          private_ip:             instance[:private_ip_address],
+          availability_zone:      instance.dig(:placement, :availability_zone),
+          container_instance_arn: ecs_instances_by_id[instance[:instance_id]].first.container_instance_arn,
+          task_registry:          task_registry,
+        )
+      end
+    end
+
+    def agents_by_container_instance_arn
+      @agents_by_container_instance_arn ||= agents.group_by(&:container_instance_arn)
+    end
+
+    def task_registry
+      @task_registry ||= TaskRegistry.new(
+        ecs_client:     ecs_client,
+        cluster_name:   cluster_name,
+        agent_registry: self,
+      )
+    end
+
+    def ec2_client
+      @ec2_client ||= Aws::EC2::Client.new(aws_client_config)
+    end
+
+    def ecs_client
+      @ecs_client ||= Aws::ECS::Client.new(aws_client_config)
+    end
+  end
+end

data/lib/knuckle_cluster/configuration.rb

@@ -0,0 +1,52 @@
+class KnuckleCluster::Configuration
+  require 'yaml'
+  DEFAULT_PROFILE_FILE = File.join(ENV['HOME'],'.ssh/knuckle_cluster').freeze
+
+  def self.load_parameters(profile:, profile_file: nil)
+    profile_file ||= DEFAULT_PROFILE_FILE
+    raise "File #{profile_file} not found" unless File.exists?(profile_file)
+
+    data = YAML.load_file(profile_file)
+
+    unless data.keys.include?(profile)
+      raise "Config file does not include profile for #{profile}"
+    end
+
+    #Figure out all the profiles to inherit from
+    tmp_data = data[profile]
+    profile_inheritance = [profile]
+    while(tmp_data && tmp_data.keys.include?('profile'))
+      profile_name = tmp_data['profile']
+      break if profile_inheritance.include? profile_name
+      profile_inheritance << profile_name
+      tmp_data = data[profile_name]
+    end
+
+    #Starting at the very lowest profile, build an options hash
+    output = {}
+    profile_inheritance.reverse.each do |prof|
+      output.merge!(data[prof] || {})
+    end
+
+    output.delete('profile')
+
+    keys_to_symbols(output)
+  end
+
+  private
+
+  def self.keys_to_symbols(data)
+    #Implemented here - beats including activesupport
+    return data unless Hash === data
+    ret = {}
+    data.each do |k,v|
+      if Hash === v
+        #Look, doesnt need to be recursive but WHY NOT?!?
+        ret[k.to_sym] = keys_to_symbols(v)
+      else
+        ret[k.to_sym] = v
+      end
+    end
+    ret
+  end
+end

data/lib/knuckle_cluster/container.rb

@@ -0,0 +1,15 @@
+module KnuckleCluster
+  class Container
+    def initialize(
+      index:,
+      name:,
+      task:
+    )
+      @index = index
+      @name = name
+      @task = task
+    end
+
+    attr_reader :index, :name, :task
+  end
+end

data/lib/knuckle_cluster/task.rb

@@ -0,0 +1,25 @@
+module KnuckleCluster
+  class Task
+    def initialize(
+      arn:,
+      container_instance_arn:,
+      agent:,
+      definition:,
+      name:,
+      task_registry:
+    )
+      @arn = arn
+      @container_instance_arn = container_instance_arn
+      @agent = agent
+      @definition = definition
+      @name = name
+      @task_registry = task_registry
+    end
+
+    attr_reader :arn, :container_instance_arn, :agent, :definition, :name, :task_registry
+
+    def containers
+      task_registry.containers_where(task: self)
+    end
+  end
+end

data/lib/knuckle_cluster/task_registry.rb

@@ -0,0 +1,73 @@
+require 'knuckle_cluster/task'
+require 'knuckle_cluster/container'
+
+module KnuckleCluster
+  class TaskRegistry
+    def initialize(ecs_client:, cluster_name:, agent_registry:)
+      @ecs_client = ecs_client
+      @cluster_name = cluster_name
+      @agent_registry = agent_registry
+    end
+
+    def tasks
+      @tasks ||= load_tasks
+    end
+
+    def containers
+      tasks && all_containers
+    end
+
+    def where(container_instance_arn:)
+      tasks_by_container_instance_arn[container_instance_arn]
+    end
+
+    def containers_where(task:)
+      containers_by_task[task]
+    end
+
+    private
+
+    attr_reader :ecs_client, :cluster_name, :agent_registry, :all_containers
+
+    def load_tasks
+      task_arns = ecs_client.list_tasks(cluster: cluster_name).task_arns
+      task_ids  = task_arns.map { |x| x[/.*\/(.*)/,1] }
+
+      return [] if task_ids.empty?
+
+      @all_containers = []
+      index = 0
+
+      ecs_client.describe_tasks(tasks: task_ids, cluster: cluster_name).tasks.flat_map do |task|
+        agent = agent_registry.find_by(container_instance_arn: task.container_instance_arn)
+
+        Task.new(
+          arn:                    task.task_arn,
+          container_instance_arn: task.container_instance_arn,
+          agent:                  agent,
+          definition:             task.task_definition_arn[/.*\/(.*):.*/,1],
+          name:                   task.task_definition_arn[/.*\/(.*):\d/,1],
+          task_registry:          self,
+        ).tap do |new_task|
+          task.containers.each do |container|
+            index += 1
+
+            all_containers << Container.new(
+              index: index,
+              name:  container.name,
+              task:  new_task,
+            )
+          end
+        end
+      end
+    end
+
+    def tasks_by_container_instance_arn
+      @tasks_by_container_instance_arn ||= tasks.group_by(&:container_instance_arn)
+    end
+
+    def containers_by_task
+      @containers_by_task ||= all_containers.group_by(&:task)
+    end
+  end
+end

data/lib/knuckle_cluster/version.rb

@@ -0,0 +1,3 @@
+module KnuckleCluster
+  VERSION = '1.0.0'
+end

data/lib/knuckle_cluster.rb

@@ -0,0 +1,196 @@
+require 'knuckle_cluster/agent_registry'
+require "knuckle_cluster/version"
+require "knuckle_cluster/configuration"
+
+require 'aws-sdk'
+require 'forwardable'
+require 'table_print'
+
+module KnuckleCluster
+  class << self
+    extend Forwardable
+
+    def new(
+        cluster_name:,
+        region: 'us-east-1',
+        bastion: nil,
+        rsa_key_location: nil,
+        ssh_username: 'ec2-user',
+        sudo: false,
+        aws_vault_profile: nil,
+        shortcuts: {},
+        tunnels: {})
+      @cluster_name      = cluster_name
+      @region            = region
+      @bastion           = bastion
+      @rsa_key_location  = rsa_key_location
+      @ssh_username      = ssh_username
+      @sudo              = sudo
+      @aws_vault_profile = aws_vault_profile
+      @shortcuts         = shortcuts
+      @tunnels           = tunnels
+      self
+    end
+
+    def connect_to_agents(command: nil, auto: false)
+      agent = select_agent(auto: auto)
+      run_command_in_agent(agent: agent, command: command)
+    end
+
+    def connect_to_containers(command: 'bash', auto: false)
+      container = select_container(auto: auto)
+      run_command_in_container(container: container, command: command)
+    end
+
+    def connect_to_container(name:, command: 'bash')
+      if shortcut = shortcuts[name.to_sym]
+        name = shortcut[:container]
+        new_command = shortcut[:command]
+        new_command += " #{command}" unless command == 'bash'
+        command = new_command
+      end
+
+      container = find_container(name: name)
+      run_command_in_container(container: container, command: command)
+    end
+
+    def container_logs(name:)
+      container = find_container(name: name)
+      subcommand = "#{'sudo ' if sudo}docker logs -f \\`#{get_container_id_command(container.name)}\\`"
+      run_command_in_agent(agent: container.task.agent, command: subcommand)
+    end
+
+    def open_tunnel(name:)
+      if tunnel = tunnels[name.to_sym]
+        agent = select_agent(auto: true)
+        open_tunnel_via_agent(tunnel.merge(agent: agent))
+      else
+        puts "ERROR: A tunnel configuration was not found for '#{name}'"
+      end
+    end
+
+    private
+
+    attr_reader :cluster_name, :region, :bastion, :rsa_key_location, :ssh_username,
+                :sudo, :aws_vault_profile, :shortcuts, :tunnels
+
+    def select_agent(auto:)
+      return agents.first if auto
+
+      puts "\nListing Agents"
+
+      tp agents,
+         :index,
+         :instance_id,
+         # :public_ip,
+         # :private_ip,
+         # :availability_zone,
+         { task: { display_method: 'tasks.name', width: 999 } },
+         { container: { display_method: 'tasks.containers.name', width: 999 } }
+
+      puts "\nConnect to which agent?"
+      agents[STDIN.gets.strip.to_i - 1]
+    end
+
+    def select_container(auto:)
+      return containers.first if auto
+
+      puts "\nListing Containers"
+
+      tp tasks,
+         { task: { display_method: :name, width: 999 } },
+         { agent: { display_method: 'agent.instance_id' } },
+         { index: { display_method: 'containers.index' } },
+         { container: { display_method: 'containers.name', width: 999 } }
+
+      puts "\nConnect to which container?"
+      containers[STDIN.gets.strip.to_i - 1]
+    end
+
+    def find_container(name:)
+      matching = containers.select { |container| container.name.include?(name) }
+      puts "\nAttempting to find a container matching '#{name}'..."
+
+      if matching.empty?
+        puts "No container with a name matching '#{name}' was found"
+        Process.exit
+      end
+
+      unique_names = matching.map(&:name).uniq
+
+      if unique_names.uniq.count > 1
+        puts "Containers with the following names were found, please be more specific:"
+        puts unique_names
+        Process.exit
+      end
+
+      # If there are multiple containers with the same name, choose any one
+      container = matching.first
+      puts "Found container #{container.name} on #{container.task.agent.instance_id}\n\n"
+      container
+    end
+
+    def run_command_in_container(container:, command:)
+      subcommand = "#{'sudo ' if sudo}docker exec -it \\`#{get_container_id_command(container.name)}\\` #{command}"
+      run_command_in_agent(agent: container.task.agent, command: subcommand)
+    end
+
+    def get_container_id_command(container_name)
+      "#{'sudo ' if sudo}docker ps --filter 'label=com.amazonaws.ecs.container-name=#{container_name}' | tail -1 | awk '{print \\$1}'"
+    end
+
+    def run_command_in_agent(agent:, command:)
+      command = generate_connection_string(agent: agent, subcommand: command)
+      system(command)
+    end
+
+    def open_tunnel_via_agent(agent:, local_port:, remote_host:, remote_port:)
+      command = generate_connection_string(
+        agent: agent,
+        port_forward: [local_port, remote_host, remote_port].join(':'),
+        subcommand: <<~SCRIPT
+          echo ""
+          echo "localhost:#{local_port} is now tunneled to #{remote_host}:#{remote_port}"
+          echo "Press Enter to close the tunnel once you are finished."
+          read
+        SCRIPT
+      )
+      system(command)
+    end
+
+    def aws_client_config
+      @aws_client_config ||= { region: region }.tap do |config|
+        config.merge!(aws_vault_credentials) if aws_vault_profile
+      end
+    end
+
+    def aws_vault_credentials
+      environment = `aws-vault exec #{aws_vault_profile} -- env | grep AWS_`
+      vars = environment.split.map { |pair| pair.split('=') }.group_by(&:first)
+      {}.tap do |credentials|
+        %i{access_key_id secret_access_key session_token}.map do |var_name|
+          credentials[var_name] = vars["AWS_#{var_name.upcase}"]&.first&.last
+        end
+      end
+    end
+
+    def generate_connection_string(agent:, subcommand: nil, port_forward: nil)
+      ip = bastion ? agent.private_ip : agent.public_ip
+      command = "ssh #{ip} -l#{ssh_username}"
+      command += " -i #{rsa_key_location}" if rsa_key_location
+      command += " -o ProxyCommand='ssh -qxT #{bastion} nc #{ip} 22'" if bastion
+      command += " -L #{port_forward}" if port_forward
+      command += " -t \"#{subcommand}\"" if subcommand
+      command
+    end
+
+    def agent_registry
+      @agent_registry ||= AgentRegistry.new(
+        aws_client_config: aws_client_config,
+        cluster_name:      cluster_name,
+      )
+    end
+
+    def_delegators :agent_registry, :agents, :tasks, :containers
+  end
+end

metadata

@@ -0,0 +1,136 @@
+--- !ruby/object:Gem::Specification
+name: knuckle_cluster
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Envato
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2017-09-18 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.14'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.14'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: aws-sdk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.8'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.8'
+- !ruby/object:Gem::Dependency
+  name: table_print
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+description: Interrogation of AWS ECS clusters, with the ability to directly connect
+  to hosts and containers.
+email:
+- rubygems@envato.com
+executables:
+- knuckle_cluster
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".travis.yml"
+- CODE_OF_CONDUCT.md
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/knuckle_cluster
+- bin/setup
+- knuckle_cluster.gemspec
+- lib/knuckle_cluster.rb
+- lib/knuckle_cluster/agent.rb
+- lib/knuckle_cluster/agent_registry.rb
+- lib/knuckle_cluster/configuration.rb
+- lib/knuckle_cluster/container.rb
+- lib/knuckle_cluster/task.rb
+- lib/knuckle_cluster/task_registry.rb
+- lib/knuckle_cluster/version.rb
+homepage: https://github.com/envato/knuckle_cluster
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.8
+signing_key: 
+specification_version: 4
+summary: Handy cluster tool
+test_files: []