bub_bot 0.2.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: fcd0e50d99abae7189edd2f1f6ae9c7b12631784
+  data.tar.gz: ab061acc31daff26688c897240d3808e1ae389fc
+SHA512:
+  metadata.gz: 91b56fd6e73b9c1fb7793a289c72a1bccd49eae9e884be248626040198c6933aa81cbff293ebda8ae3dfa553110619f4a3a6424b8e5282684a6da1c02c5ea8e4
+  data.tar.gz: a352763574e9182999139871426a202cbc28dab42d83d3ecc94243f48216ec86081327ee131dda544148bd6029ab4081278f204299a889b54b24fd88f3e63a39

data/.gitignore

@@ -0,0 +1,11 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+test_config.yml
+deploy_web.sh

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.1
+before_install: gem install bundler -v 1.13.6

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at kevin@kevinkuchta.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in bub_bot.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Kevin Kuchta
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,45 @@
+# Dev instructions
+
+These are just notes to myself.  //TODO: rewrite this section so it's useful to people who are not me.
+
+Use ec2tunnel alias to set up a tunnel (it's already rigged up to slack on the ktest team)
+`redis-server --save "" --appendonly no`
+`bundle exec bin/bub_bot test_config.yml`
+
+Slack setup:
+- Creat an app
+- Add a bot user
+- Add an event subscription
+  - Use the *bot event* for messages.channel
+- Add an incoming webhook
+<!--- Add event subscription-->
+  <!--- Enter hostname (verification should happen automatically now)-->
+  <!--- Add messages.channel permission-->
+- More notes on this in test_config.yml.  Before I ship all this I'll need to test out the slack setup instructions.
+
+# Questions
+- What's the syntax for deploying different branches to different targets?
+  `bub deploy burrito, kk_web_change to web, kk_core_change to core` maybe?  Ideally something simpler.
+  - We'll want a shorthand for deploying the same branch to both (`bub deploy burrito kk_small_fixes to both`). Or maybe not.
+  - If you leave one off, does it deploy develop, or maybe it doesn't deploy to that branch at all?  eg `bub deploy kk_small_fixes to web`
+
+# Deploy
+- Deploy implicitly takes the server
+- `bub deploy cannoli kk_some_change` deploys that branch to both servers on cannoli
+  - if that branch is found in both core and web, deploy it
+  - if it's found in only one, deploy it there and deploy develop in the other
+- `bub deploy cannoli core kk_foo web kk_bar`
+  - deploy branches to servers
+  - try to accept 'kk_foo to core', 'kk_foo on core', 'core kk_foo'
+  - ignore commas and 'and'
+- `bub deploy cannoli core kk_foo`
+  - if only one target specified, I guess deploy default branch to the other
+  - I suppose we want to always deploy to all systems so it's not in a surprising state
+    - Maybe support `none` or `skip` as a branch if we want to explicitly skip deploying to it.
+- `bub deploy core kk_foo`
+  - Takes and deploys to the first available server
+  - We *could* have this deploy to any server you already have claimed instead, I guess
+    - Seems like that could be surprising (in a bad way)
+    - Maybe only do that if you have only one server claimed?
+    - I dunno, still seems kinda sketch
+

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/bub_bot

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require 'bub_bot'
+
+BubBot::CLI.new(ARGV).start

data/braindump.md

@@ -0,0 +1,28 @@
+This is a running log of whatever I was thinking when I stopped work for the day.
+
+Brain dump for the next time I pick this up:
+- Ok, `take` works.  There's a nice litle parsing system for flexible command parsing.
+- We've got a custom error class that, if thrown, prints the error message to slack too.
+- Next up: rubocop + some cleanup, then start in on the deploy command.
+ - Deploy should be largely configured by the config mechanisms
+  - Although there should be helpers for common deploy strategies (eg git pushing and/or running a script)
+- Ok, I described the deploy command in the readme.  I also put together an example config in test_config.yml for deploys.  Now all you have to do is actually implement it.
+
+
+- Ok, I've written most of the deploy command's parsing.  It probably works, but it'll need testing.  Also needs more error handling.  See the TODOs in the command file.
+- Next up is implementing the git logic.
+  - Old bub implemented deploy by just cloning, then pushing, then deleting it all.
+  - New bub needs to keep a persistent git repo around, if only so we can get a list of branch names to check commands against.
+  - Might need a repo manager class to keep track of the state of those things.
+
+I wrote a bunch of git integration stuff, but it turns out you can get a list of remote branches _without_ cloning the rep!  So most of my work today isn't needed yet (although it might be for deploying).
+  - If I want to, I can do the old clone->deploy->delete cycle for deploys.
+  - Alternately, I can try to keep the repo around between deploys to speed things up.
+
+So I ended up using most of that git stuff anyway.  This is because... git deploy now works!  Woo!  We're even reusing repos!  Todo next:
+- Clean up the deploy command (it needs to support more things, like "take latest" and "claim whatever we're deploying to").
+- Make sure we're only deploying to one repo at a time.  We _could_ use nora's solution of just cloning a separate copy of the repo for each deploy, but I'd prefer to be able to reuse repos.
+  - Actually, maybe this isn't much of an issue?  We only need to keep a lock on a repo while we're getting set up for a deploy.  Once git push starts, we don't care that much about what happens to the filesystem.
+- Implement script-based deploys (for web)
+- Implement before/after deploy hooks (eg for turning envs on and off when not in use)
+  - Wait, maybe this should be handled separately.

data/bub_bot.gemspec

@@ -0,0 +1,37 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'bub_bot/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "bub_bot"
+  spec.version       = BubBot::VERSION
+  spec.authors       = ["Kevin Kuchta"]
+  spec.email         = ["kevin@kevinkuchta.com"]
+
+  spec.summary       = %q{A little server reservation and deploy bot for heroku and aptible}
+  spec.homepage      = "https://github.com/kkuchta/bub_bot"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.bindir        = "bin"
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "rack", "~> 1.0"
+  spec.add_dependency "thin", "~> 1.0"
+  spec.add_dependency "git"
+  spec.add_dependency "activesupport"
+  spec.add_dependency "faraday"
+  spec.add_dependency 'actionview'
+  spec.add_dependency "redis"
+  spec.add_dependency 'slack-ruby-client'
+
+  spec.add_development_dependency "bundler", "~> 1.13"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.add_development_dependency "rspec-mocks", "~> 3.5"
+  spec.add_development_dependency "pry-byebug"
+end

data/example_config.yml

@@ -0,0 +1,56 @@
+# Ok, slack's apis + integrations are a big mess.  This was much simpler a few
+# years ago.  Some learnings:
+#
+# - Distributing as a slack app isn't going to work.  That seems built around
+#   externally-hosted servers.  You get the web-api credentials during this
+#   oauth-like flow that happens when a team clicks the "add app" button.
+# - Wait!  No!  In the app config, under "Add features + functionality"
+#   > Permissions > Bot User OAuth Access Token, there's a bot user oauth token
+#   that seems to work for accessing the web api.
+#
+
+redis_host: localhost:6379
+
+bot_name: bub
+
+# Rack options
+Port: 65222
+
+# Bot oauth token
+bot_oauth_token: abcd-123456
+
+slack_channel: general
+
+# H'ok, so:
+# deploy_targets lists each codebase.  Deploy defines how to deploy.  Any string
+# in here will get run through erb.  Some parameters that will be provided to erb
+# are:
+#  - server
+#  - branch
+#
+# deploy types are:
+#  - script (just some local script to run; should be in git repo?)
+#  - git (a git remote to push to)
+#  - http (some web callback to hit)
+
+deploy_targets:
+  core:
+    git: 'git@github.com:MyCo/foo.git'
+    deploy:
+      git: 'git@beta.aptible.com:myco-<%=server %>/foo-rails.git'
+    #default_branch: develop - TODO not yet supported
+  web:
+    git: 'git@github.com:MyCo/web.git'
+
+    # Script params: repo_dir, branch, server to deploy to
+    # (repo should already be on the right branch)
+    deploy:
+      script: 'deploy_web.sh'
+    #default_branch: develop - TODO not yet supported
+
+servers:
+  #- cannoli
+  #- hotdog
+  - burrito
+  - gyro
+

data/lib/bub_bot/cli.rb

@@ -0,0 +1,42 @@
+require 'yaml'
+
+class BubBot::CLI
+  DEFAULT_CONFIG_FILENAME = 'bub_bot.yml'
+  def initialize(args)
+    @args = args
+
+    @filename = args[0] || DEFAULT_CONFIG_FILENAME
+  end
+
+  def start
+    print_usage && return unless check_usage
+
+    configure_from_file(@filename)
+
+    BubBot.start
+  end
+
+  private
+
+  def configure_from_file(filename)
+    file_data = YAML.load_file(filename)
+
+    BubBot.configure do |config|
+      BubBot::Configuration::OPTIONS.each do |option_name|
+        config.public_send((option_name.to_s + '=').to_sym, file_data[option_name.to_s])
+      end
+    end
+  end
+
+  def check_usage
+    @args.count <= 1
+  end
+
+  def print_usage
+    puts <<USAGE
+    Usage: bub_bot [config_filename]
+
+    Default to #{DEFAULT_CONFIG_FILENAME} if config_filename is not provided
+USAGE
+  end
+end

data/lib/bub_bot/configuration.rb

@@ -0,0 +1,34 @@
+# TODO: merge with some defaults
+class BubBot::Configuration
+  # Any of these options will be passed on to rack, rather than handled by us.
+  RACK_OPTIONS = %i(
+    Port
+  )
+
+  # These options will be handled by us.
+  BUB_BOT_OPTIONS = %i(
+    servers
+    bot_oauth_token
+    bot_name
+    deploy_targets
+    slack_channel
+    redis_host
+  )
+
+  OPTIONS = RACK_OPTIONS + BUB_BOT_OPTIONS
+
+  attr_accessor *OPTIONS
+
+  def rack_options_hash
+    RACK_OPTIONS.each_with_object({}) do |option_name, options_hash|
+      options_hash[option_name] = public_send(option_name)
+    end
+  end
+
+  def verify_options
+    # TODO: verify that deploy_targets, etc, are formatted correctly and print
+    # useful error messages otherwise.
+    true
+  end
+
+end

data/lib/bub_bot/deploy_manager.rb

@@ -0,0 +1,131 @@
+require 'bub_bot/repo'
+require 'bub_bot/redis_connection'
+require 'erb'
+
+class BubBot::DeployManager
+  def deploy(server, target_name, branch)
+    if DeployState[server, target_name].deploying?
+      raise RespondableError.new("A deploy to #{target_name} on #{server} is already in progress.  Not deploying.")
+    end
+
+    begin
+      DeployState[server, target_name].set(true)
+      target_config = target(target_name)
+
+      unless deploy_config = target_config['deploy']
+        raise "Missing deploy config for #{target_name}"
+      end
+
+      locals = {
+        server: server
+      }
+
+      # Handle each type of deploy here.
+      # TODO: maybe handle multiple deploys for each target?  Right now the
+      # workaround to do that is to just have a script-type deploy that does that.
+      if deploy_git_remote = deploy_config['git']
+        deploy_git_remote = ERB.new(deploy_git_remote).result(get_binding(locals))
+        repo(target_name, server).push(branch, deploy_git_remote)
+      elsif deploy_script = deploy_config['script']
+        puts "xdeploying web script #{deploy_script}"
+        repo = repo(target_name, server)
+        puts "Checking out..."
+        repo.checkout(branch)
+        puts "Pulling..."
+        repo.pull
+        puts "Running script..."
+        success = Kernel.system("./#{deploy_script} #{repo.repo_dir} #{branch} #{server}")
+        puts "Success = #{success}"
+        unless success
+          raise RespondableError.new("Deploy script failed for server #{server} and target #{target_name}")
+        end
+      elsif deploy_url = deploy_config['http']
+        raise RespondableError.new('Sorry, deploys by http request are not supported yet')
+      end
+
+      DeployState[server, target_name].set(false)
+    rescue
+      DeployState[server, target_name].set(false)
+      raise
+    end
+  end
+
+  def target_names
+    targets.keys - ['all']
+  end
+
+  def branches(target_name)
+    Repo.branches(target(target_name)['git'])
+  end
+
+  private
+
+  def repo(target_name, server)
+    @repos ||= {}
+    target = target(target_name)
+    @repos[target_name + '__' + server] ||= Repo.new(target_name, target['git'], server)
+  end
+
+  # Returns a hash of config data for the target with this name
+  def target(target_name)
+    targets.find { |name, _| name == target_name }.last
+  end
+
+  def targets
+    BubBot.configuration.deploy_targets
+  end
+
+  # Gets a binding object with the given variables defined in it.  You'd *think*
+  # there'd be a simpler way.  Well, ok, there is, but there's no simpler way that
+  # doesn't *also* polute the binding with variables from the outer scope.
+  def get_binding(variables)
+    obj = Class.new {
+      attr_accessor *variables.keys
+      def get_binding(); binding end
+    }.new
+    variables.each { |name, value| obj.public_send(:"#{name}=", value) }
+    obj.get_binding
+  end
+end
+
+class DeployState
+  ROOT_KEY = 'bub_deploy_status'.freeze
+
+  def self.[](server, target)
+    (@_deploy_states ||= {})[key(server, target)] ||= DeployState.new(server, target)
+  end
+
+  def self.key(server, target)
+    "#{server}__#{target}"
+  end
+
+  def initialize(server, target)
+    @server = server
+    @target = target
+  end
+
+  def key
+    self.class.key(@server, @target)
+  end
+
+  def deploying?
+    deployed_at = redis.hget(ROOT_KEY, key)
+
+    # If we have a super-old deployed_at, assume something went wrong in the
+    # deploy and we failed to capture that.
+    return deployed_at && Time.parse(deployed_at) > 30.minutes.ago
+  end
+
+  def set(is_deploying)
+    puts "set deploying to #{is_deploying} for #{key}"
+    if is_deploying
+      redis.hset(ROOT_KEY, key, Time.now)
+    else
+      redis.hdel(ROOT_KEY, key)
+    end
+  end
+
+  def redis
+    BubBot::RedisConnection.instance
+  end
+end

data/lib/bub_bot/redis_connection.rb

@@ -0,0 +1,16 @@
+require 'redis'
+require 'singleton'
+
+# Just a simple wrapper around a redis connection.  Call .instance on this, then
+# call anything you'd normally call on a Redis on object on this instead.  Eg
+# `RedisConnection.instance.hgetall(...)`.
+class BubBot::RedisConnection
+  include Singleton
+
+  def method_missing(method, *args, &block)
+    redis.respond_to?(method) ? redis.public_send(method, *args, &block) : super
+  end
+  def redis
+    @redis ||= Redis.new(url: 'redis://' + BubBot.configuration.redis_host)
+  end
+end

data/lib/bub_bot/repo.rb

@@ -0,0 +1,115 @@
+require 'git'
+
+class Repo
+  # When we create a temproary remote to push to, use this name.
+  PUSH_REMOTE_NAME = 'push_remote'
+
+  def initialize(target, origin_remote, server)
+    puts "target = #{target}"
+    puts "remote = #{origin_remote}"
+    puts "server = #{server}"
+    @target = target
+    @origin_remote = origin_remote
+    @server = server
+  end
+
+  # Gets a list of branch names without actually cloning the repo.
+  def self.branches(origin_remote)
+    puts 'repo.branches'
+    # git ls-remote returns a list of lines like this:
+    #   ea656e141760b0d8ba49d92506427322120ce945	refs/heads/some-branch-name
+    ls_remote_output = `git ls-remote --heads #{origin_remote}`
+    ls_remote_output.split("\n").map {|line| line.split('/').last }
+  end
+
+  def git
+    return @_git if @_git
+
+    if Dir.exists?(repo_dir) && Dir.exists?(repo_dir + '/.git')
+      puts "Repo exists!"
+      @_git = Git.open(repo_dir)
+      # TODO: handle other errors beside "dir doesn't exist"
+    else
+      puts "Cloning repo"
+      @_git = Git.clone(@origin_remote, repo_dir_name, path: dir, depth: 1, :log => Logger.new(STDOUT))
+      puts 'here'
+    end
+    @_git
+  end
+
+  # Push is async
+  def push(branch, remote, &on_complete)
+    # TODO: handle people pushing while another push is still going, since we
+    # have some shared state in the form of the filesystem
+
+    puts 'Pushing'
+    git.remotes.find{ |remote| remote.name == 'push_remote' }&.remove
+    puts 'Maybe removed old remote'
+
+    git.add_remote('push_remote', remote)
+    puts 'added remote'
+
+    Kernel.system("cd #{repo_dir}; git remote set-branches --add origin '#{branch}'")
+    puts "Pushing #{branch} to #{remote}"
+
+    puts "about to git fetch origin for branch #{branch}"
+    git.fetch('origin', branch: branch)
+    puts 'about to finally push'
+    git.push(remote, "+origin/#{branch}:master")
+    puts 'Finished final push'
+    on_complete.call if on_complete
+  end
+
+  def clean
+    # TODO: make sure there's no weird changes on the git repo
+    git.clean(force: true)
+    git.reset_hard
+  end
+
+  def fetch
+    git.fetch
+  end
+
+  def checkout(branch_name)
+    clean
+    cmd("remote set-branches origin '*'")
+    fetch
+    git.checkout(branch_name)
+  end
+
+  def pull
+    git.pull
+  end
+
+  # We name repo dirs after server + name (eg `burrito__core`).  This lets
+  # multiple deploys to the same server (eg deploying both core and web to burrito
+  # at the same time) to work, as well as multiple deploys to the same target
+  # (eg deploying core on both burrito and gyro) to work.
+  #
+  # Note that simultanious deploys to the same target _and_ server (eg two deploys
+  # at once to burrito__core) won't work.  Both deploys would use the same git
+  # working directory and step on eachother's toes.  That's fine, though, because
+  # they'd _also_ step on eachother's toes in the actual target environment.  We
+  # should prevent this case in the UI.
+  def repo_dir
+    "#{dir}/#{repo_dir_name}"
+  end
+
+  def repo_dir_name
+    "#{@server}__#{@target}"
+  end
+
+  def dir
+    "/tmp/bub_repos"
+  end
+
+  private
+
+  # The git library doesn't support everything, so sometimes we run arbitrary commands.
+  def cmd(command)
+    git_command = "git --git-dir=#{repo_dir}/.git --work-tree=#{repo_dir} #{command}"
+    puts "Running #{git_command}"
+    result = Kernel.system(git_command)
+    puts "Result=#{result}"
+  end
+end