buckler 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: db7db01a0faa13f5047d6248d3fe5ea5c64da8e2
+  data.tar.gz: 97a290a47b2fc14be5ae66fef9f1da37fcb6ae47
+SHA512:
+  metadata.gz: b7e85bf75a278d549dfbe63aa2f948182ef1dde3cc2e092a8874bf00b02aa97d693d476dbf8dc0b553e2739e6087fe792ad90786a8c507281c184912f93559c7
+  data.tar.gz: 39ea505766afccb930f0b8aad05293cf8a82da2df088172fc6597910d589fb339e80b6c160f82eb3a3b18b2793aa3508d97d818d69e52441f7b2b30f4de8d8e7

data/.gitignore

@@ -0,0 +1,66 @@
+# Special ignores for this project
+
+Gemfile.lock
+
+# Ignore local dev only files
+
+*.dump
+*.gem
+*.log
+*.rbc
+*.cache
+*.pid
+*.sqlite3
+*.sqlite3-journal
+.bundle
+.config
+.env
+.env.test
+.yardoc
+config/database.yml
+coverage
+export
+InstalledFiles
+lib/bundler/man
+node_modules
+pkg
+public/system/**/*
+rdoc
+doc
+rerun.txt
+spec/reports
+spec/tmp/**/*
+tags
+test/tmp
+test/version_tmp
+vendor/**/*
+vim/.netrwhist
+zeus.json
+_yardoc
+
+# Ignore compilation junk
+
+**.orig
+*.rbc
+*.sassc
+*.sw[nop]
+.rspec
+.sass-cache
+capybara-*.html
+pickle-email-*.html
+rerun.txt
+
+# Ignore filesystem trash
+
+*.DS_Store
+*Thumbs.db
+.DS_Store?
+ehthumbs.db
+Thumbs.db
+
+# Specifically DO NOT ignore these repo settings
+
+!.gitignore
+!.slugignore
+!.ruby-version
+!.yardops

data/.ruby-version

@@ -0,0 +1 @@
+2.3.1

data/.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+rvm:
+  - 2.2.3
+  - 2.2.4
+  - 2.2.5
+  - 2.3.0
+  - 2.3.1
+script: "rake test"

data/.yardopts

@@ -0,0 +1,4 @@
+--readme README.md
+--charset utf-8
+--markup markdown
+--exclude /test

data/CONTRIBUTING.md

@@ -0,0 +1,29 @@
+This project welcomes contributions. While contributing, you must adhere to the project [code of conduct](#the-code-of-conduct).
+
+Here’s how you can make Buckler better:
+
+  - [Report an issue](https://github.com/csuhta/buckler/issues/new) or ask questions
+  - [Suggest new features](https://github.com/threespot/csuhta/buckler/new)
+  - Check, vet, diagnose, or add details to existing issues.
+  - Improve documentation. There are probably still typos or unclear instructions. These are easy wins.
+  - Submit pull requests or bugfixes.
+
+Thanks for your help!
+
+### The Code of Conduct
+
+This project strongly values contributors from anywhere, regardless of gender, gender expression, sexual orientation, ability, physical appearance, body type, race, age, or religion. As a result, the maintainers have agreed to and enforces this code of conduct in order to provide a harassment-free experience for everyone who participates in the development of Buckler.
+
+Harassment includes offensive comments related to gender, gender expression, sexual orientation, ability, physical appearance, body type, race, age, or religion. It also includes the posting of sexual images, deliberate intimidation, stalking, and unwelcome sexual attention.
+
+Further, harassment includes ad-hominem attacks, grandstanding, derogatory comments, trolling, public or private community disruption, insults, or other unprofessional conduct.
+
+Harassment is not tolerated in the issue tracker, code, events, chat channels, over emails, or any other way you can communicate with a contributor. Anyone who is asked to cease harassment is expected to comply immediately. Maintainers are not exempt from this policy.
+
+If anyone engages in harassing behavior, maintainers may take appropriate action, up to and including warning the offender, deletion of comments, removal from the project’s codebase and communication systems, and escalation to GitHub support.
+
+If you are being harassed, notice that someone else is being harassed, or have any other concerns, please open an issue or contact a maintainer.
+
+### License
+
+This document is published from the United States. To the extent possible under law, Corey Csuhta and the Buckler contributors have waived all copyright and related or neighboring rights to this document worldwide. The text in this file may be re-used for any purpose and without credit. ✨

data/Gemfile

@@ -0,0 +1,2 @@
+source "https://rubygems.org"
+gemspec

data/LICENSE.md

@@ -0,0 +1,9 @@
+### The MIT License
+
+Copyright (c) Corey Csuhta
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,96 @@
+### Buckler
+
+[![Build Status](https://travis-ci.org/csuhta/buckler.svg?branch=master)](https://travis-ci.org/csuhta/buckler)
+
+Buckler is a Ruby command line tool for performing common actions on Amazon S3 buckets. It’s more do-what-you-want and less overwhelmingly powerful than the AWS CLI. It’s also designed to work with Heroku applications.
+
+### Installing Buckler
+
+Buckler requires at least Ruby 2.2.3. Get Buckler from Rubygems. The terminal command is called `bucket`.
+
+```shell
+gem install buckler
+bucket help
+```
+
+### Command Reference
+
+```shell
+# Get a list of your buckets
+bucket list
+
+# Run any command with additional debuging info
+bucket list --verbose
+
+# Get a list of all S3 regions
+bucket regions
+
+# Create a new bucket on your account
+bucket create new-bucket-name
+
+# Create a new bucket on your account in a different region
+bucket create new-bucket-name --region eu-west-1
+
+# Remove all objects from one of your buckets
+bucket empty bucket-name
+
+# Destroy one of your buckets
+bucket destroy bucket-name
+
+# Copy the contents of one bucket into another
+# This command also conveniently copies
+# ACLs, headers, metadata, and a lot more.
+bucket sync source-bucket-name target-bucket-name
+
+# Get detailed help with Buckler commands
+bucket help
+bucket help sync
+bucket help create
+bucket help destroy
+bucket help empty
+bucket help list
+bucket help regions
+```
+
+### Providing Credentials
+
+You will need a AWS Access Key ID and AWS Secret Access Key pair with permission to mange your S3 buckets. **Do not use your root keys.** [Generate a new set of keys with S3 permissions only](http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#lock-away-credentials).
+
+When you run the `bucket` command, Buckler tries to automatically discover AWS credentials around your working directory.
+
+**Dotenv**: If the current folder has a file named `.env`, Buckler will look for variables called `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` in the file. [See Heroku’s documentation on this environment file format](https://devcenter.heroku.com/articles/heroku-local#set-up-your-local-environment-variables).
+
+**Heroku**: If the current folder has a Git repository with a Heroku remote, Buckler will ask your Heroku application for variables named `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` using `heroku config:get`
+
+**Command Line Options**: You can set the pair directly by providing the command line options `--id` and `--secret`
+
+```shell
+bucket list --id YOUR_AWS_ID --secret YOUR_AWS_SECRET
+```
+
+**Environment Variables**: You can set the pair directly as environment variables.
+
+```shell
+AWS_ACCESS_KEY_ID=your-id AWS_SECRET_ACCESS_KEY=your-secret bucket list
+```
+
+### Developing Buckler
+
+**The Buckler test suite will create and destroy test buckets on your AWS account which will cost you more than 0 money.** Existing buckets won’t be affected.
+
+```shell
+# Fork or clone Buckler
+git clone -o github git@github.com:csuhta/buckler.git
+cd buckler
+
+# Set your credentials in a .env
+echo "AWS_ACCESS_KEY_ID=your-id" >> .env
+echo "AWS_SECRET_ACCESS_KEY_ID=your-secret" >> .env
+
+# Work on stuff, then run the test suite
+rake test
+```
+
+### License
+
+Buckler is free software, and may be redistributed under the terms of the [MIT license](https://github.com/csuhta/bucklet/blob/master/LICENSE.md). Hope you like it! ❤️

data/Rakefile

@@ -0,0 +1,16 @@
+BUCKLER_ROOT = File.dirname(__FILE__)
+BUCKLER_EXECUTABLE = "#{BUCKLER_ROOT}/bin/bucket"
+
+desc "Open an irb session preloaded with this library"
+task :console do
+  exec "irb -rubygems -I lib -r buckler.rb"
+end
+
+desc "Run Buckler’s test suite"
+task :test do
+  require "./test/run"
+end
+
+# Creates the build/install/release tasks
+require "bundler/setup"
+Bundler::GemHelper.install_tasks

data/bin/bucket

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+require "buckler"
+include Buckler::Logging
+
+begin
+  Buckler::Commands::Root.run(ARGV)
+  exit true
+rescue Aws::S3::Errors::InvalidAccessKeyId
+  alert "Invalid AWS Access Key ID: #{Buckler.aws_access_key_id}"
+  exit false
+rescue Aws::S3::Errors::SignatureDoesNotMatch
+  alert "Invalid AWS Secret Access Key provided for Access Key ID #{Buckler.aws_access_key_id}"
+  exit false
+rescue Interrupt
+  alert "\nCommand aborted."
+  exit false
+end

data/buckler.gemspec

@@ -0,0 +1,31 @@
+$LOAD_PATH.push File.expand_path("../lib", __FILE__)
+require "buckler/version"
+
+Gem::Specification.new do |s|
+
+  s.name          = "buckler"
+  s.version       = Buckler::VERSION::STRING
+  s.platform      = Gem::Platform::RUBY
+  s.licenses      = ["MIT"]
+  s.authors       = ["Corey Csuhta"]
+  s.homepage      = "https://github.com/csuhta/buckler"
+  s.summary       = "Perform common actions on S3 buckets from the command line"
+  s.description   = "Buckler is a Ruby command line tool for performing common actions on Amazon S3 buckets. It’s more do-what-you-want and less overwhelmingly powerful than the AWS CLI. It’s also designed to work with Heroku applications."
+
+  s.executables   = ["bucket"]
+  s.files = `git ls-files`.split("\n")
+  s.test_files = `git ls-files --directory test`.split("\n")
+  s.require_paths = ["lib"]
+
+  s.required_ruby_version = ">= 2.2.3"
+
+  s.add_dependency "activesupport", "~> 5.0"
+  s.add_dependency "aws-sdk", "~> 2.0"
+  s.add_dependency "dotenv", "~> 2.1"
+  s.add_dependency "cri", "~> 2.7"
+
+  s.add_development_dependency "rake", "~> 11.2"
+  s.add_development_dependency "minitest", "~> 5.7"
+  s.add_development_dependency "yard", "~> 0.6"
+
+end

data/lib/buckler.rb

@@ -0,0 +1,29 @@
+require "mkmf"
+require "cri"
+require "dotenv"
+require "etc"
+require "active_support/core_ext/object"
+require "active_support/core_ext/string"
+require "active_support/core_ext/array"
+require "active_support/core_ext/enumerable"
+require "active_support/inflector"
+
+require "aws-sdk"
+
+require "buckler/version"
+require "buckler/strings"
+require "buckler/logging"
+require "buckler/thread_dispatch"
+require "buckler/monkey_patches"
+
+require "buckler/regions"
+require "buckler/aws"
+require "buckler/heroku"
+require "buckler/actions"
+require "buckler/actions/create_bucket"
+require "buckler/actions/destroy_bucket"
+require "buckler/actions/empty_bucket"
+require "buckler/actions/list_buckets"
+require "buckler/actions/list_regions"
+require "buckler/actions/sync_buckets"
+require "buckler/commands"

data/lib/buckler/actions.rb

@@ -0,0 +1,90 @@
+module Buckler
+
+  # Generate an Aws::S3::Bucket for the given `name`
+  # Also checks that the bucket is real and we have access to it.
+  # The only way to truly test bucket access is to try to read an item from it.
+  # Exit with a message if there is no access.
+  # Prerequisite: `Buckler.discover_aws_credentials!`
+
+  def self.get_bucket!(name)
+
+    unless name.present?
+      alert "No bucket name provided"
+      exit false
+    end
+
+    @bucket = Aws::S3::Bucket.new(name, client:@s3)
+
+    unless @bucket.exists?
+      alert "No such bucket “#{name}”"
+      exit false
+    end
+
+    @bucket.objects(max_keys:1).first # Tests bucket access
+    return @bucket
+
+  rescue Aws::S3::Errors::NoSuchBucket
+    
+    alert "No such bucket “#{name}”"
+    exit false
+
+  rescue Aws::S3::Errors::AccessDenied
+
+    alert "Access denied for bucket #{name}"
+    exit false
+
+  end
+
+  # Prints a warning message about irreversable changes to the screen.
+  # The user is required to confirm by typing the given `name_required`
+  # `additional_lines` are printed before the warning.
+  # If `confirmation` matches `name_required`, this method is a no-op.
+  # The program ends if the confirmation is not provided.
+
+  def self.require_confirmation!(name_required:, confirmation:nil, additional_lines:[])
+
+    return true if confirmation == name_required
+
+    alert "WARNING: Destructive Action"
+    additional_lines.each do |line|
+      log line
+    end
+    log "Depending on your S3 settings, this command may permanently"
+    log "delete objects from the bucket #{name_required.bucketize}."
+    log "To proceed, type “#{name_required}” or re-run this command with --confirm #{name_required}"
+    print "> ".dangerize
+
+    confirmation = STDIN.gets.chomp
+
+    if confirmation == name_required
+      return true
+    else
+      alert "Invalid confirmation “#{name_required}”, aborting"
+      exit false
+    end
+
+  end
+
+  # Prints a table neatly to the screen.
+  # The given `table_array` must be an Array of Arrays of Strings.
+  # Each inner array is a single row of the table, strings are cells of the row.
+
+  def self.puts_table!(table_array)
+
+    column_sizes = []
+
+    table_array.first.count.times do |column_index|
+      column_sizes << table_array.collect{ |row| row[column_index] }.collect(&:to_s).collect(&:length).max + 3
+    end
+
+    table_array.each do |line|
+      chart_row = ""
+      line.each_with_index do |column, index|
+        chart_row << column.to_s.ljust(column_sizes[index])
+      end
+      log chart_row
+    end
+
+  end
+
+end

data/lib/buckler/actions/create_bucket.rb

@@ -0,0 +1,53 @@
+module Buckler
+
+  def self.create_bucket!(name:nil, region:nil)
+
+    unless name.present?
+      alert "No bucket name provided."
+      alert "Usage: bucket create <bucket-name> --region <region>"
+      exit false
+    end
+
+    region ||= "us-east-1"
+    unless valid_region?(region)
+      log "Invalid region “#{region}”"
+      log "Use `bucket regions` to see a list of all S3 regions"
+      exit false
+    end
+
+    connect_to_s3!(region:region)
+    @bucket = Aws::S3::Bucket.new(name, client:@s3)
+
+    if @bucket.exists?
+      alert "Bucket #{@bucket.name} already exists"
+      exit false
+    end
+
+    log "Creating bucket #{name.bucketize} on #{region}…"
+
+    options = {
+      acl: "private"
+    }
+
+    unless region.eql?("us-east-1")
+      options[:create_bucket_configuration] = {
+        location_constraint: region
+      }
+    end
+
+    @bucket.create(options)
+    @bucket.wait_until_exists
+
+    log "Bucket #{name.bucketize} is how available for use ✔"
+    exit true
+
+  rescue Aws::S3::Errors::BucketAlreadyExists
+
+    alert "The bucket name “#{name}” is already taken."
+    alert "Bucket names must be unique across the entire AWS ecosystem."
+    alert "Select a different bucket name and re-run your command."
+    exit false
+
+  end
+
+end