vps 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 19a1e8448e7c6d7549da881d3c4410dffd9eee99
+  data.tar.gz: 8ed547a6a851bfc83a7be9df20d8781d8bf39d6b
+SHA512:
+  metadata.gz: 589a2204c45d5eb76a578258d6ac56b053f7efd99fec980ed81b8b9f58e2d86d25918ddd0eff90cb91be63ba1d5f0bae4b26e9bedb7636d264a1e6c377693391
+  data.tar.gz: 477b312e229c9cef09c4f75a9e7e8f0aff05be8d9882940b9c03721049563addc569ace2c57739cb73c4a32390c8fdd78f8743af161037183be871745652d791

data/.gitignore

@@ -0,0 +1,8 @@
+.DS_Store
+.bundle
+.yardoc
+.rvmrc
+Gemfile.lock
+doc
+pkg
+test/coverage

data/CHANGELOG.md

@@ -0,0 +1,15 @@
+## VPS CHANGELOG
+
+### Version 0.2.0 (December 31, 2019)
+
+* Provide ability to add both custom HTTP and HTTPS Nginx configs
+
+### Version 0.1.2 (September 29, 2019)
+
+* Run postload tasks at the end of deployment
+* Fix generating Rails and Rack Dockerfile (installing the correct Bundler version)
+* Include configured :services in the generated docker-compose.yml file
+
+### Version 0.1.1 (August 12, 2019)
+
+* Initial release

data/Gemfile

@@ -0,0 +1,3 @@
+source "https://rubygems.org"
+
+gemspec

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2019 Paul Engel
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,107 @@
+# VPS
+
+Zero-config deployments of Plug, Phoenix, Rack and Rails apps on a clean Ubuntu server using Docker and Let's Encrypt
+
+**NOTE: This is an experimental project**
+
+## Introduction
+
+Despite of having my fair share of experience in software development, hosting web applications has never really been my strongest skill set. So as a real programmer, I want to use a command line interface for easy deployments of my web applications (using HTTPS or not).
+
+Important for me is not want to be locked onto a specific hosting provider and also, I want the server to be dispensable because "I know that my CLI has got my back".
+
+So enter the Ruby gem `VPS` which makes it able to deploy my web application on a totally clean Ubuntu server by roughly just executing five simple commands :muscle:
+
+## Installation
+
+Run the following command to install `VPS`:
+
+    $ gem install "vps"
+
+## Usage
+
+VPS is a command-line-interface, you can print help instructions:
+
+    $ vps help
+    Commands:
+      vps -v, [--version]                          # Show VPS version number
+      vps deploy HOST [TOOL]                       # Deploy web application to the server
+      vps domain                                   # Manage upstream domains
+      vps domain add HOST:UPSTREAM DOMAIN [EMAIL]  # Add domain to host upstream (email recommended for https)
+      vps domain help [COMMAND]                    # Describe subcommands or one specific subcommand
+      vps domain list HOST[:UPSTREAM]              # List domains of host
+      vps domain remove HOST[:UPSTREAM] [DOMAIN]   # Remove domain from host upstream
+      vps edit [HOST]                              # Edit the VPS configuration(s)
+      vps help [COMMAND]                           # Describe available commands or one specific command
+      vps init HOST                                # Execute an initial server setup
+      vps install HOST [TOOL]                      # Install software on the server
+      vps service                                  # Manage host services
+      vps service add HOST [SERVICE]               # Add service to host configuration
+      vps service help [COMMAND]                   # Describe subcommands or one specific subcommand
+      vps service list HOST                        # List services of host configuration
+      vps service remove HOST [SERVICE]            # Remove service from host configuration
+      vps upstream                                 # Manage host upstreams
+      vps upstream add HOST[:UPSTREAM] PATH        # Add upstream to host configuration
+      vps upstream help [COMMAND]                  # Describe subcommands or one specific subcommand
+      vps upstream list HOST                       # List upstreams of host configuration
+      vps upstream remove HOST[:UPSTREAM]          # Remove upstream from host configuration
+
+### Deploying a Plug / Phoenix / Rack / Rails application to a totally clean installed Ubuntu server
+
+Let's say the SSH host is called `silver_surfer` and that the application is located at `~/Sources/spider_web`.
+
+Just execute the following commands:
+
+    $ vps init silver_surfer
+    $ vps install silver_surfer docker
+    $ vps upstream add silver_surfer ~/Sources/spider_web
+    $ vps domain add silver_surfer:spider_web http://spider.web
+    $ vps deploy silver_surfer
+
+Et voilà. Your awesome website is online, powered by Docker and Nginx! :D
+
+### Want to use a HTTPS domain?
+
+No problem, just specify so and make sure you also pass a valid email address (which is recommended). During deployment `certbot` will be added to the docker compose config and the SSL certificates will be created using [`init-letsencrypt.sh`](https://github.com/archan937/vps/blob/master/templates/docker/init-letsencrypt.sh.erb).
+
+    $ vps init silver_surfer
+    $ vps install silver_surfer docker
+    $ vps upstream add silver_surfer ~/Sources/spider_web
+    $ vps domain add silver_surfer:spider_web https://spider.web your.valid@email-address.com
+    $ vps deploy silver_surfer
+
+Cool, huh? :D
+
+### Want to add a (commonly used) service?
+
+Easy. Just run `vps service add <yourhost>` (e.g. `vps service add silver_surfer`) and follow the instructions.
+
+## Credits
+
+Thanks Philipp Medien (@pentacent_hq) for writing about using Nginx and Let's Encrypt with Docker:
+
+https://medium.com/@pentacent / [the-blog-post](https://medium.com/@pentacent/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71)
+
+Thanks Dmitry Fedosov (@dimafeng) for writing about scripting Docker based deployments:
+
+http://dimafeng.com / [the-blog-post](http://dimafeng.com/2015/10/17/docker-distribution)
+
+## TODO
+
+* Add documentation about adding docker-compose and Nginx related configs plus adding pre- and postload tasks
+
+## Contact me
+
+For support, remarks and requests, please mail me at [pm_engel@icloud.com](mailto:pm_engel@icloud.com).
+
+## License
+
+Copyright (c) 2019 Paul Engel, released under the MIT license
+
+http://github.com/archan937 – http://twitter.com/archan937 – pm_engel@icloud.com
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,10 @@
+#!/usr/bin/env rake
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+task :default => :test
+
+Rake::TestTask.new do |test|
+  test.pattern = "test/**/test_*.rb"
+end

data/VERSION

@@ -0,0 +1 @@
+0.1.2

data/bin/vps

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+
+require "rubygems"
+require "vps/cli"
+
+begin
+  VPS::CLI.start
+rescue Interrupt
+  puts "Abort."
+end

data/config/services.yml

@@ -0,0 +1,63 @@
+---
+memcached:
+  restart: on-failure
+mysql:
+  restart: on-failure
+  ports:
+    - 3306:3306
+  environment:
+    - MYSQL_DATABASE
+    - MYSQL_USER
+    - MYSQL_PASSWORD
+    - MYSQL_ROOT_PASSWORD
+  volumes:
+    - mysql:/var/lib/mysql
+mariadb:
+  restart: on-failure
+  ports:
+    - 3306:3306
+  environment:
+    - MYSQL_DATABASE
+    - MYSQL_USER
+    - MYSQL_PASSWORD
+    - MYSQL_ROOT_PASSWORD
+  volumes:
+    - mariadb:/var/lib/mysql
+postgres:
+  restart: on-failure
+  ports:
+    - 5432:5432
+  environment:
+    - POSTGRES_DB
+    - POSTGRES_USER
+    - POSTGRES_PASSWORD
+  volumes:
+    - postgres:/var/lib/postgres
+mongo:
+  restart: on-failure
+  environment:
+    - MONGO_INITDB_DATABASE
+    - MONGO_INITDB_ROOT_USERNAME
+    - MONGO_INITDB_ROOT_PASSWORD
+  volumes:
+    - mongo-db:/data/db
+    - mongo-config:/data/configdb
+redis:
+  restart: on-failure
+  volumes:
+    - redis:/data
+rabbitmq:
+  restart: on-failure
+  environment:
+    - RABBITMQ_ERLANG_COOKIE
+    - RABBITMQ_DEFAULT_USER
+    - RABBITMQ_DEFAULT_PASS
+  volumes:
+    - rabbitmq-etc:/etc/rabbitmq
+    - rabbitmq-lib:/var/lib/rabbitmq
+    - rabbitmq-logs:/var/log/rabbitmq
+clickhouse:
+  image: yandex/clickhouse-server
+  restart: on-failure
+  volumes:
+    - clickhouse:/var/lib/clickhouse

data/lib/vps.rb

@@ -0,0 +1,88 @@
+require "vps/version"
+
+module VPS
+  extend self
+
+  ROOT = File.expand_path("#{__FILE__}/../..")
+  PLAYBOOKS = "#{ROOT}/playbooks"
+  TEMPLATES = "#{ROOT}/templates"
+
+  def config_path(host, path = "config.yml")
+    File.expand_path("~/.vps/#{host}/#{path}")
+  end
+
+  def read_template(path)
+    File.read("#{TEMPLATES}/#{path}")
+  end
+
+  def read_config(host, key = nil)
+    config =
+      if File.exists?(path = config_path(host))
+        YAML.load_file(path)
+      elsif key.nil?
+        {
+          :user => nil,
+          :tool => nil,
+          :release_path => nil,
+          :services => nil,
+          :upstreams => nil,
+          :volumes => nil,
+          :preload => nil,
+          :postload => nil
+        }
+      end
+
+    if config
+      config = with_indifferent_access(config)
+      if key
+        with_indifferent_access(config[key])
+      else
+        config[:services] ||= {}
+        config[:upstreams] ||= []
+        config[:volumes] ||= []
+        config
+      end
+    end
+  end
+
+  def write_config(host, changes)
+    config = read_config(host) || {}
+    changed = false
+
+    %w(services upstreams volumes).each do |key|
+      value = changes[key]
+      changes[key] = nil if value && value.empty?
+    end
+
+    changes.each do |key, value|
+      if !config.include?(key) || (config[key] != value)
+        config[key] = value
+        changed = true
+      end
+    end
+
+    if changed
+      path = config_path(host)
+      config = JSON.parse(config.to_json)
+      FileUtils.mkdir_p(File.dirname(path))
+      File.write(path, config.to_yaml)
+    end
+  end
+
+private
+
+  def with_indifferent_access(object)
+    case object
+    when Hash
+      object.inject({}.with_indifferent_access) do |hash, (key, value)|
+        hash[key] = with_indifferent_access(value)
+        hash
+      end
+    when Array
+      object.collect{|item| with_indifferent_access(item)}
+    else
+      object
+    end
+  end
+
+end

data/lib/vps/cli.rb

@@ -0,0 +1,62 @@
+require "fileutils"
+require "json"
+require "yaml"
+require "ostruct"
+require "uri"
+require "thor"
+require "erubis"
+require "inquirer"
+require "net/http"
+require "net/ssh"
+
+require "active_support/dependencies/autoload"
+require "active_support/core_ext/string/inflections"
+require "active_support/core_ext/hash"
+require "active_support/number_helper"
+
+require "vps"
+require "vps/core_ext/string"
+require "vps/core_ext/ostruct"
+require "vps/cli/service"
+require "vps/cli/upstream"
+require "vps/cli/domain"
+require "vps/cli/playbook"
+
+module VPS
+  class CLI < Thor
+
+    class Error < StandardError; end
+
+    Playbook.all.each do |playbook|
+      desc playbook.usage, playbook.description
+      method_options playbook.options if playbook.options
+      define_method playbook.command do |*args|
+        start = Time.now
+        playbook.run!(args, options)
+        puts "\nDone. ".cyan + "#{(Time.now - start).round(3)}s".gray
+      end
+    end
+
+    desc "edit [HOST]", "Edit the VPS configuration(s)"
+    def edit(host = nil)
+      `#{ENV["EDITOR"]} #{VPS.config_path(host, "")}`
+    end
+
+    register(Upstream, "upstream", "upstream", "Manage host upstreams")
+    register(Service, "service", "service", "Manage host services")
+    register(Domain, "domain", "domain", "Manage upstream domains")
+
+    desc "-v, [--version]", "Show VPS version number"
+    map %w(-v --version) => :version
+    def version
+      puts "vps #{VPS::VERSION}"
+    end
+
+  private
+
+    def method_missing(method, *_args)
+      raise Error, "Unrecognized command \"#{method}\". Please consult `vps help`."
+    end
+
+  end
+end

data/lib/vps/cli/domain.rb

@@ -0,0 +1,59 @@
+module VPS
+  class CLI < Thor
+    class Domain < Thor
+
+      desc "add HOST:UPSTREAM DOMAIN [EMAIL]", "Add domain to host upstream (email recommended for https)"
+      def add(host_and_upstream, domain, email = nil)
+        return unless domain.match(/^https?:\/\/([a-z0-9\-]{2,}\.)+[a-z]{2,}$/)
+
+        host, name = host_and_upstream.split(":")
+        config = VPS.read_config(host)
+
+        if (upstream = config[:upstreams].detect{|upstream| upstream[:name] == name})
+          upstream[:email] = email if email
+          upstream[:domains].push(domain).uniq!
+          VPS.write_config(host, config)
+        end
+      end
+
+      desc "remove HOST[:UPSTREAM] [DOMAIN]", "Remove domain from host upstream"
+      def remove(host_and_upstream, domain = nil)
+        host, name = host_and_upstream.split(":")
+        config = VPS.read_config(host)
+
+        unless name
+          list = config[:upstreams].collect{|upstream| upstream[:name]}.sort
+          name = list[Ask.list("From which upstream do you want to remove a domain?", list)]
+        end
+
+        if (upstream = config[:upstreams].detect{|upstream| upstream[:name] == name})
+          unless domain
+            list = upstream[:domains]
+            domain = list[Ask.list("Which domain do you want to remove?", list)]
+          end
+
+          if upstream[:domains].reject!{|x| x == domain}
+            VPS.write_config(host, config)
+          end
+        end
+      end
+
+      desc "list HOST[:UPSTREAM]", "List domains of host"
+      def list(host_and_optional_upstream)
+        host, name = host_and_optional_upstream.split(":")
+        config = VPS.read_config(host)
+
+        domains = config[:upstreams].collect do |upstream|
+          if name.nil? || upstream[:name] == name
+            upstream[:domains]
+              .collect{|domain| "  ~> #{domain}"}
+              .unshift("* #{upstream[:name]}:")
+          end
+        end.flatten.compact
+
+        puts domains
+      end
+
+    end
+  end
+end