smartmachine 0.8.0 → 1.0.0

data/README.md

@@ -0,0 +1,147 @@
+# SmartMachine
+
+Git push should deploy.
+
+SmartMachine is a full-stack deployment framework for rails optimized for admin programmer happiness and peaceful administration. It encourages natural simplicity by favoring convention over configuration.
+
+Before you begin, you should install ruby on your system.
+
+Deploy your Rails apps to your own server with - git push production main
+
+### How does it help?
+
+After you run the below commands, you get.
+1. Setup of basic best practices of setting up and securing a VPS server.
+2. Setup and installation of Docker.
+3. Setup and installation of docker based Mysql, Solr, Nginx, App Prereceiver.
+4. Deployment of your Rails apps to your own server with - git push production main
+
+### Prerequisites
+
+If using SmartMachine on a server, perform the below steps before proceeding.
+
+Ensure that you have debian LTS installed on the server.
+Running the below command should say some latest version of debian LTS.
+
+    $ cat /etc/issue
+
+Then complete, getting started with linode.
+
+    https://www.linode.com/docs/getting-started/
+
+And then secure your server.
+
+    https://www.linode.com/docs/security/securing-your-server/
+
+## Installation
+
+Install SmartMachine at the command prompt:
+
+    $ gem install smartmachine
+
+Then create a new machine and move into it:
+
+    $ smartmachine new yourmachinename
+    $ cd yourmachinename
+
+here "yourmachinename" is the machine name you can choose
+
+Add your credentials using:
+
+    $ smartmachine credentials:edit
+
+Add your environment details using: - Coming Soon
+
+    $ smartmachine environment:edit
+
+Add your users using:
+
+    $ smartmachine grid nginx users:edit
+
+Install docker, and add UFW rules for Docker if specified at the end of installation.
+
+    $ smartmachine docker install
+
+Install the engine and buildpacker:
+
+    $ smartmachine engine install
+    $ smartmachine buildpacker install rails
+
+## Usage
+
+### Choose Your Grids
+
+Choose only the grids you need. You can start or stop a grid at anytime using <b>up</b> or <b>down</b> commands respectively.
+
+#### 1. Nginx Grid
+Lets you run a nginx web server fully equipped with https encryption using letsencrypt.
+    
+    $ smartmachine grid nginx up
+    $ smartmachine grid nginx down
+
+#### 2. Prereceiver Grid
+Lets you push rails apps to your server without any additional configuration or downtime using <b>git push production main</b>.
+
+    $ smartmachine grid prereceiver install
+    $ smartmachine grid prereceiver up
+    $ smartmachine grid prereceiver down
+    $ smartmachine grid prereceiver uninstall
+
+#### 3. Mysql Grid
+Lets you run a mysql server instance with as many databases as you need.
+
+    $ smartmachine grid mysql up
+    $ smartmachine grid mysql down
+
+#### 4. Minio Grid
+Lets you run minio server instance with file storage persistance.
+
+    $ smartmachine grid minio up
+    $ smartmachine grid minio down
+
+#### 5. Elasticsearch Grid
+Lets you run elasticsearch server instance with data persistance.
+
+    $ smartmachine grid elasticsearch up
+    $ smartmachine grid elasticsearch down
+
+#### 4. Redis Grid
+Lets you run redis server instance for cache storage and job queueing.
+
+    $ smartmachine grid redis up
+    $ smartmachine grid redis down
+
+#### 6. Scheduler Grid - Coming Soon
+Lets you setup scheduling services like database backups, etc.
+
+    $ smartmachine grid scheduler install
+    $ smartmachine grid scheduler up
+    $ smartmachine grid scheduler down
+    $ smartmachine grid scheduler uninstall
+
+### Setup Your Apps
+
+Create your apps and manage them.
+
+#### 1. New App on the server
+Lets you create a new bare app on the server.
+
+    $ smartmachine app create <APPNAME> <APPDOMAIN> <USERNAME>
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/plainsource/smartmachine. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/plainsource/smartmachine/blob/main/CODE_OF_CONDUCT.md).
+
+## License
+
+The gem is available as open source under the terms of the [AGPL License](https://www.gnu.org/licenses/agpl-3.0.html).
+
+## Code of Conduct
+
+Everyone interacting in the SmartMachine project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/plainsource/smartmachine/blob/main/CODE_OF_CONDUCT.md).

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "smart_machine"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/exe/smartmachine

@@ -0,0 +1,3 @@
+#!/usr/bin/env ruby
+
+require 'smart_machine/commands'

data/lib/smart_machine/apps/app.rb

@@ -1,182 +1,166 @@
-# The main SmartMachine App driver
 module SmartMachine
-	class Apps
-		class App < SmartMachine::Base
-			def initialize
-			end
-
-			# Creating App!
-			#
-			# Example:
-			#   >> SmartMachine::Apps::Rails.create
-			#   => Creation Complete
-			#
-			# Arguments:
-			#   appname => (String)
-			# 	username => (String)
-			def self.create(*args)
-				args.flatten!
-				appname = args.empty? ? '' : args.shift
-				username = args.empty? ? '' : args.shift
-
-				raise "Please provide appname and username" if appname.empty? || username.empty?
-
-				if SmartMachine::Docker.running?
-					repository_path = "#{SmartMachine.config.user_home_path}/.smartmachine/apps/repositories/#{appname}.git"
-					container_path = "#{SmartMachine.config.user_home_path}/.smartmachine/apps/containers/#{appname}"
-					print "-----> Creating Application ... "
-
-					# Checking if app with given name already exists
-					if Dir.exist?(repository_path)
-						puts "failed. App with name '#{appname}' already exists."
-						exit
-					end
-
-					# Creating Directories
-					FileUtils.mkdir_p(repository_path)
-					FileUtils.mkdir_p(container_path)
-
-					# Initializing bare repo and pre-receive
-					Dir.chdir(repository_path) do
-						%x[git init --bare]
-						%x[chmod +x #{SmartMachine.config.user_home_path}/.smartmachine/grids/prereceiver/pre-receive]
-						%x[ln -s #{SmartMachine.config.user_home_path}/.smartmachine/grids/prereceiver/pre-receive #{repository_path}/hooks/pre-receive]
-						puts "done"
-					end
-
-					# Creating Environment File
-					if File.exist?("#{SmartMachine.config.user_home_path}/.smartmachine/config/environment.rb")
-						require "#{SmartMachine.config.user_home_path}/.smartmachine/config/environment"
-					end
-					unless File.exist? "#{container_path}/env"
-						print "-----> Creating App Environment ... "
-						page = <<~HEREDOC
-							## System
-							USERNAME=#{username}
-							KEEP_RELEASES=3
-
-							## Docker
-							VIRTUAL_HOST=#{appname}.#{SmartMachine.config.apps_domain}
-							LETSENCRYPT_HOST=#{appname}.#{SmartMachine.config.apps_domain}
-							LETSENCRYPT_EMAIL=#{SmartMachine.config.sysadmin_email}
-							LETSENCRYPT_TEST=false
-						HEREDOC
-						puts "done" if system("echo '#{page}' > #{container_path}/env")
-					end
-				end
-			end
-
-			def self.destroy(*args)
-				args.flatten!
-				appname = args.empty? ? '' : args.shift
-
-				raise "Please provide appname" if appname.empty?
-
-				if SmartMachine::Docker.running?
-					# Stopping & Removing old container
-					self.stop(appname)
-
-					# Destroying Directories
-					print "-----> Deleting App #{appname} ... "
-					repository_path = "#{SmartMachine.config.user_home_path}/.smartmachine/apps/repositories/#{appname}.git"
-					container_path = "#{SmartMachine.config.user_home_path}/.smartmachine/apps/containers/#{appname}"
-					FileUtils.rm_r(repository_path)
-					FileUtils.rm_r(container_path)
-					puts "done"
-				end
-			end
-			
-			def self.start(*args)
-				args.flatten!
-				appname = args.empty? ? '' : args.shift
-				app_version = args.empty? ? 0 : args.shift.to_i
-
-				raise "Please provide appname" if appname.empty?
-
-				logger.formatter = proc do |severity, datetime, progname, message|
-					severity_text = { "DEBUG" => "\u{1f527} #{severity}:", "INFO" => " \u{276f}", "WARN" => "\u{2757} #{severity}:",
-						"ERROR" => "\u{274c} #{severity}:", "FATAL" => "\u{2b55} #{severity}:", "UNKNOWN" => "\u{2753} #{severity}:"
-					}
-					"\t\t\t\t#{severity_text[severity]} #{message}\n"
-				end
-
-				if SmartMachine::Docker.running?
-					container_path = "#{SmartMachine.config.user_home_path}/.smartmachine/apps/containers/#{appname}"
-
-					Dir.chdir("#{container_path}/releases") do
-						# Getting App Version
-						if app_version == 0
-							app_versions = Dir.glob('*').select { |f| File.directory? f }.sort
-							app_version = app_versions.last
-						end
-						container_path_with_version = "#{container_path}/releases/#{app_version}"
-
-						logger.info "Launching Application ..."
-
-						app = SmartMachine::Apps::Rails.new
-						app.start(appname, container_path, container_path_with_version)
-					end
-				end
-
-				logger.formatter = nil
-			end
-
-			def self.stop(*args)
-				args.flatten!
-				appname = args.empty? ? '' : args.shift
-
-				raise "Please provide appname" if appname.empty?
-
-				container_name = appname
-
-				if SmartMachine::Docker.running?
-					container_id = `docker ps -a -q --filter='name=^#{container_name}$'`.chomp
-					unless container_id.empty?
-						logger.debug "Stopping & Removing container #{container_name} ..."
-						if system("docker stop #{container_name} && docker rm #{container_name}", out: File::NULL)
-							logger.debug "Stopped & Removed container #{container_name} ..."
-						end
-					else
-						logger.debug "Container '#{container_name}' does not exist to stop."
-					end
-				end
-			end
-
-			def self.clean_up(container_path)
-				env_vars = SmartMachine::Apps::App.get_env_vars(container_path)
-				return unless env_vars
-
-				logger.info "Cleaning up ..."
-
-				# Clean up very old versions
-				Dir.chdir("#{container_path}/releases") do
-					app_versions = Dir.glob('*').select { |f| File.directory? f }.sort
-					destroy_count = app_versions.count - env_vars['KEEP_RELEASES'].to_i
-					if destroy_count > 0
-						logger.debug "Deleting older application releases ..."
-						destroy_count.times do
-							FileUtils.rm_r(File.join(Dir.pwd, app_versions.shift))
-						end
-					end
-				end
-			end
-
-			def self.get_env_vars(container_path)
-				unless File.exist? "#{container_path}/env"
-					logger.fatal "Environment could not be loaded ... Failed."
-					return false
-				end
-
-				env_vars = {}
-				File.open("#{container_path}/env").each_line do |line|
-					line.chomp!
-					next if line.empty? || line.start_with?('#')
-				    key, value = line.split "="
-				    env_vars[key] = value
-				end
-
-				env_vars
-			end
-		end
-	end
-end
+  module Apps
+    class App < SmartMachine::Base
+      def initialize(appname:, container_number: nil, username: nil)
+        @home_dir = File.expand_path('~')
+        @appname = appname
+        @container_number = container_number
+        @username = username
+
+        @repository_path = "#{@home_dir}/machine/apps/repositories/#{@appname}.git"
+        @container_path = "#{@home_dir}/machine/apps/containers/#{@appname}"
+      end
+
+      def creater(appdomain:, prereceiver_name:)
+        raise "Please provide appname and username" if @appname.empty? || @username.empty?
+        prereceiver_name = prereceiver_name.to_s
+
+        print "-----> Creating Application ... "
+
+        # Checking if app with given name already exists
+        if Dir.exist?(@repository_path)
+          puts "failed. App with name '#{@appname}' already exists."
+          exit
+        end
+
+        # Creating Directories
+        FileUtils.mkdir_p(@repository_path)
+        FileUtils.mkdir_p(@container_path)
+
+        # Initializing bare repo and pre-receive
+        Dir.chdir(@repository_path) do
+          %x[git init --bare]
+          %x[ln -s ../../../../grids/prereceiver/#{prereceiver_name}/pre-receive hooks/pre-receive]
+          puts "done"
+        end
+
+        # Creating Environment File
+        if File.exist?("#{@home_dir}/machine/config/environment.rb")
+          require "#{@home_dir}/machine/config/environment"
+        end
+        unless File.exist? "#{@container_path}/env"
+          print "-----> Creating App Environment ... "
+          page = <<~HEREDOC
+                  ## System
+                  USERNAME=#{@username}
+                  KEEP_RELEASES=3
+
+                  ## Docker
+                  VIRTUAL_HOST=#{@appname}.#{appdomain}
+                  LETSENCRYPT_HOST=#{@appname}.#{appdomain}
+                  LETSENCRYPT_EMAIL=#{@username}
+                  LETSENCRYPT_TEST=false
+               HEREDOC
+          puts "done" if system("echo '#{page}' > #{@container_path}/env")
+        end
+      end
+
+      def destroyer
+        raise "Please provide appname" if @appname.empty?
+
+        # Checking if app with given name exists
+        unless Dir.exist?(@repository_path)
+          raise "App with name '#{@appname}' does not exist. Please provide a valid appname."
+        end
+
+        container_id = `docker ps -a -q --filter='name=^#{@appname}$' --filter='status=running'`.chomp
+        if container_id.empty?
+          # Destroying Directories
+          print "-----> Deleting App #{@appname} ... "
+          FileUtils.rm_r(@repository_path)
+          FileUtils.rm_r(@container_path)
+          puts "done"
+        end
+      end
+
+      def uper(version:)
+        raise "Please provide appname" if @appname.empty?
+
+        # Checking if app with given name exists
+        unless Dir.exist?(@repository_path)
+          raise "App with name '#{@appname}' does not exist. Please provide a valid appname."
+        end
+
+        logger.formatter = proc do |severity, datetime, progname, message|
+          severity_text = { "DEBUG" => "\u{1f527} #{severity}:", "INFO" => " \u{276f}", "WARN" => "\u{2757} #{severity}:",
+                           "ERROR" => "\u{274c} #{severity}:", "FATAL" => "\u{2b55} #{severity}:", "UNKNOWN" => "\u{2753} #{severity}:"
+                          }
+          "\t\t\t\t#{severity_text[severity]} #{message}\n"
+        end
+
+        Dir.chdir("#{@container_path}/releases") do
+          # Getting App Version
+          if version == 0
+            versions = Dir.glob('*').select { |f| File.directory? f }.sort
+            version = versions.last
+          end
+
+          logger.info "Launching Application ..."
+
+          buildpacker_rails = SmartMachine::Buildpackers::Rails.new(appname: @appname, version: version)
+          buildpacker_rails.uper
+        end
+
+        logger.formatter = nil
+      end
+
+      def downer
+        raise "Please provide appname" if @appname.empty?
+
+        # Checking if app with given name exists
+        unless Dir.exist?(@repository_path)
+          raise "App with name '#{@appname}' does not exist. Please provide a valid appname."
+        end
+
+        container_name = @appname
+        container_name += "_" + @container_number if @container_number
+
+        container_id = `docker ps -a -q --filter='name=^#{container_name}$'`.chomp
+        unless container_id.empty?
+          logger.debug "Stopping & Removing container #{container_name} ..."
+          if system("docker stop #{container_name} && docker rm #{container_name}", out: File::NULL)
+            logger.debug "Stopped & Removed container #{container_name} ..."
+          end
+        else
+          logger.debug "Container '#{container_name}' does not exist to stop."
+        end
+      end
+
+      def cleaner
+        env_vars = get_env_vars
+        return unless env_vars
+
+        logger.info "Cleaning up ..."
+
+        # Clean up very old versions
+        Dir.chdir("#{@container_path}/releases") do
+          versions = Dir.glob('*').select { |f| File.directory? f }.sort
+          destroy_count = versions.count - env_vars['KEEP_RELEASES'].to_i
+          if destroy_count > 0
+            logger.debug "Deleting older application releases ..."
+            destroy_count.times do
+              FileUtils.rm_r(File.join(Dir.pwd, versions.shift))
+            end
+          end
+        end
+      end
+
+      def get_env_vars
+        unless File.exist? "#{@container_path}/env"
+          logger.fatal "Environment could not be loaded ... Failed."
+          return false
+        end
+
+        env_vars = {}
+        File.open("#{@container_path}/env").each_line do |line|
+          line.chomp!
+          next if line.empty? || line.start_with?('#')
+          key, value = line.split "="
+          env_vars[key] = value
+        end
+
+        env_vars
+      end
+    end
+  end
+end

data/lib/smart_machine/apps/container.rb

@@ -0,0 +1,120 @@
+module SmartMachine
+  module Apps
+    class Container < SmartMachine::Base
+      def initialize(name:, appname:, appversion:)
+        @name = name
+        @appname = appname
+        @appversion = appversion
+        @appimage = "smartmachine/apps/#{@appname}:#{@appversion}"
+
+        @home_dir = File.expand_path('~')
+      end
+
+      def create!(using_command:, using_buildpacker: false)
+        unless using_buildpacker
+          unless system("docker image inspect #{@appimage}", [:out, :err] => File::NULL)
+            raise "Error: App Image with name #{@appimage} not found. Could not create #{@name} container."
+          end
+        end
+
+        logger.debug "-----> Creating container #{@name} ... "
+        command = [
+          "docker create",
+          "--name='#{@name}'",
+          "--env-file='#{@home_dir}/machine/apps/containers/#{@appname}/env'",
+          "--user `id -u`:`id -g`",
+          "--workdir /app",
+          "--expose='3000'",
+          "--volume='#{@home_dir}/smartmachine/config/environment.rb:#{@home_dir}/machine/config/environment.rb'",
+          "--volume='#{@home_dir}/smartmachine/apps/containers/#{@appname}/releases/#{@appversion}:/app'",
+          "--volume='#{@home_dir}/smartmachine/apps/containers/#{@appname}/app/vendor/bundle:/app/vendor/bundle'",
+          "--volume='#{@home_dir}/smartmachine/apps/containers/#{@appname}/app/public/assets:/app/public/assets'",
+          "--volume='#{@home_dir}/smartmachine/apps/containers/#{@appname}/app/public/packs:/app/public/packs'",
+          "--volume='#{@home_dir}/smartmachine/apps/containers/#{@appname}/app/node_modules:/app/node_modules'",
+          "--volume='#{@home_dir}/smartmachine/apps/containers/#{@appname}/app/storage:/app/storage'",
+          "--restart='always'",
+          "--init",
+          # "--network='nginx-network'",
+          "#{using_buildpacker ? "smartmachine/buildpackers/rails:#{SmartMachine.version}" : @appimage}"
+        ]
+        command.push(using_command) if using_command.present?
+
+        if system(command.compact.join(" "), out: File::NULL)
+          logger.debug "done"
+          return true
+        else
+          raise "Error: Could not create #{@name} container"
+        end
+
+        return false
+      end
+
+      def start!
+        logger.debug "-----> Starting container #{@name} ... "
+        if system("docker start #{@name}", out: File::NULL)
+          logger.debug "done"
+          return true
+        else
+          raise "Error: Could not start #{@name} container"
+        end
+
+        return false
+      end
+
+      def stop!
+        container_id = `docker ps -a -q --filter='name=^#{@name}$'`.chomp
+        unless container_id.empty?
+          logger.debug "Stopping & Removing container #{@name} ..."
+          if system("docker stop #{@name} && docker rm #{@name}", out: File::NULL)
+            logger.debug "Stopped & Removed container #{@name} ..."
+            return true
+          else
+            raise "Error: Container '#{@name}' could not be stopped and removed."
+          end
+        else
+          raise "Error: Container '#{@name}' does not exist to stop."
+        end
+
+        return false
+      end
+
+      def connect_to_network!(network_name:)
+        unless `docker network ls -q --filter name=^#{network_name}$`.chomp.empty?
+          if system("docker network connect #{network_name} #{@name}", out: File::NULL)
+            logger.debug "Connected to network #{network_name}."
+            return true
+          else
+            raise "Error: Could not connect to #{network_name} network."
+          end
+        else
+          raise "Error: The network with name #{network_name} was not found."
+        end
+
+        return false
+      end
+
+      def commit_app_image!
+        if create!(using_command: "smartmachine buildpacker packer rails", using_buildpacker: true)
+          logger.debug "-----> Starting attached container #{@name} ... "
+          if system("docker start --attach #{@name}")
+            logger.debug "-----> Committing container #{@name} to image... "
+            if system("docker commit #{@name} #{@appimage}", out: File::NULL)
+              stop!
+              return true
+            else
+              stop!
+              raise "Error: Could not commit #{@name} container to image."
+            end
+          else
+            stop!
+            raise "Error: Could not start attached #{@name} container"
+          end
+        else
+          raise "Error: Could not create #{@name} container"
+        end
+
+        return false
+      end
+    end
+  end
+end