remote_database_importer 0.1.1 → 0.1.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 83f84dda973912024d022d895763155d64da66822f5d6f7e771cfcf54562dd71
-  data.tar.gz: 9a2fb0fe981aef23c4778765cbf9f5e70e033c16d43c8d1a2a145b2abaf4f8ce
+  metadata.gz: cbe347c485be41742ebab8205d5d803b12cace72407f351c6e05ba9988fb8a80
+  data.tar.gz: 21ab68290d08a418b5921ec81ef46e927c41e3ffe7e99c2916acb27561cd9d30
 SHA512:
-  metadata.gz: 07be1da942f02d49869a62c64f3a1f4b5f81c23d624a604c42b1b0b844da94dcba18b86e824fd64cd147c7d4132f14832e0f23741e4470988c8f7fede720fcf0
-  data.tar.gz: 87a22ec48fcf6efb7c85bad1f183a6e1b0e3a0f9226fdf0dd3289b12bd1bfbad197ab0fbed2cdc3aebfafc56a421b62edc61db5a2c4407a885098fb842437400
+  metadata.gz: facbe97e036a224e1fa47fabe1e1774147d88154e1f77ff299dde2df89a7e8780c65bf7d5706b78e84d0928f01aebbf59add7f301ae2feb8022c50f6b64a6755
+  data.tar.gz: a3c042ad221b0b3273b8fe6b202bf724f9abecabe0fdea6a10acdbb848964426bb4c68d97cdb7b25bd90929155c2d44a0137f065e6943a8428077a8a48af7d0a

data/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## [0.1.2] - 2022-10-29
+
+- Terminate current DB sessions after dump remote database. So the chance of an open session is smaller
+- Print out where the config was saved
+- Print how long the import went
+
 ## [0.1.0] - 2022-10-24
 
 - Initial release

data/Gemfile.lock

@@ -1,9 +1,8 @@
 PATH
   remote: .
   specs:
-    remote_database_importer (0.1.0)
+    remote_database_importer (0.1.3)
       colorize (~> 0.8)
-      thor (~> 1.2)
       tty-config (~> 0.6)
       tty-spinner (~> 0.9)
 
@@ -53,7 +52,6 @@ GEM
     standard (1.16.1)
       rubocop (= 1.35.1)
       rubocop-performance (= 1.14.3)
-    thor (1.2.1)
     tty-config (0.6.0)
     tty-cursor (0.7.1)
     tty-spinner (0.9.3)

data/README.md

@@ -7,6 +7,7 @@ Its well possible that unexpected errors can occur.
 ## Features
 - Define multiple environments (such as staging, production etc.)
 - Rails intergration via rake task
+- Define custom commands that should run after successful import
 - Decide for yourself if the dump should be done over ssh or if pg_dump should connect to the DB port directly
 - It can therefore be used for almost all hosting providers (Heroku, Kubernetes, self-hosted, etc.)
 
@@ -33,18 +34,20 @@ Whenever you want current live data, you can run the command:
 rake remote_database:import
 ```
 
-![Import Job Demo](readme_images/import-job.gif)
+![Import-Job sample](readme_assets/import-job.gif)
 
 ### Config
 The settings for the different environments is stored in the `remote_database_importer.yml` file.  
 When you first run the rake task, it will dynamically create this file for you.
 
 
-![asdf](readme_images/config_sample.png)
+![Config sample](readme_assets/config_sample.png)
 
 ### DB Access
-The easiest and fastest way is to exchange your ssh-key with the server beforehand, so you don't have to enter a password.  
-Otherwise during the rake task execution a password entry is required.
+The dump of the remote database happens with the tool: [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html). 
+There are two opions for connecting to the remote databse:
+- pg_dump connects directly to the databse
+- pg_dump connects through a shh tunnel to the databse
 
 The effective dump call is as follows:
 ```ruby
@@ -53,9 +56,14 @@ or
 "pg_dump -Fc 'host=HOST dbname=DB_NAME user=DB_USER port=POSTGRES_PORT' > DB_DUMP_LOCATION"
 ```
 
+### Password Input
+Because this gem doesn't store passwords you will have to enter the passwords manually during the rake task.  
+Depending on the choosen connection model the password prompt will be for the ssh connection or the DB access.  
+If you choose to dump the databse over an ssh_tunnel, the easiest way will be to exchange your ssh-key with the server beforehand, so you don't have to enter a password.  
+
 ## Limitations
 - At the moment only Postgres databases are supported
-- It has to run inside a Rails app. There is a CLI command `remote_database_importer import` as an alternative to the rake task, but there are still some Rails commands like `rails db:drop db:create - rails db:migrate`, which makes it currently not possible to use the gem outside of Rails
+- It has to run inside a Rails app.
 - Not suitable for very large databases, you could run into SSH timeouts
 
 ## Contributing

data/lib/remote_database_importer/config.rb

@@ -15,6 +15,8 @@ module RemoteDatabaseImporter
         puts "===========================================================".colorize(:green)
         puts "Hi there! There is no config file yet, lets create one! 😄"
         create_default_config
+        config_location = [@config.filename, @config.extname].join
+        puts "Created config file: #{config_location}"
         puts "===========================================================".colorize(:green)
       end
       @config.read
@@ -49,10 +51,10 @@ module RemoteDatabaseImporter
 
         puts "Connection settings:".colorize(:green)
         host = ask("Enter the IP or hostname of the DB server:", default: "myawesomeapp.com")
-        dump_type = ask("Should the dump happen over a ssh connection or can pg_dump access the DB port directly? (if the DB lives on a seperat server pg_dump the way to go)", default: "pg_dump", options: ["ssh", "pg_dump"])
+        dump_type = ask("Should the DB dump happen over a ssh tunnel or can pg_dump connect to the DB port directly?", default: "pg_dump", options: ["ssh_tunnel", "pg_dump"])
 
         ssh_user, ssh_port, postgres_port = nil
-        if dump_type == "ssh"
+        if dump_type == "ssh_tunnel"
           ssh_user = ask("Enter the username for the SSH connection:", default: "deployer")
           ssh_port = ask("Enter the port for the SSH connection:", default: "22")
         else
@@ -85,15 +87,12 @@ module RemoteDatabaseImporter
         end
       end
 
+      puts "Define custom commands that run after successful import:".colorize(:green)
+      custom_commands = ask("Enter semicolon separated commands that should run after importing the DB:", default: "rake db:migrate; echo 'All Done'")
+      puts
+
+      @config.set(:custom_commands, value: custom_commands)
       @config.write
     end
-
-    # TODO: validate user input
-    # private
-    # def validate_config(config)
-    #   config.each do |key, value|
-    #
-    #   end
-    # end
   end
 end

data/lib/remote_database_importer/operation.rb

@@ -32,6 +32,7 @@ module RemoteDatabaseImporter
 
     def import
       select_environment
+      time_start = Time.now
       multi_spinner = TTY::Spinner::Multi.new("[:spinner] Import remote DB", format: :dots_3)
       tasks = create_tasks_and_spinners(multi_spinner)
 
@@ -42,19 +43,20 @@ module RemoteDatabaseImporter
         return "Can't continue, following task failed: #{task[:command]} - checkout the logfile: #{LOG_FILE}" unless task_execution_was_successful
         task[:spinner].stop("... Done!")
       end
+      puts seconds_to_human_readable_time(Time.now - time_start)
     end
 
     private
 
     def create_tasks_and_spinners(multi_spinner)
       tasks = [
-        {name: "Terminate current DB sessions", command: terminate_current_db_sessions},
         {name: "Dump remote DB", command: dump_remote_db},
+        {name: "Terminate current DB sessions", command: terminate_current_db_sessions},
         {name: "Drop and create local DB", command: drop_and_create_local_db},
         {name: "Restore remote DB", command: restore_db},
-        {name: "Run migrations", command: run_migrations},
         {name: "Remove logfile", command: remove_logfile},
-        {name: "Remove dumpfile", command: remove_dumpfile}
+        {name: "Remove dumpfile", command: remove_dumpfile},
+        {name: "Custom commands", command: custom_commands}
       ]
       tasks.each.with_index(1) do |task, index|
         task[:spinner] = multi_spinner.register "#{index}/#{tasks.length} :spinner #{task[:name]}"
@@ -77,7 +79,7 @@ module RemoteDatabaseImporter
       ssh_port = env["connection"]["ssh_port"]
       postgres_port = env["connection"]["postgres_port"]
 
-      if dump_type == "ssh"
+      if dump_type == "ssh_tunnel"
         "ssh #{ssh_user}@#{host} -p #{ssh_port} 'pg_dump -Fc -U #{db_user} -d #{db_name} -h localhost -C' > #{db_dump_location}"
       else
         "pg_dump -Fc 'host=#{host} dbname=#{db_name} user=#{db_user} port=#{postgres_port}' > #{db_dump_location}"
@@ -92,10 +94,6 @@ module RemoteDatabaseImporter
       "pg_restore --jobs 8 --no-privileges --no-owner --dbname #{@config.fetch("local_db_name")} #{db_dump_location}"
     end
 
-    def run_migrations
-      "rake db:migrate > #{LOG_FILE}"
-    end
-
     def remove_logfile
       "rm #{LOG_FILE}"
     end
@@ -104,8 +102,22 @@ module RemoteDatabaseImporter
       "rm #{db_dump_location}"
     end
 
+    def custom_commands
+      @config.fetch("custom_commands")
+    end
+
     def db_dump_location
       "tmp/#{@current_environment["database"]["name"]}.dump"
     end
+
+    def seconds_to_human_readable_time(secs)
+      [[60, :seconds], [60, :minutes], [24, :hours], [Float::INFINITY, :days]].map { |count, name|
+        if secs > 0
+          secs, n = secs.divmod(count)
+
+          "#{n.to_i} #{name}" unless n.to_i == 0
+        end
+      }.compact.reverse.join(" ")
+    end
   end
 end

data/lib/remote_database_importer/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RemoteDatabaseImporter
-  VERSION = "0.1.1"
+  VERSION = "0.1.3"
 end

metadata

@@ -1,29 +1,15 @@
 --- !ruby/object:Gem::Specification
 name: remote_database_importer
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.1.3
 platform: ruby
 authors:
 - Leon Vogt
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-10-24 00:00:00.000000000 Z
+date: 2022-11-06 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: thor
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.2'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.2'
 - !ruby/object:Gem::Dependency
   name: tty-config
   requirement: !ruby/object:Gem::Requirement
@@ -70,8 +56,7 @@ description: Dump remote databases and import it locally. At the moment only Pos
   databases are supported
 email:
 - nonick@nonick.ch
-executables:
-- remote_database_importer
+executables: []
 extensions: []
 extra_rdoc_files: []
 files:
@@ -83,16 +68,14 @@ files:
 - LICENSE.txt
 - README.md
 - Rakefile
-- bin/remote_database_importer
 - lib/railtie.rb
 - lib/remote_database_importer.rb
 - lib/remote_database_importer/config.rb
 - lib/remote_database_importer/operation.rb
 - lib/remote_database_importer/version.rb
 - lib/tasks/remote_database_importer.rake
-- readme_images/.DS_Store
-- readme_images/config_sample.png
-- readme_images/import-job.gif
+- readme_assets/config_sample.png
+- readme_assets/import-job.gif
 - sig/remote_database_importer.rbs
 homepage: https://github.com/leon-vogt/remote_database_importer
 licenses:

data/bin/remote_database_importer

@@ -1,14 +0,0 @@
-#!/usr/bin/env ruby
-
-require 'thor'
-require_relative '../lib/remote_database_importer/operation'
-
-class RemoteDatabaseImporterCLI < Thor
-  desc "Restore remote database and import it locally", "asdf"
-  def import
-    importer = RemoteDatabaseImporter::Operation.new
-    importer.import
-  end
-end
-
-RemoteDatabaseImporterCLI.start