remote_database_importer 0.1.1 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 83f84dda973912024d022d895763155d64da66822f5d6f7e771cfcf54562dd71
4
- data.tar.gz: 9a2fb0fe981aef23c4778765cbf9f5e70e033c16d43c8d1a2a145b2abaf4f8ce
3
+ metadata.gz: cbe347c485be41742ebab8205d5d803b12cace72407f351c6e05ba9988fb8a80
4
+ data.tar.gz: 21ab68290d08a418b5921ec81ef46e927c41e3ffe7e99c2916acb27561cd9d30
5
5
  SHA512:
6
- metadata.gz: 07be1da942f02d49869a62c64f3a1f4b5f81c23d624a604c42b1b0b844da94dcba18b86e824fd64cd147c7d4132f14832e0f23741e4470988c8f7fede720fcf0
7
- data.tar.gz: 87a22ec48fcf6efb7c85bad1f183a6e1b0e3a0f9226fdf0dd3289b12bd1bfbad197ab0fbed2cdc3aebfafc56a421b62edc61db5a2c4407a885098fb842437400
6
+ metadata.gz: facbe97e036a224e1fa47fabe1e1774147d88154e1f77ff299dde2df89a7e8780c65bf7d5706b78e84d0928f01aebbf59add7f301ae2feb8022c50f6b64a6755
7
+ data.tar.gz: a3c042ad221b0b3273b8fe6b202bf724f9abecabe0fdea6a10acdbb848964426bb4c68d97cdb7b25bd90929155c2d44a0137f065e6943a8428077a8a48af7d0a
data/CHANGELOG.md CHANGED
@@ -1,5 +1,11 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.1.2] - 2022-10-29
4
+
5
+ - Terminate current DB sessions after dump remote database. So the chance of an open session is smaller
6
+ - Print out where the config was saved
7
+ - Print how long the import went
8
+
3
9
  ## [0.1.0] - 2022-10-24
4
10
 
5
11
  - Initial release
data/Gemfile.lock CHANGED
@@ -1,9 +1,8 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- remote_database_importer (0.1.0)
4
+ remote_database_importer (0.1.3)
5
5
  colorize (~> 0.8)
6
- thor (~> 1.2)
7
6
  tty-config (~> 0.6)
8
7
  tty-spinner (~> 0.9)
9
8
 
@@ -53,7 +52,6 @@ GEM
53
52
  standard (1.16.1)
54
53
  rubocop (= 1.35.1)
55
54
  rubocop-performance (= 1.14.3)
56
- thor (1.2.1)
57
55
  tty-config (0.6.0)
58
56
  tty-cursor (0.7.1)
59
57
  tty-spinner (0.9.3)
data/README.md CHANGED
@@ -7,6 +7,7 @@ Its well possible that unexpected errors can occur.
7
7
  ## Features
8
8
  - Define multiple environments (such as staging, production etc.)
9
9
  - Rails intergration via rake task
10
+ - Define custom commands that should run after successful import
10
11
  - Decide for yourself if the dump should be done over ssh or if pg_dump should connect to the DB port directly
11
12
  - It can therefore be used for almost all hosting providers (Heroku, Kubernetes, self-hosted, etc.)
12
13
 
@@ -33,18 +34,20 @@ Whenever you want current live data, you can run the command:
33
34
  rake remote_database:import
34
35
  ```
35
36
 
36
- ![Import Job Demo](readme_images/import-job.gif)
37
+ ![Import-Job sample](readme_assets/import-job.gif)
37
38
 
38
39
  ### Config
39
40
  The settings for the different environments is stored in the `remote_database_importer.yml` file.
40
41
  When you first run the rake task, it will dynamically create this file for you.
41
42
 
42
43
 
43
- ![asdf](readme_images/config_sample.png)
44
+ ![Config sample](readme_assets/config_sample.png)
44
45
 
45
46
  ### DB Access
46
- The easiest and fastest way is to exchange your ssh-key with the server beforehand, so you don't have to enter a password.
47
- Otherwise during the rake task execution a password entry is required.
47
+ The dump of the remote database happens with the tool: [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html).
48
+ There are two opions for connecting to the remote databse:
49
+ - pg_dump connects directly to the databse
50
+ - pg_dump connects through a shh tunnel to the databse
48
51
 
49
52
  The effective dump call is as follows:
50
53
  ```ruby
@@ -53,9 +56,14 @@ or
53
56
  "pg_dump -Fc 'host=HOST dbname=DB_NAME user=DB_USER port=POSTGRES_PORT' > DB_DUMP_LOCATION"
54
57
  ```
55
58
 
59
+ ### Password Input
60
+ Because this gem doesn't store passwords you will have to enter the passwords manually during the rake task.
61
+ Depending on the choosen connection model the password prompt will be for the ssh connection or the DB access.
62
+ If you choose to dump the databse over an ssh_tunnel, the easiest way will be to exchange your ssh-key with the server beforehand, so you don't have to enter a password.
63
+
56
64
  ## Limitations
57
65
  - At the moment only Postgres databases are supported
58
- - It has to run inside a Rails app. There is a CLI command `remote_database_importer import` as an alternative to the rake task, but there are still some Rails commands like `rails db:drop db:create - rails db:migrate`, which makes it currently not possible to use the gem outside of Rails
66
+ - It has to run inside a Rails app.
59
67
  - Not suitable for very large databases, you could run into SSH timeouts
60
68
 
61
69
  ## Contributing
@@ -15,6 +15,8 @@ module RemoteDatabaseImporter
15
15
  puts "===========================================================".colorize(:green)
16
16
  puts "Hi there! There is no config file yet, lets create one! 😄"
17
17
  create_default_config
18
+ config_location = [@config.filename, @config.extname].join
19
+ puts "Created config file: #{config_location}"
18
20
  puts "===========================================================".colorize(:green)
19
21
  end
20
22
  @config.read
@@ -49,10 +51,10 @@ module RemoteDatabaseImporter
49
51
 
50
52
  puts "Connection settings:".colorize(:green)
51
53
  host = ask("Enter the IP or hostname of the DB server:", default: "myawesomeapp.com")
52
- dump_type = ask("Should the dump happen over a ssh connection or can pg_dump access the DB port directly? (if the DB lives on a seperat server pg_dump the way to go)", default: "pg_dump", options: ["ssh", "pg_dump"])
54
+ dump_type = ask("Should the DB dump happen over a ssh tunnel or can pg_dump connect to the DB port directly?", default: "pg_dump", options: ["ssh_tunnel", "pg_dump"])
53
55
 
54
56
  ssh_user, ssh_port, postgres_port = nil
55
- if dump_type == "ssh"
57
+ if dump_type == "ssh_tunnel"
56
58
  ssh_user = ask("Enter the username for the SSH connection:", default: "deployer")
57
59
  ssh_port = ask("Enter the port for the SSH connection:", default: "22")
58
60
  else
@@ -85,15 +87,12 @@ module RemoteDatabaseImporter
85
87
  end
86
88
  end
87
89
 
90
+ puts "Define custom commands that run after successful import:".colorize(:green)
91
+ custom_commands = ask("Enter semicolon separated commands that should run after importing the DB:", default: "rake db:migrate; echo 'All Done'")
92
+ puts
93
+
94
+ @config.set(:custom_commands, value: custom_commands)
88
95
  @config.write
89
96
  end
90
-
91
- # TODO: validate user input
92
- # private
93
- # def validate_config(config)
94
- # config.each do |key, value|
95
- #
96
- # end
97
- # end
98
97
  end
99
98
  end
@@ -32,6 +32,7 @@ module RemoteDatabaseImporter
32
32
 
33
33
  def import
34
34
  select_environment
35
+ time_start = Time.now
35
36
  multi_spinner = TTY::Spinner::Multi.new("[:spinner] Import remote DB", format: :dots_3)
36
37
  tasks = create_tasks_and_spinners(multi_spinner)
37
38
 
@@ -42,19 +43,20 @@ module RemoteDatabaseImporter
42
43
  return "Can't continue, following task failed: #{task[:command]} - checkout the logfile: #{LOG_FILE}" unless task_execution_was_successful
43
44
  task[:spinner].stop("... Done!")
44
45
  end
46
+ puts seconds_to_human_readable_time(Time.now - time_start)
45
47
  end
46
48
 
47
49
  private
48
50
 
49
51
  def create_tasks_and_spinners(multi_spinner)
50
52
  tasks = [
51
- {name: "Terminate current DB sessions", command: terminate_current_db_sessions},
52
53
  {name: "Dump remote DB", command: dump_remote_db},
54
+ {name: "Terminate current DB sessions", command: terminate_current_db_sessions},
53
55
  {name: "Drop and create local DB", command: drop_and_create_local_db},
54
56
  {name: "Restore remote DB", command: restore_db},
55
- {name: "Run migrations", command: run_migrations},
56
57
  {name: "Remove logfile", command: remove_logfile},
57
- {name: "Remove dumpfile", command: remove_dumpfile}
58
+ {name: "Remove dumpfile", command: remove_dumpfile},
59
+ {name: "Custom commands", command: custom_commands}
58
60
  ]
59
61
  tasks.each.with_index(1) do |task, index|
60
62
  task[:spinner] = multi_spinner.register "#{index}/#{tasks.length} :spinner #{task[:name]}"
@@ -77,7 +79,7 @@ module RemoteDatabaseImporter
77
79
  ssh_port = env["connection"]["ssh_port"]
78
80
  postgres_port = env["connection"]["postgres_port"]
79
81
 
80
- if dump_type == "ssh"
82
+ if dump_type == "ssh_tunnel"
81
83
  "ssh #{ssh_user}@#{host} -p #{ssh_port} 'pg_dump -Fc -U #{db_user} -d #{db_name} -h localhost -C' > #{db_dump_location}"
82
84
  else
83
85
  "pg_dump -Fc 'host=#{host} dbname=#{db_name} user=#{db_user} port=#{postgres_port}' > #{db_dump_location}"
@@ -92,10 +94,6 @@ module RemoteDatabaseImporter
92
94
  "pg_restore --jobs 8 --no-privileges --no-owner --dbname #{@config.fetch("local_db_name")} #{db_dump_location}"
93
95
  end
94
96
 
95
- def run_migrations
96
- "rake db:migrate > #{LOG_FILE}"
97
- end
98
-
99
97
  def remove_logfile
100
98
  "rm #{LOG_FILE}"
101
99
  end
@@ -104,8 +102,22 @@ module RemoteDatabaseImporter
104
102
  "rm #{db_dump_location}"
105
103
  end
106
104
 
105
+ def custom_commands
106
+ @config.fetch("custom_commands")
107
+ end
108
+
107
109
  def db_dump_location
108
110
  "tmp/#{@current_environment["database"]["name"]}.dump"
109
111
  end
112
+
113
+ def seconds_to_human_readable_time(secs)
114
+ [[60, :seconds], [60, :minutes], [24, :hours], [Float::INFINITY, :days]].map { |count, name|
115
+ if secs > 0
116
+ secs, n = secs.divmod(count)
117
+
118
+ "#{n.to_i} #{name}" unless n.to_i == 0
119
+ end
120
+ }.compact.reverse.join(" ")
121
+ end
110
122
  end
111
123
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RemoteDatabaseImporter
4
- VERSION = "0.1.1"
4
+ VERSION = "0.1.3"
5
5
  end
Binary file
File without changes
metadata CHANGED
@@ -1,29 +1,15 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: remote_database_importer
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Leon Vogt
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-10-24 00:00:00.000000000 Z
11
+ date: 2022-11-06 00:00:00.000000000 Z
12
12
  dependencies:
13
- - !ruby/object:Gem::Dependency
14
- name: thor
15
- requirement: !ruby/object:Gem::Requirement
16
- requirements:
17
- - - "~>"
18
- - !ruby/object:Gem::Version
19
- version: '1.2'
20
- type: :runtime
21
- prerelease: false
22
- version_requirements: !ruby/object:Gem::Requirement
23
- requirements:
24
- - - "~>"
25
- - !ruby/object:Gem::Version
26
- version: '1.2'
27
13
  - !ruby/object:Gem::Dependency
28
14
  name: tty-config
29
15
  requirement: !ruby/object:Gem::Requirement
@@ -70,8 +56,7 @@ description: Dump remote databases and import it locally. At the moment only Pos
70
56
  databases are supported
71
57
  email:
72
58
  - nonick@nonick.ch
73
- executables:
74
- - remote_database_importer
59
+ executables: []
75
60
  extensions: []
76
61
  extra_rdoc_files: []
77
62
  files:
@@ -83,16 +68,14 @@ files:
83
68
  - LICENSE.txt
84
69
  - README.md
85
70
  - Rakefile
86
- - bin/remote_database_importer
87
71
  - lib/railtie.rb
88
72
  - lib/remote_database_importer.rb
89
73
  - lib/remote_database_importer/config.rb
90
74
  - lib/remote_database_importer/operation.rb
91
75
  - lib/remote_database_importer/version.rb
92
76
  - lib/tasks/remote_database_importer.rake
93
- - readme_images/.DS_Store
94
- - readme_images/config_sample.png
95
- - readme_images/import-job.gif
77
+ - readme_assets/config_sample.png
78
+ - readme_assets/import-job.gif
96
79
  - sig/remote_database_importer.rbs
97
80
  homepage: https://github.com/leon-vogt/remote_database_importer
98
81
  licenses:
@@ -1,14 +0,0 @@
1
- #!/usr/bin/env ruby
2
-
3
- require 'thor'
4
- require_relative '../lib/remote_database_importer/operation'
5
-
6
- class RemoteDatabaseImporterCLI < Thor
7
- desc "Restore remote database and import it locally", "asdf"
8
- def import
9
- importer = RemoteDatabaseImporter::Operation.new
10
- importer.import
11
- end
12
- end
13
-
14
- RemoteDatabaseImporterCLI.start
Binary file
Binary file