subspace 3.0.0.rc1 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: df006855b51c155540821c2e5fd82968ebf79580e3e85f1149eeef576bb63672
-  data.tar.gz: 3548c10a76fa9117c130d73e198c42097373a1f3942f47fa47722741e2368f92
+  metadata.gz: b24a2573b737094142caaf93e8b7380b4f4bf756c611fe81012a9adcc0d8d026
+  data.tar.gz: c0fe4b9c23cf613c28ddf832a4bf67ac48055fcc51a32b1ffe8ebc4ad143d14a
 SHA512:
-  metadata.gz: c5ed6d9c43e07d68482e1481378e9295bda85c2b133cd50ac4fb18787c6d755945ca4f58c83031ade513e0d01233ee088032506d427f080a07730aaf9deb22df
-  data.tar.gz: 7d7779fe933a7297a4a682cc3a76fce740c74d2bd84306760ab7248e50c34251e3401039082900918a588660b1b82afd76b42072ae13bb23dafac37af2da4c9b
+  metadata.gz: a758a05af7793e9f5ee377f1008238678ee94d5fc3d5ff5e03289cb4b1736288e3d829efa36440b3744045358a5340ae4934107fd29a034bda9042b4cfcc9254
+  data.tar.gz: da042e9f7c0f6871ac50bb64c3525ef4ce54bdcebd9ed1b3bdb6d5a8602332dd93551a47f1fda97215c1180d9550e9f87bf0030453c1f9f5d388c3dc944eeaca

data/CHANGELOG.md

@@ -12,7 +12,17 @@ This project attempts to follow [semantic versioning](https://semver.org/).
 
 ## Unreleased
 
-## 3.0.0rc1
+## 3.0.0
+  * Install redis from vendor repos (BREAKING, see README)
+  * Removed outdated awscli role
+  * Added `subspace secrets rekey` to generate and rekey ansible-vault secrets
+  * Update tailscale role
+  * Don't default to using pemfile (use tailscale instead!)
+  * Add `subspace inventory keyscan` to fix ssh fingerprints
+  * Use `sidekiq_workers` var in systemd
+  * Tailscale hostname is now {{project_name}}-{{hostname}}
+
+## 3.0.0.rc1
   * Added infrastructure management via Terraform!
   * Added new `subspace exec` command for manual remote management
   * BREAKING: Consolidated inventory file into config/provision/inventory.env.yml

data/README.md

@@ -101,11 +101,23 @@ common     | authorized\_keys          | updates the authorized\_keys file for t
 rails      | appyml                    |
 monit      | monit                     | All tasks in the monit role have been tagged 'monit'
 
-### `subspace vars <environment> [--edit] [--create]`
+### `subspace secrets <environment> [--edit] [--create]`
 
-Manage environment variables on different platforms.  The default action is simply to show the vars defined for an environment.  Pass --edit to edit them in the system editor.
+The `secrets` command will manage encrypted secrets for different environments.  The default action is simply to show the secrets defined for an environment.  Pass --edit to edit them in the system editor (vim, etc).
 
-The new system uses a file in `config/provision/templates/application.yml.template` that contains environment variables for all environments.  The configuration that is not secret is visible and version controlled, while the secrets are stored in the vault files for their environments. The default file created by `subspace init`  looks like this:
+This uses `ansible-vault` under the hood and requires a vault password file.  You will need to get the `.vault_pass` from from a teammate out of band (secrets.10fw.ne, 1password, sticky-note, etc), and put it into `config/provision/.vault_pass`
+
+These secrets are used during provisioning to populate variables in a few different places:
+1. `config/application.yml`, which uses the `figaro` gem to manage environment variables in rails.
+2. `config/database.yml`, which handles the database connection password.
+
+
+Subspace uses a template file in `config/provision/templates/application.yml.template` that contains environment variables for all environments.  If you have non-secret variables that change based on the target server, you can simply put that in plaintext in the template file. This was designed so the configuration that is not secret is visible and version controlled, while the secret values are stored in the vault files for their environments.
+
+NOTE: application.yml should be in the `.gitignore`, since subspace creates a new version on the server and symlinks it on top of whatever is checked in. You should make changes to the template file instead, which should be checked in to version control.
+
+
+The default template created by `subspace init`  looks like this:
 
 ```
 # These environment variables are applied to all environments, and can be secret or not
@@ -113,7 +125,7 @@ The new system uses a file in `config/provision/templates/application.yml.templa
 # This is secret and can be changed on all three environment easily by using subspace vars <env> --edit
 SECRET_KEY_BASE: {{secret_key_base}}
 
-#This is not secret, and is the same value for all environments
+# This is not secret, and is the same value for all environments
 ENABLE_SOME_FEATURE: false
 
 development:
@@ -127,14 +139,12 @@ production:
 
 ```
 
-Further, you can use the extremely command to create a local copy of `config/application.yml`
+You can also use this command to automatically create a local version of `config/application.yml` based on the template and encrypted secrets for a specific environment.
 
     # Create a local copy of config/application.yml with the secrets encrypted in secrets/development.yml
     $ subspace vars development --create
 
-This can get you up and running in development securely, the only thing you need to distribute to new team members is the vault password.  Grab it from a teammate and put it into `config/provision/.vault_pass`
-
-NOTE: application.yml should be in the `.gitignore`, since subspace creates a new version on the server and symlinks it on top of whatever is checked in.
+This can get you up and running quickly in development securely.
 
 ## Procedure for updating on projects
 
@@ -146,13 +156,13 @@ Then,
 
 * `subspace provision production`
 
-If you get an error saying you need a vault password file, you should be able to find it in 1Password. You might also need to update `ansible`.
+If you get an error saying you need a vault password file, you need to get it from somoene on the team (see above). You might also need to update `ansible`.
 
 You'll want to do this for each environment (ie: `subspace provision qa`, etc.). Best to start with staging and work your way up.
 
 # Host configuration
 
-We need to know some info about hosts, but not much.  See the files for details, it's mostly the hostname and the user that can administer the system, eg `ubuntu` on AWS/ubuntu, `ec2-user`, or even `root` (not recommended)
+We need to know some info about hosts, but not much.  See the files for details, it's mostly the hostname and the user that can administer the system, eg `ubuntu` on AWS/ubuntu, `ec2-user`, or even `root` (not recommended, but used on linode/Digital Ocean)
 
 # Role Configuration
 
@@ -251,20 +261,34 @@ Defaults:
 
 ## letsencrypt
 
-By default, this creates a single certificate for every server alias/server name in the configuration file.
-If you'd like more control over the certs created, you can define the variables `le_ssl_certs` as follows:
+This creates a single certificate for every server alias/server name in the configuration file.
+
+    letsencrypt_email: "me@example.com"
+    server_name: app.example.com
+
+
+If you'd like more control over the cert, you can customize the variable `le_ssl_cert` as follows:
 
-    le_ssl_certs:
-      - cert_name: mycert
-        domains:
-          - mydomain.example.com
-          - otherdomain.example.com
-      - cert_name: othersite
-        domains:
-          - othersite.example.com
+    le_ssl_cert:
+      cert_name: "{{server_name}}"
+      preferred_challenges: "http"
+      plugin: standalone
+      domains: "{{ [server_name] + server_aliases }}"
+
+For example, to force a manual DNS challenge you can do the following:
+
+    le_ssl_cert:
+      cert_name: star_example
+      preferred_challenges: dns
+      plugin: manual
+      domains:
+        - example.com
+        - "*.example.com"
+
+(you will need to futz around the first time and manually install the DNS record, but it should work on renewals)
+
+Note that this role needs to be included _before_ the webserver (apache or nginx) role
 
-Note that this role needs to be included _before_ the webserver (apache or
-nginx) role
 
 ## logrotate
 
@@ -291,14 +315,6 @@ Installs memcache on the server.  By default, memcache will only listen on local
     # bind to all interfaces
     memcache_bind: "0.0.0.0"
 
-## monit
-
-## mysql
-
-## mysql2_gem
-
-## newrelic
-
 ## newrelic-infra
 This role will install the next-gen "Newrelic One" infrastructure agent which can perform a few different functions for newrelic.  The previous "newrelic" role is deprecated.
 
@@ -369,13 +385,13 @@ The full list of distributions is here: https://github.com/nodesource/distributi
     database_user: "{{project_name}}"
 
 ## puma
-Use the puma app server for your rails app.  Usually combined with nginx to server as a static file server and reverse proxy. 
+Use the puma app server for your rails app.  Usually combined with nginx to server as a static file server and reverse proxy.
 
 **Prerequesites:**
   - add `gem puma` to your gemfile
   - add `config/puma/` to the `linked_dirs` config in capistrano's `deploy.rb`
 
-This role will generate a reasonable `puma.rb` and configure it to be controlled by systemd. 
+This role will generate a reasonable `puma.rb` and configure it to be controlled by systemd.
 
 **Variables:**
 
@@ -387,7 +403,7 @@ This role will generate a reasonable `puma.rb` and configure it to be controlled
 
 Provisions for a rails app.  This one is probably pretty important.
 
-We no longer provider default values, so make sure to define all the following variables: 
+We no longer provider default values, so make sure to define all the following variables:
 
     rails_env: production
     database_pool: 5
@@ -407,6 +423,11 @@ Installs redis on the server.
     # Change to * if you want this available everywhere instead of localhost
     redis_bind: 127.0.0.1
 
+As of Subspace 3.0, this uses the official redis apt repo instead of the debian/ubuntu ones.  If you previously had installed redis from the distro, you will need to manually uninstall, purge, and reinstall.  This should not delete any data but back it up just in case.
+
+    sudo apt-get purge redis-server
+    sudo apt-get install redis-server
+
 ## resque
 
 Install monitoring and automatic startup for resque workers via monit. You MUST set the `job_queues` variable as follows:

data/ansible/roles/delayed_job/tasks/main.yml

@@ -1,45 +1,28 @@
 ---
   - set_fact: delayed_job_installed="true"
 
-  - name: Monit Stop All
-    shell: monit stop all
+  - name: Install systemd delayed_job script
     become: true
-    ignore_errors: yes
-
-  - name: Wait for monit to stop
-    shell: monit status | grep Monitored | wc -l | awk '{print $1 $2}'
-    register: monit_stopped
-    retries: 10
-    until: monit_stopped.stdout == "0"
-    delay: 10
-    become: true
-
-  - name: Install delayed_job monit script
+    vars:
+      job_queue: "{{ item }}"
+      loop_index: "{{ loop_index }}"
     template:
-      src: delayed-job-monit-rc
-      dest: /etc/monit/conf.d/delayed_job_{{project_name}}_{{rails_env}}
+      src: delayed-job-systemd.service
+      dest: /etc/systemd/system/delayed_job_{{ item }}{{ loop_index }}.service
+    loop: "{{ job_queues }}"
+    loop_control:
+      index_var: loop_index
+      loop_var: item
+
+  - name: Enable systemd delayed_job service
     become: true
+    systemd:
+      name: "delayed_job_{{ item }}{{ loop_index }}"
+      daemon_reload: true
+      enabled: yes
+      state: started
+    loop: "{{ job_queues }}"
+    loop_control:
+      loop_var: item
+      index_var: loop_index
 
-  - name: Remove old upstart files
-    file:
-      path: /etc/init/delayed-job.conf
-      state: absent
-    become: true
-
-  - name: Remove old monit files
-    file:
-      path: /etc/monit/conf.d/delayed_job
-      state: absent
-    become: true
-
-  - name: reload_monit
-    shell: monit reload
-    become: true
-
-  - name: wait
-    pause:
-      seconds: 3
-
-  - name: restart monit services
-    shell: monit restart all
-    become: true

data/ansible/roles/delayed_job/templates/delayed-job-systemd.service

@@ -0,0 +1,33 @@
+[Unit]
+Description=Start delayed_job_{{job_queue}}{{loop_index}} instance
+After=syslog.target network.target
+
+[Service]
+Type=simple
+
+# Uncomment this if you are going to use this as a system service
+# if using as a user service then leave commented out, or you will get an error trying to start the service
+# !!! Change this to your deploy user account if you are using this as a system service !!!
+User=deploy
+Group=deploy
+UMask=0002
+
+Environment=RAILS_ENV={{rails_env}}
+
+WorkingDirectory=/u/apps/{{project_name}}/current
+ExecStart=/usr/local/bin/bundle exec {{delayed_job_command}} --identifier={{job_queue}}{{loop_index}} --queue={{job_queue}} start
+ExecStop=/usr/local/bin/bundle exec {{delayed_job_command}} --identifier={{job_queue}}{{loop_index}} --queue={{job_queue}} stop
+TimeoutSec=120
+PIDFile=/u/apps/{{project_name}}/shared/tmp/pids/delayed_job_{{job_queue}}{{loop_index}}.pid
+
+# if we crash, restart
+RestartSec=1
+Restart=on-failure
+
+StandardOutput=syslog
+StandardError=syslog
+# This will default to "bundler" if we don't specify it
+SyslogIdentifier=delayed_job
+
+[Install]
+WantedBy=multi-user.target

data/ansible/roles/letsencrypt/defaults/main.yml

@@ -1,12 +1,12 @@
 ---
   certbot_dir: "/opt/certbot"
-  apache_ssl_config: |
-    SSLCertificateFile /etc/letsencrypt/live/{{server_name}}/cert.pem
-    SSLCertificateKeyFile /etc/letsencrypt/live/{{server_name}}/privkey.pem
-    Include /etc/letsencrypt/options-ssl-apache.conf
-    SSLCertificateChainFile /etc/letsencrypt/live/{{server_name}}/chain.pem
+  le_ssl_cert:
+    cert_name: "{{server_name}}"
+    preferred_challenges: "http"
+    plugin: standalone
+    domains: "{{ [server_name] + server_aliases }}"
 
   nginx_ssl_config: |
-    ssl_certificate /etc/letsencrypt/live/{{server_name}}/fullchain.pem;
-    ssl_certificate_key /etc/letsencrypt/live/{{server_name}}/privkey.pem;
+    ssl_certificate /etc/letsencrypt/live/{{le_ssl_cert.cert_name}}/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/{{le_ssl_cert.cert_name}}/privkey.pem;
     include /etc/letsencrypt/options-ssl-nginx.conf;

data/ansible/roles/letsencrypt/tasks/main.yml

@@ -41,15 +41,25 @@
       delay: 1
       state: stopped
 
-  - name: Run default
-    when: le_ssl_certs is not defined
+  - name: Generate SSL Certificate
     become: true
-    command: "{{certbot_bin}} certonly --email {{letsencrypt_email}} --domains {{([server_name] + server_aliases) | join(',')}} --cert-name {{server_name}} --standalone --agree-tos --expand --non-interactive"
-
-  - name: Generate SSL Certificates
-    become: true
-    with_items: "{{le_ssl_certs|default([])}}"
-    command: "{{certbot_bin}} certonly --email {{letsencrypt_email}} --domains {{item.domains | join(',')}} --cert-name {{item.cert_name}} --standalone --agree-tos --expand --non-interactive"
+    command:
+      argv:
+        - "{{ certbot_bin }}"
+        - certonly
+        - "--email"
+        - "{{ letsencrypt_email }}"
+        - "--domains"
+        - "{{ le_ssl_cert.domains | join(',') }}"
+        - "--preferred-challenges"
+        - "{{ le_ssl_cert.preferred_challenges }}"
+        - "--cert-name"
+        - "{{ le_ssl_cert.cert_name }}"
+        - "--{{ le_ssl_cert.plugin }}"
+        - "--manual-auth-hook=/bin/yes"
+        - "--agree-tos"
+        - "--expand"
+        - "--non-interactive"
 
   - name: Update nginx default options
     when: "'nginx' in role_names"
@@ -57,12 +67,6 @@
       url: https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf
       dest: /etc/letsencrypt/options-ssl-nginx.conf
 
-  - name: Update apache default options
-    when: "'apache' in role_names"
-    get_url:
-      url: https://raw.githubusercontent.com/certbot/certbot/master/certbot-apache/certbot_apache/options-ssl-apache.conf
-      dest: /etc/letsencrypt/options-ssl-apache.conf
-
   - name: start webserver after standalone mode
     debug: msg="Startup webserver"
     notify: start webserver
@@ -74,16 +78,6 @@
       env: yes
       job: /usr/bin:/bin:/usr/sbin
 
-  - name: Setup cron job to auto renew
-    become: true
-    when: "'apache' in role_names"
-    cron:
-      name: Auto-renew SSL
-      job: "{{certbot_bin}} renew --no-self-upgrade --apache >> /var/log/cron.log 2>&1"
-      hour: "0"
-      minute: "33"
-      state: present
-
   - name: Setup cron job to auto renew
     become: true
     when: "'nginx' in role_names"

data/ansible/roles/puma/templates/puma-systemd.service

@@ -23,6 +23,7 @@ WorkingDirectory=/u/apps/{{project_name}}/current
 
 # Helpful for debugging socket activation, etc.
 # Environment=PUMA_DEBUG=1
+Environment=RAILS_ENV={{rails_env}}
 
 # SystemD will not run puma even if it is in your path. You must specify
 # an absolute URL to puma. For example /usr/local/bin/puma

data/ansible/roles/redis/tasks/main.yml

@@ -1,9 +1,27 @@
 ---
-  - name: Install Redis
+  - name: Add an Apt signing key for redis repo
+    ansible.builtin.apt_key:
+      url: https://packages.redis.io/gpg
+      state: present
+
+  - name: Add redis repository into sources list
+    ansible.builtin.apt_repository:
+      repo: deb https://packages.redis.io/deb {{ ansible_distribution_release }} main
+      state: present
+    register: redis_apt_repo
+
+  - name: Purge distro redis package
+    apt:
+      name: redis-server
+      state: absent
+      purge: true
+    when: redis_apt_repo.changed
+
+  - name: Install Redis from official repo
     become: true
     apt:
-      pkg: redis-server
-      state: present
+      name: redis-server
+      state: latest
       update_cache: true
 
   - name: Set bind IP

data/ansible/roles/ruby-common/README.md

@@ -23,7 +23,7 @@ Role Variables
 
 > ruby_version: This variable controls the version of Ruby that will be compiled and installed. It should correspond with the tarball filename excluding the ".tar.gz" extension (e.g. "ruby-1.9.3-p484").
 
-> ruby_checksum: The SHA256 checksum of the gzipped tarball that will be downloaded and compiled.
+> ruby_checksum: The checksum of the gzipped tarball that will be downloaded and compiled. (prefix with algorithm, e.g. sha256:abcdef01234567890)
 
 > ruby_download_location: The URL that the tarball should be retrieved from. Using the ruby_version variable within this variable is a good practice (e.g. "http://cache.ruby-lang.org/pub/ruby/1.9/{{ ruby_version }}.tar.gz").
 

data/ansible/roles/ruby-common/tasks/main.yml

@@ -40,7 +40,7 @@
 - name: Download the Ruby source code
   get_url: url={{ ruby_download_location }}
            dest=/usr/local/src/
-           sha256sum={{ ruby_checksum }}
+           checksum={{ ruby_checksum }}
   become: true
 
 - name: Generate the Ruby installation script

data/ansible/roles/sidekiq/templates/sidekiq-systemd.service

@@ -32,7 +32,7 @@ Type=notify
 WatchdogSec=10
 
 WorkingDirectory=/u/apps/{{project_name}}/current
-ExecStart=/usr/local/bin/bundle exec sidekiq -e {{rails_env}} --queue {{hostname}} {{ job_queues | map('regex_replace',  '^(.*)$', '--queue \\1') | join(' ') }}
+ExecStart=/usr/local/bin/bundle exec sidekiq -e {{rails_env}} --queue {{hostname}} {{ job_queues | map('regex_replace',  '^(.*)$', '--queue \\1') | join(' ') }} -c {{sidekiq_workers}}
 
 # Use `systemctl kill -s TSTP sidekiq` to quiet the Sidekiq process
 
@@ -46,6 +46,7 @@ UMask=0002
 # Greatly reduce Ruby memory fragmentation and heap usage
 # https://www.mikeperham.com/2018/04/25/taming-rails-memory-bloat/
 Environment=MALLOC_ARENA_MAX=2
+Environment=RAILS_ENV={{rails_env}}
 
 # if we crash, restart
 RestartSec=1

data/ansible/roles/tailscale/tasks/main.yml

@@ -19,4 +19,4 @@
 
   - name: "Join the tailnet"
     become: true
-    command: tailscale up --auth-key {{tailscale_auth_key}} {{tailscale_options}}
+    command: tailscale up --ssh --auth-key={{tailscale_auth_key}} --hostname={{project_name}}-{{hostname}} --accept-risk=lose-ssh {{tailscale_options}}

data/bin/console

@@ -6,9 +6,5 @@ require "subspace"
 # You can add fixtures and/or initialization code here to make experimenting
 # with your gem easier. You can also use a different console, if you like.
 
-# (If you use this, don't forget to add pry to your Gemfile!)
-# require "pry"
-# Pry.start
-
 require "irb"
 IRB.start

data/lib/subspace/cli.rb

@@ -155,6 +155,7 @@ class Subspace::Cli
 
         capistrano - generate config/deploy/[env].rb.  Requires the --env option.
         list       - list the current inventory as understood by subspace.
+        keyscan    - Update ~/.known_hosts with new host key fingerprints
         EOS
       c.option "--env ENVIRONMENT", "Optional: Limit function to a specific environment (aka group)"
       c.when_called Subspace::Commands::Inventory

data/lib/subspace/commands/ansible.rb

@@ -3,18 +3,18 @@ module Subspace
     module Ansible
       def ansible_playbook(*args)
         args.push "--diff"
-        args.push "--private-key"
-        args.push "subspace.pem"
         ansible_command("ansible-playbook", *args)
       end
 
       def ansible_command(command, *args)
         update_ansible_cfg
+        retval = false
         Dir.chdir "config/subspace" do
           say ">> Running #{command} #{args.join(' ')}"
-          system(command, *args, out: $stdout, err: $stderr)
+          retval = system(command, *args, out: $stdout, err: $stderr)
           say "<< Done"
         end
+        retval
       end
 
       private

data/lib/subspace/commands/inventory.rb

@@ -8,6 +8,8 @@ class Subspace::Commands::Inventory < Subspace::Commands::Base
       capistrano_deployrb
     when "list"
       list_inventory
+    when "keyscan"
+      keyscan_inventory
     else
       say "Unknown or missing command to inventory: #{command}"
       say "try subspace inventory [list, capistrano]"
@@ -15,12 +17,19 @@ class Subspace::Commands::Inventory < Subspace::Commands::Base
   end
 
   def list_inventory
-    inventory.find_hosts!(@env || "all").each do |host_name|
-      host = inventory.hosts[host_name]
+    inventory.find_hosts!(@env || "all").each do |host|
       puts "#{host.name}\t#{host.vars["ansible_host"]}\t(#{host.group_list.join ','})"
     end
   end
 
+  def keyscan_inventory
+    inventory.find_hosts!(@env || "all").each do |host|
+      ip = host.vars["ansible_host"]
+      system %Q(ssh-keygen -R #{ip})
+      system %Q(ssh-keyscan -Ht ed25519 #{ip} >> "$HOME/.ssh/known_hosts")
+    end
+  end
+
   def capistrano_deployrb
     if @env.nil?
       puts "Please provide an environment e.g: subspace inventory capistrano --env production"
@@ -29,8 +38,8 @@ class Subspace::Commands::Inventory < Subspace::Commands::Base
 
     say "# config/deploy/#{@env}.rb"
     say "# Generated by Subspace"
-    inventory.find_hosts!(@env).each do |host_name|
-      host = inventory.hosts[host_name]
+    inventory.find_hosts!(@env).each do |host|
+      host = inventory.hosts[host.name]
       db_role = false
       roles = host.group_list.map do |group_name|
         if group_name =~ /web/

data/lib/subspace/commands/secrets.rb

@@ -1,15 +1,19 @@
 class Subspace::Commands::Secrets < Subspace::Commands::Base
   def initialize(args, options)
-    @environment = args.first
-    @action = if options.edit
-      "edit"
-    elsif options.create
-      "create"
+    if args.first == "rekey"
+      rekey
     else
-      "view"
-    end
+      @environment = args.first
+      @action = if options.edit
+        "edit"
+      elsif options.create
+        "create"
+      else
+        "view"
+      end
 
-    run
+      run
+    end
   end
 
   def run
@@ -40,6 +44,22 @@ class Subspace::Commands::Secrets < Subspace::Commands::Base
     system "cat", "config/application.yml"
   end
 
+  def rekey
+    secret_files = Dir.glob("config/subspace/secrets/*.yml").map {|x| "secrets/#{File.basename(x)}"}
+    exit unless agree("This will re-key your secrets with a new random vault_pass. (#{secret_files}).  Proceed? (yes to continue) ")
+
+
+    say "Writing new password to .vault_pass.new"
+    File.write "config/subspace/.vault_pass.new", SecureRandom.base64(24) + "\n"
+    success = ansible_command "ansible-vault", "rekey", "--vault-password-file", ".vault_pass", "--new-vault-password-file", ".vault_pass.new", "-v", *secret_files
+    if success
+      FileUtils.mv "config/subspace/.vault_pass", "config/subspace/.vault_pass.old"
+      FileUtils.mv "config/subspace/.vault_pass.new", "config/subspace/.vault_pass"
+    else
+      say "Something went wrong, not changing .vault_pass"
+    end
+  end
+
   private
 
   def application_yml_template

data/lib/subspace/commands/ssh.rb

@@ -18,11 +18,15 @@ class Subspace::Commands::Ssh < Subspace::Commands::Base
       return
     end
     host_vars = inventory.hosts[@host].vars
-    user = host_vars["ansible_user"]
+    if host_vars.key?('ansible_ssh_user')
+      say "Supposed to be ansible_user not ansible_ssh_user"
+    end
+    user = @user || host_vars["ansible_user"]
     host = host_vars["ansible_host"]
     port = host_vars["ansible_port"] || 22
-    pem = host_vars["ansible_ssh_private_key_file"] || 'subspace.pem'
-    cmd = "ssh #{user}@#{host} -p #{port} -i config/subspace/#{pem} #{pass_through_params.join(" ")}"
+    pem = host_vars["ansible_ssh_private_key_file"]
+    pem_cmd = "-i config/subspace/#{pem}" if pem
+    cmd = "ssh #{user}@#{host} -p #{port} #{pem_cmd} #{pass_through_params.join(" ")}"
     say "> #{cmd} \n"
     exec cmd
   end

data/lib/subspace/commands/terraform.rb

@@ -73,7 +73,7 @@ module Subspace
       def update_inventory
         puts "Apply succeeded, updating inventory."
         Dir.chdir "config/subspace/terraform/#{@env}" do
-          @output = JSON.parse `terraform output -json oxenwagen`
+          @output = JSON.parse `terraform output -json inventory`
         end
         inventory.merge(@output)
         inventory.write

data/lib/subspace/inventory.rb

@@ -60,20 +60,20 @@ module Subspace
     end
 
     def merge(inventory_json)
-      inventory_json["inventory"]["hostnames"].each_with_index do |host, i|
+      inventory_json["hostnames"].each_with_index do |host, i|
         if hosts[host]
           old_ip = hosts[host].vars["ansible_host"]
-          new_ip = inventory_json["inventory"]["ip_addresses"][i]
+          new_ip = inventory_json["ip_addresses"][i]
           if old_ip != new_ip
             say "  * Host '#{host}' IP address changed! You may need to update the inventory! (#{old_ip} => #{new_ip})"
           end
           next
         end
         hosts[host] = Host.new(host)
-        hosts[host].vars["ansible_host"] = inventory_json["inventory"]["ip_addresses"][i]
-        hosts[host].vars["ansible_user"] = inventory_json["inventory"]["users"][i]
+        hosts[host].vars["ansible_host"] = inventory_json["ip_addresses"][i]
+        hosts[host].vars["ansible_user"] = inventory_json["users"][i]
         hosts[host].vars["hostname"] = host
-        hosts[host].group_list = inventory_json["inventory"]["groups"][i].split(/\s/)
+        hosts[host].group_list = inventory_json["groups"][i].split(/\s/)
       end
     end
 

data/lib/subspace/version.rb

@@ -1,3 +1,3 @@
 module Subspace
-  VERSION = "3.0.0.rc1"
+  VERSION = "3.0.0"
 end

data/template/subspace/terraform/template/main-oxenwagen.tf.erb

@@ -63,7 +63,7 @@ module oxenwagen {
   # lb_alternate_names = []
 }
 
-output "oxenwagen" {
+output "inventory" {
   value = module.oxenwagen
 }
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: subspace
 version: !ruby/object:Gem::Version
-  version: 3.0.0.rc1
+  version: 3.0.0
 platform: ruby
 authors:
 - Brian Samson
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2022-07-29 00:00:00.000000000 Z
+date: 2023-01-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -113,7 +113,6 @@ files:
 - ansible/roles/apache/handlers/main.yml
 - ansible/roles/apache/tasks/main.yml
 - ansible/roles/apache/templates/server_status.conf
-- ansible/roles/awscli/tasks/main.yml
 - ansible/roles/collectd/defaults/main.yml
 - ansible/roles/collectd/handlers/main.yml
 - ansible/roles/collectd/tasks/main.yml
@@ -138,15 +137,13 @@ files:
 - ansible/roles/delayed_job/README.md
 - ansible/roles/delayed_job/defaults/main.yml
 - ansible/roles/delayed_job/handlers/main.yml
-- ansible/roles/delayed_job/meta/main.yml
 - ansible/roles/delayed_job/tasks/main.yml
 - ansible/roles/delayed_job/templates/delayed-job-monit-rc
+- ansible/roles/delayed_job/templates/delayed-job-systemd.service
 - ansible/roles/letsencrypt/defaults/main.yml
 - ansible/roles/letsencrypt/tasks/legacy.yml
 - ansible/roles/letsencrypt/tasks/main.yml
 - ansible/roles/letsencrypt/tasks/modern.yml
-- ansible/roles/letsencrypt_dns/defaults/main.yml
-- ansible/roles/letsencrypt_dns/tasks/main.yml
 - ansible/roles/logrotate/LICENSE
 - ansible/roles/logrotate/README.md
 - ansible/roles/logrotate/defaults/main.yml
@@ -312,11 +309,11 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
-rubygems_version: 3.3.18
+rubygems_version: 3.3.3
 signing_key:
 specification_version: 4
 summary: Ansible-based server provisioning for rails projects

data/ansible/roles/awscli/tasks/main.yml

@@ -1,10 +0,0 @@
----
-  - name: Install pip
-    apt:
-      pkg: python-pip
-      state: latest
-    become: true
-
-  - name: Install awscli
-    pip:
-      name: awscli

data/ansible/roles/delayed_job/meta/main.yml

@@ -1,5 +0,0 @@
----
-  dependencies:
-    - {
-      role: monit
-    }

data/ansible/roles/letsencrypt_dns/defaults/main.yml

@@ -1,4 +0,0 @@
----
-  nginx_ssl_config: |
-    ssl_certificate /etc/letsencrypt/live/{{server_name}}/fullchain.crt;
-    ssl_certificate_key /etc/letsencrypt/live/{{server_name}}/privkey.pem;

data/ansible/roles/letsencrypt_dns/tasks/main.yml

@@ -1,133 +0,0 @@
-- name: Update repositories cache and install pip and setuptools package
-  apt:
-    name: [python-pip, python-setuptools]
-    update_cache: yes
-
-- pip:
-    name: [pyopenssl, boto]
-  tags:
-    - cert
-
-- name: Creates private key directory
-  file:
-    path: "/etc/letsencrypt/live/{{ server_name }}"
-    state: directory
-  tags:
-    - cert
-
-- name: Generate an OpenSSL private key with the default values (4096 bits, RSA)
-  openssl_privatekey:
-    path: "/etc/letsencrypt/live/{{ server_name }}/privkey.pem"
-  register: privkey
-  tags:
-    - cert
-
-- name: Generate an OpenSSL account key with the default values (4096 bits, RSA)
-  openssl_privatekey:
-    path: "/etc/letsencrypt/live/{{ server_name }}/account.pem"
-  tags:
-    - cert
-
-- name: Generate an OpenSSL Certificate Signing Request
-  openssl_csr:
-    path: "/etc/letsencrypt/live/{{ server_name }}/server.csr"
-    privatekey_path: "/etc/letsencrypt/live/{{ server_name }}/privkey.pem"
-    country_name: US
-    email_address: "{{ letsencrypt_email }}"
-    subject_alt_name: "{{ item.value | map('regex_replace', '^', 'DNS:') | list }}"
-  when: privkey is changed
-  register: csr
-  with_dict:
-    dns_server:
-    - "{{ server_name }}"
-    - "*.{{ server_name }}"
-  tags:
-    - cert
-
-- name: Create a challenge using an account key from a variable.
-  acme_certificate:
-    acme_version: 2
-    account_key_src: "/etc/letsencrypt/live/{{ server_name }}/account.pem"
-    csr: "/etc/letsencrypt/live/{{ server_name }}/server.csr"
-    cert: "/etc/letsencrypt/live/{{ server_name }}/server.crt"
-    fullchain: "/etc/letsencrypt/live/{{ server_name }}/fullchain.crt"
-    chain: "/etc/letsencrypt/live/{{ server_name }}/intermediate.crt"
-    challenge: dns-01
-    acme_directory: https://acme-v02.api.letsencrypt.org/directory
-    terms_agreed: yes
-    remaining_days: 60
-  when: csr is changed
-  register: le_challenge
-  tags:
-    - cert
-
-- name: Install txt record on route53
-  route53:
-    zone: "{{ route53_zone }}"
-    type: TXT
-    ttl: 60
-    state: present
-    wait: yes
-    record: "{{ item.key }}"
-    value: "{{ item.value | map('regex_replace', '^(.*)$', '\"\\1\"' ) | list }}"
-    aws_access_key: "{{ AWS_ACCESS_KEY_ID }}"
-    aws_secret_key: "{{ AWS_SECRET_ACCESS_KEY }}"
-    overwrite: yes
-  loop: "{{ le_challenge.challenge_data_dns | default({}) | dict2items }}"
-  tags:
-    - cert
-
-- name: Flush dns cache
-  become: true
-  command: "systemd-resolve --flush-caches"
-  when: le_challenge is changed
-  tags:
-    - cert
-
-- name: "Wait for DNS"
-  when: le_challenge is changed
-  pause:
-    minutes: 2
-  tags:
-    - cert
-
-- name: Let the challenge be validated and retrieve the cert and intermediate certificate
-  acme_certificate:
-    acme_version: 2
-    account_key_src: "/etc/letsencrypt/live/{{ server_name }}/account.pem"
-    csr: "/etc/letsencrypt/live/{{ server_name }}/server.csr"
-    cert: "/etc/letsencrypt/live/{{ server_name }}/server.crt"
-    fullchain: "/etc/letsencrypt/live/{{ server_name }}/fullchain.crt"
-    chain: "/etc/letsencrypt/live/{{ server_name }}/intermediate.crt"
-    challenge: dns-01
-    acme_directory: https://acme-v02.api.letsencrypt.org/directory
-    remaining_days: 60
-    terms_agreed: yes
-    data: "{{ le_challenge }}"
-  when: le_challenge is changed
-  tags:
-    - cert
-
-- name: Delete txt record on route53
-  route53:
-    zone: "{{ route53_zone }}"
-    type: TXT
-    ttl: 60
-    state: absent
-    wait: yes
-    record: "{{ item.key }}"
-    value: "{{ item.value | map('regex_replace', '^(.*)$', '\"\\1\"' ) | list }}"
-    aws_access_key: "{{ AWS_ACCESS_KEY_ID }}"
-    aws_secret_key: "{{ AWS_SECRET_ACCESS_KEY }}"
-    overwrite: yes
-  loop: "{{ le_challenge.challenge_data_dns | default({}) | dict2items }}"
-  tags:
-    - cert
-
-- name: restart webserver
-  debug: msg="restart webserver"
-  notify: restart webserver
-  changed_when: true
-  when: le_challenge is changed
-  tags:
-    - cert