ops_backups 0.1.0 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c11d46d1083b3b1b45c330e4b74db480d82d80ebd49356a74934dd63409f2c01
-  data.tar.gz: d41344f6c9066beac9ba54ab9e96165fec31b33335b85af0ec26beebf2fba875
+  metadata.gz: 3024502438b6655823c65f47d783165f1d91718f98ee68b1b0ff945f20497ffe
+  data.tar.gz: d5dfa0b6d4b128a557213e30591d7121245b1544cf1f58a7617cfb102de61584
 SHA512:
-  metadata.gz: 64d6bb17c0bc7e78cf236d4a2077aa3cbb0cf520576544e8b75c16a9745c19e1b2a4886ad548c8446ac6405f1e366c17873ac540a9ec641e98a45002cf5b4571
-  data.tar.gz: 93ba27887a32e0a361187ef94189c6a574849f65fe5542a0f09fe329c1bfb42121675258ed97a8ce9b85136a9634902028869eb9e945b7c4163307556e3b7150
+  metadata.gz: 420151cb8f9cd51361d45525f80409ea535957f9c54702826ef93f5e32f370515a4ffd3e799a8e9c4731db9b8f2fa616a40702ca0421d5e334d313cdcb0aa5a5
+  data.tar.gz: 329f8fbbbcda7b9052cc4760afce2141ddb9120ad77dfd08c5fcc066cfdcb5b71f0e1e10b3a27157ade75dd3e8ece8635adf59db1a5b141f821c3efd751b23e2

data/README.md

@@ -1,132 +1,35 @@
-# Ops
+# OpsBackups
 
-## Description
-Ops is a Ruby gem designed to enhance and to backup your PostgreSQL database.
+A Ruby gem that provides a simple way to backup (self)-hosted postgres databases to
+ActiveStorage services like S3, Google Cloud Storage, etc.
 
 ## Usage
 
-### Configuration
-
-To configure the Ops gem, you need to set up the following environment variables:
-- `DATABASE_URL`: The URL of the PostgreSQL database to backup.
-
-Set up Active Storage for backups (e.g., Amazon S3 for production). We expect a service named `:backup_storage` to be configured.
-
-```yaml
-backup_storage:
-  service: S3
-  access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
-  secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
-  region: <%= ENV['AWS_REGION'] %>
-  bucket: backups
-```
-### Backup Method
-
-The `Ops::Backup` class provides a method to perform a PostgreSQL database backup:
-
-- `db_pg_backup(exclude_tables: [], tag: nil)`: This method performs a full backup of the database, optionally excluding specified tables. The backup is tagged and saved as an attachment.
-
-example:
+## Installation
 
-```ruby
-Ops::Backup.new.db_pg_backup(exclude_tables: ["versions"], tag: "db_pg_unveresioned")
+```bash
+bundle add ops_backups
+# install migrations
+# configure solid queu recurring job for a basic backup
+rails generate ops_backups:install
 ```
 
-### Backup Retention Strategies
-
-Ops gem provides two backup retention strategies:
-
-1. **Tiered Cleanup Policy**:
-    - Keeps all backups from the last day.
-    - Keeps the last backup of each day from the last week (except the last day).
-    - Keeps the last backup of each week from the last month (except the last week).
-    - Keeps the last backup of each month before the last month.
-
-    example:
-    ```ruby
-    Ops::CleanupTiered.new.cleanup("db_pg_unversioned")
-    ```
-
-    This policy can be applied using the `Ops::CleanupTieredJob`.
-
-2. **Limit Cleanup Policy**:
-    - Keeps the last N backups, where N is specified by the `limit` parameter.
-
-    example:
-    ```ruby
-    Ops::CleanupLimit.new.cleanup("db_pg_unversioned", limit: 14)
-    ```
-
-    This policy can be applied using the `Ops::CleanupLimitJob`.
-
-### Using SolidQueue Schedule
+# ActiveAdmin Integration
 
-1. Schedule the backup job using SolidQueue:
-    ```ruby
-    SolidQueue.schedule(Ops::BackupDbJob.new, cron: '0 0 * * *', args: { tag: "db_pg_full", exclude_tables: [], cleanup: "retain_last_limit" }) # This schedules the backup job to run daily at midnight
-    ```
+If you are using ActiveAdmin, you can manage your backups through the admin interface. The generator rails generate ops_backups:activeadmin will create the necessary example configuration.
 
-2. Schedule the cleanup job with a tiered policy:
-    ```ruby
-    SolidQueue.schedule(Ops::CleanupTieredJob.new, cron: '0 1 * * *', args: { tag: "db_pg_full" }) # This schedules the tiered cleanup job to run daily at 1 AM
-    ```
-
-3. Schedule the cleanup job with a limit policy:
-    ```ruby
-    SolidQueue.schedule(Ops::CleanupLimitJob.new, cron: '0 2 * * *', args: { tag: "db_pg_full", limit: 14 }) # This schedules the limit cleanup job to run daily at 2 AM
-    ```
-
-### Using `recurring.yml` from SolidQueue
-
-Alternatively, you can configure recurring tasks using `recurring.yml` in your SolidQueue configuration:
-
-1. Create or update the `recurring.yml` file in your application's config directory:
-    ```yaml
-    # filepath: config/recurring.yml
-    backup_db_job:
-      cron: "0 0 * * *"
-      class: "Ops::BackupDbJob"
-      args:
-        tag: "db_pg_full"
-        exclude_tables: []
-        cleanup: "retain_last_limit"
-
-    cleanup_tiered_job:
-      cron: "0 1 * * *"
-      class: "Ops::CleanupTieredJob"
-      args:
-        tag: "db_pg_full"
-
-    cleanup_limit_job:
-      cron: "0 2 * * *"
-      class: "Ops::CleanupLimitJob"
-      args:
-        tag: "db_pg_full"
-        limit: 14
-    ```
-
-
-
-## Installation
-Add this line to your application's Gemfile:
-
-```ruby
-gem "ops"
-```
-
-And then execute:
 ```bash
-$ bundle
+# add an active admin resource
+rails generate ops_backups:activeadmin
 ```
 
-Or install it yourself as:
-```bash
-$ gem install ops
-```
+## Available Actions
+
+- Download Backup: Download the backup file.
+- Create Versioned Backup: Create a new versioned backup.
+- Create Unversioned Backup: Create a new unversioned backup, excluding specific tables.
 
-## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/koenhandekyn/ops.
 
 ## License
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/app/controllers/ops_backups/application_controller.rb

@@ -0,0 +1,4 @@
+module OpsBackups
+  class ApplicationController < ActionController::Base
+  end
+end

data/app/helpers/{ops → ops_backups}/application_helper.rb

@@ -1,4 +1,4 @@
-module Ops
+module OpsBackups
   module ApplicationHelper
   end
 end

data/app/jobs/ops_backups/application_job.rb

@@ -0,0 +1,4 @@
+module OpsBackups
+  class ApplicationJob < ActiveJob::Base
+  end
+end

data/app/jobs/{ops/backups → ops_backups}/backup_db_job.rb

@@ -1,15 +1,18 @@
 # typed: true
 # frozen_string_literal: true
 
-class Ops::Backups::BackupDbJob < ApplicationJob
+class OpsBackups::BackupDbJob < ApplicationJob
   queue_as :operations
 
   # perform a full backup of the database
   # @param tag [String] the tag to assign to the backup
   # @param exclude_tables [Array<String>] the list of tables to exclude from the backup
   # @param cleanup_policy [String] the cleanup policy to apply to the backup, one of "retain_tiered_cleanup_policy" or "retain_last_limit_cleanup_policy"
-  def perform(tag: "db_pg_full", exclude_tables: [], cleanup: nil)
-    Ops::Backup.new.db_pg_backup(tag:, exclude_tables:)
-    Ops::Backup.send("#{cleanup}_cleanup_policy", tag: tag) if cleanup.present?
+  def perform(options = {})
+    exclude_tables = options[:exclude_tables] || []
+    tag = options[:tag] || (exclude_tables.empty? ? "db_pg_full" : "db_pg_partial")
+    cleanup = options[:cleanup]
+    OpsBackups::Backup.new.db_pg_backup(exclude_tables:, tag:)
+    OpsBackups::Backup.send("#{cleanup}_cleanup_policy", tag: tag) if cleanup.present?
   end
 end

data/app/jobs/{ops/backups → ops_backups}/cleanup_limit_job.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Cleanup job for limit backup policy
-class Ops::Backups::CleanupLimitJob < ApplicationJob
+class OpsBackups::CleanupLimitJob < ApplicationJob
   queue_as :operations
 
   # @param [String] tag
@@ -10,6 +10,6 @@ class Ops::Backups::CleanupLimitJob < ApplicationJob
   #
   # @example Tasks::CleanupLimit.perform_now(tag: "db_pg_full", limit: 14)
   def perform(tag: "db_pg_full", limit: 14)
-    Ops::Backup.retain_last_limit_cleanup_policy(tag: tag, limit: limit)
+    OpsBackups::Backup.retain_last_limit_cleanup_policy(tag: tag, limit: limit)
   end
 end

data/app/jobs/{ops/backups → ops_backups}/cleanup_tiered_job.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Cleanup job for tiered backup policy
-class Ops::Backups::CleanupTieredJob < ApplicationJob
+class OpsBackups::CleanupTieredJob < ApplicationJob
   queue_as :operations
 
   # @param [String] tag
@@ -9,6 +9,6 @@ class Ops::Backups::CleanupTieredJob < ApplicationJob
   #
   # @example Tasks::CleanupTiered.perform_now(tag: "db_pg_full")
   def perform(tag: "db_pg_full")
-    Ops::Backup.retain_tiered_cleanup_policy(tag: tag)
+    OpsBackups::Backup.retain_tiered_cleanup_policy(tag: tag)
   end
 end

data/app/mailers/ops_backups/application_mailer.rb

@@ -0,0 +1,6 @@
+module OpsBackups
+  class ApplicationMailer < ActionMailer::Base
+    default from: "from@example.com"
+    layout "mailer"
+  end
+end

data/app/models/ops_backups/application_record.rb

@@ -0,0 +1,5 @@
+module OpsBackups
+  class ApplicationRecord < ActiveRecord::Base
+    self.abstract_class = true
+  end
+end

data/app/models/{ops/backups → ops_backups}/backup.rb

@@ -1,24 +1,36 @@
-module Ops
-  puts "Ops module loaded"
-
-  class Ops::Backups::Backup < ApplicationRecord
+module OpsBackups
+  class Backup < ActiveRecord::Base
     # has_one_attached :backup_file, service: :backup_storage
-    has_one_attached :backup_file
+    has_one_attached :backup_file, service: :backups
+    self.table_name = "ops_backups"
 
     default_scope { order(updated_at: :desc) }
 
-    def db_pg_backup(exclude_tables: [], tag: nil)
-      tag ||= exclude_tables.empty? ? "db_pg_full" : "db_pg_partial"
-      Rails.logger.info("Backing up database, skipping tables: \#{exclude_tables.join(", ")}")
+    def self.ransackable_attributes(auth_object = nil)
+      [ "created_at", "id", "name", "new_id", "tag", "updated_at" ]
+    end
+
+    def self.test
+      "self.test"
+    end
+
+    def hoho
+      "hihi"
+    end
+
+    def db_pg_backup(tag: nil, exclude_tables: [])
       db_url = ENV["DATABASE_URL"]
+      tag ||= exclude_tables.empty? ? "db_pg_full" : "db_pg_partial" # if tag.empty?
       self.tag = tag
-      self.name = filename = "pg_\#{db_url.split('/').last}_backup_\#{Time.now.to_i}.dump"
+      self.name = "pg_#{db_url.split('/').last}_backup_#{Time.now.to_i}.dump"
       save!
-
-      Tempfile.new("pgbackup") do |tempfile|
+      Rails.logger.info("Backing up database")
+      # exclude_tables = []
+      filename = self.name
+      Tempfile.open("pgbackup") do |tempfile|
         begin
           excluded_tables_param = exclude_tables.map { |table| "--exclude-table-data=\#{table}" }.join(" ")
-          command = ["pg_dump", "--no-owner", excluded_tables_param, "-v", "-Fc", "-f", tempfile.path, db_url]
+          command = [ "pg_dump", "--no-owner", excluded_tables_param, "-v", "-Fc", "-f", tempfile.path, db_url ].reject(&:empty?)
 
           stdout, stderr, status = Open3.capture3(*command)
 

data/app/views/layouts/{ops → ops_backups}/application.html.erb

@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <html>
 <head>
-  <title>Ops</title>
+  <title>Ops backups</title>
   <%= csrf_meta_tags %>
   <%= csp_meta_tag %>
 
   <%= yield :head %>
 
-  <%= stylesheet_link_tag "ops/application", media: "all" %>
+  <%= stylesheet_link_tag    "ops_backups/application", media: "all" %>
 </head>
 <body>
 

data/config/routes.rb

@@ -1,2 +1,2 @@
-Ops::Backups::Engine.routes.draw do
+OpsBackups::Engine.routes.draw do
 end

data/lib/generators/ops_backups/activeadmin/activeadmin_generator.rb

@@ -0,0 +1,10 @@
+class OpsBackups::ActiveadminGenerator < Rails::Generators::Base
+  source_root File.expand_path("templates", __dir__)
+
+  desc "Copies the ActiveAdmin backup configuration to your application."
+
+  def copy_admin_backup
+    copy_file "backup.rb", Rails.root.join("app", "admin", "ops_backups", "backup.rb")
+    say "Copied admin/backup.rb to app/admin/ops_backups/backup.rb.", :green
+  end
+end

data/lib/generators/ops_backups/activeadmin/templates/backup.rb

@@ -0,0 +1,42 @@
+ActiveAdmin.register OpsBackups::Backup do
+  menu parent: "Ops", label: I18n.t("admin.ops.backup")
+
+  # Set the default sort order
+  config.sort_order = "updated_at DESC"
+
+  actions :index, :destroy
+
+  filter :name
+  filter :tag
+
+  index title: I18n.t("admin.ops.backup") do
+    selectable_column
+    column :name do |backup|
+      link_to(backup.name, download_backup_admin_ops_backups_backup_path(backup), class: "member_link")
+    end
+    column :tag
+    column :size do |backup|
+      backup.backup_file.attached? ? number_to_human_size(backup.backup_file.byte_size) : "N/A"
+    end
+    column :updated_at
+    column :duration do |backup|
+      Time.at((backup.updated_at - backup.created_at)).utc.strftime("%H:%M:%S")
+    end
+    actions
+  end
+
+  member_action :download_backup, method: :get do
+    redirect_to resource.backup_file.url(disposition: :attachment), allow_other_host: true
+  end
+
+  # an action that creates a new backup
+  collection_action :backup_db, method: :post do
+    OpsBackups::BackupDbJob.perform_later(tag: "db_pg_full")
+    redirect_to admin_ops_backups_backups_path, notice: I18n.t("admin.ops.backup_scheduled")
+  end
+
+  # add a button to the top of the index page
+  action_item :backup_db, only: :index do
+    link_to(I18n.t("admin.ops.backup_db"), backup_db_admin_ops_backups_backups_path, method: :post, class: "action-item-button")
+  end
+end

data/lib/generators/ops_backups/install/USAGE

@@ -0,0 +1,8 @@
+Description:
+    Explain the generator
+
+Example:
+    bin/rails generate install Thing
+
+    This will create:
+        what/will/it/create

data/lib/generators/ops_backups/install/install_generator.rb

@@ -0,0 +1,57 @@
+class OpsBackups::InstallGenerator < Rails::Generators::Base
+  source_root File.expand_path("templates", __dir__)
+
+  desc "Copies migrations and adds a storage service configuration to storage.yml"
+
+  def copy_migrations
+    rake "railties:install:migrations FROM=ops_backups"
+    say "Migrations copied to your application.", :green
+  end
+
+  def add_storage_service
+    storage_file = Rails.root.join("config", "storage.yml")
+
+    if File.exist?(storage_file)
+      service_config = <<~YAML
+
+        backups:
+          service: Disk
+          root: <%= Rails.root.join("storage") %>
+          # service: S3
+          # access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
+          # secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
+          # region: <%= ENV['AWS_REGION'] %>
+          # bucket: allcrux-backups
+      YAML
+
+      append_to_file storage_file, service_config
+      say "Added 'backups' service to storage.yml.", :green
+    else
+      say "config/storage.yml not found. Please create it and re-run this generator.", :red
+    end
+  end
+
+  def add_recurring_jobs
+    recurring_file = Rails.root.join("config", "recurring.yml")
+
+    if File.exist?(recurring_file)
+      recurring_config = <<~YAML.lines.map { |line| "  #{line}" }.join
+        backup_db:
+          class: OpsBackups::BackupDbJob
+          args: [tag: "db_pg_backup", cleanup: "retain_tiered_cleanup_policy"]
+          schedule: every hour
+      YAML
+
+      append_to_file recurring_file, recurring_config
+      say "Added 'recurring' jobs to recurring.yml.", :green
+    else
+      say "config/recurring.yml not found. Please create it and re-run this generator.", :red
+    end
+
+    # # copy the admin/backup.rb file to app/admin/ops_backups/backup.rb
+    # def copy_admin_backup
+    #   copy_file "admin/backup.rb", Rails.root.join("app", "admin", "ops_backups", "backup.rb")
+    #   say "Copied admin/backup.rb to app/admin/ops_backups/backup.rb.", :green
+    # end
+  end
+end

data/lib/ops_backups/engine.rb

@@ -0,0 +1,11 @@
+module OpsBackups
+  class Engine < ::Rails::Engine
+    isolate_namespace OpsBackups
+
+    initializer :append_migrations do |app|
+      unless app.root.to_s.match? root.to_s
+        app.config.paths["db/migrate"].concat(config.paths["db/migrate"].expanded)
+      end
+    end
+  end
+end

data/lib/ops_backups/version.rb

@@ -0,0 +1,3 @@
+module OpsBackups
+  VERSION = "0.1.2"
+end

data/lib/ops_backups.rb

@@ -0,0 +1,6 @@
+require "ops_backups/version"
+require "ops_backups/engine"
+
+module OpsBackups
+  # Your code goes here...
+end

data/lib/tasks/{ops_tasks.rake → ops_backups_tasks.rake}

@@ -1,4 +1,4 @@
 # desc "Explaining what the task does"
-# task :ops do
+# task :ops_backups do
 #   # Task goes here
 # end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ops_backups
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.2
 platform: ruby
 authors:
 - Koen Handekyn
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-11-17 00:00:00.000000000 Z
+date: 2024-11-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -35,22 +35,27 @@ files:
 - MIT-LICENSE
 - README.md
 - Rakefile
-- app/assets/stylesheets/ops/application.css
-- app/controllers/ops/backups/application_controller.rb
-- app/helpers/ops/application_helper.rb
-- app/jobs/ops/backups/application_job.rb
-- app/jobs/ops/backups/backup_db_job.rb
-- app/jobs/ops/backups/cleanup_limit_job.rb
-- app/jobs/ops/backups/cleanup_tiered_job.rb
-- app/models/ops/backups/application_record.rb
-- app/models/ops/backups/backup.rb
-- app/views/layouts/ops/application.html.erb
+- app/assets/stylesheets/ops_backups/application.css
+- app/controllers/ops_backups/application_controller.rb
+- app/helpers/ops_backups/application_helper.rb
+- app/jobs/ops_backups/application_job.rb
+- app/jobs/ops_backups/backup_db_job.rb
+- app/jobs/ops_backups/cleanup_limit_job.rb
+- app/jobs/ops_backups/cleanup_tiered_job.rb
+- app/mailers/ops_backups/application_mailer.rb
+- app/models/ops_backups/application_record.rb
+- app/models/ops_backups/backup.rb
+- app/views/layouts/ops_backups/application.html.erb
 - config/routes.rb
 - db/migrate/20241114173612_create_ops_backups.rb
-- lib/ops/backups/backups.rb
-- lib/ops/backups/engine.rb
-- lib/ops/backups/version.rb
-- lib/tasks/ops_tasks.rake
+- lib/generators/ops_backups/activeadmin/activeadmin_generator.rb
+- lib/generators/ops_backups/activeadmin/templates/backup.rb
+- lib/generators/ops_backups/install/USAGE
+- lib/generators/ops_backups/install/install_generator.rb
+- lib/ops_backups.rb
+- lib/ops_backups/engine.rb
+- lib/ops_backups/version.rb
+- lib/tasks/ops_backups_tasks.rake
 homepage: https://github.com/koenhandekyn/ops-backups
 licenses:
 - MIT

data/app/controllers/ops/backups/application_controller.rb

@@ -1,6 +0,0 @@
-module Ops
-  module Backups
-    class ApplicationController < ActionController::Base
-    end
-  end
-end

data/app/jobs/ops/backups/application_job.rb

@@ -1,6 +0,0 @@
-module Ops
-  module Backups
-    class ApplicationJob < ActiveJob::Base
-    end
-  end
-end

data/app/models/ops/backups/application_record.rb

@@ -1,8 +0,0 @@
-module Ops
-  module Backups
-    class ApplicationRecord < ActiveRecord::Base
-      self.abstract_class = true
-      self.table_name_prefix = "ops_"
-    end
-  end
-end

data/lib/ops/backups/backups.rb

@@ -1,8 +0,0 @@
-require "ops/backups/version"
-require "ops/backups/engine"
-
-module Ops
-  module Backups
-    # Your code goes here...
-  end
-end

data/lib/ops/backups/engine.rb

@@ -1,7 +0,0 @@
-module Ops
-  module Backups
-    class Engine < ::Rails::Engine
-      isolate_namespace Ops::Backups
-    end
-  end
-end

data/lib/ops/backups/version.rb

@@ -1,5 +0,0 @@
-module Ops
-  module Backups
-    VERSION = "0.1.0"
-  end
-end