bup 0.2.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 02ca6df7a29ad6290ab682bd214dc097c70e6a7a464bbbea22b1b4d100465ec7
-  data.tar.gz: 7bf6a98055fed3353f131e56033c876860ec6f4e22b995c4f002ffedd5e8462b
+  metadata.gz: 3228f83fe11f72b1e77210809dec6e3d8a4ccbdc849fa756c97f525803e04357
+  data.tar.gz: ea48181ef6b1adb871c2b558a6bfb9bdb50ef364459ec0fc453837d33dc57589
 SHA512:
-  metadata.gz: f96824b5eaa8fa4089497aa3eff4daf4abf5f041931fac622ac0163aed57d79036cced77b50a771fb5c675b6abaa1473e1018b9312c8e8d5935d12fc6255c601
-  data.tar.gz: 31aaf6faab04f3e7806bc60b082aac8bd0177aa4ade5fb40e02189289e75fef8d401aaedfce4f4abd4454aeb6679c881af0391dc732169d6eb362a820c970c9a
+  metadata.gz: 106db1b2e171e056b9bf19763f269898a2b5eeea115ce30918353fab95147850d124ffb53bcb64dfd35c46f220c71019ac0de8a3276263f93fc79c39a42a53b6
+  data.tar.gz: 64217767bfc2cbcfe73b7c1963175481cdfa4d8318ee2e2f4f92317bccddb1690244e48666891a78b5aa1d61da8876f90a80c8495850c62a3800619b2a4983d4

data/.rubocop.yml

@@ -1,5 +1,8 @@
 AllCops:
   TargetRubyVersion: 3.0
+  Exclude:
+    - lib/bup/tar.rb
+    - lib/bup/config.rb
 
 Style/StringLiterals:
   Enabled: true

data/CHANGELOG.md

@@ -1,5 +1,32 @@
 ## [Unreleased]
 
+## [0.6.0] - 2021-11-29
+
+- Add pre_cmds list to allow executing command before backup, such as dumping a database.
+- Add BUP_DESTINATION that holds the backup destination directory.
+  This may be used to put dumped database files in before a backup and clean 
+  them up with a post_cmd.
+- Refactor the tar.rb file for more simple individual function calls
+  and to do a little less redundant processing.
+- Replaced --type=[full|incremental] with --incremental and --full options.
+
+## [0.5.0] - 2021-11-13
+
+- Allow for a history size of 0.
+- Add default_profile setting.
+
+## [0.4.0] - 2021-11-13
+
+- Bug fix for empty post_cmds list.
+
+## [0.3.0] - 2021-11-13
+
+- Communicate the backup file base name through the environment variable BUP_FILENAME.
+- Rubocop fixes or ignore files.
+- Add post commands, incase you want to default copy something around.
+
+## [0.2.0] - 2021-11-13
+
 - List profiles.
 - Bug fixes against empty backup directories.
 - Bug fix to print buffers using read_nonblock from popen3 streams.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    bup (0.2.0)
+    bup (0.5.0)
 
 GEM
   remote: https://rubygems.org/

data/README.md

@@ -1,43 +1,66 @@
 # Bup
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/bup`. To experiment with that code, run `bin/console` for an interactive prompt.
+Bup is a simple backup driver script that uses `tar` to backup files.
+The idea is to simplify creating backups, full and incremental, and leave
+restoration as a project for the user to untar the file where their contents
+is needed.
 
-TODO: Delete this and the text above, and describe your gem
-
-## Installation
-
-Add this line to your application's Gemfile:
-
-```ruby
-gem 'bup'
-```
-
-And then execute:
-
-    $ bundle install
-
-Or install it yourself as:
-
-    $ gem install bup
+Features like backup profiles, file dating, history rotation,
+incremental backups, and post-processing scripting is provided.
 
 ## Usage
 
-TODO: Write usage instructions here
-
-## Development
-
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
-
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+Create a file ~/.buprc with the default contents...
+
+```yaml
+---
+default_profile: default
+profiles:
+  default:
+    description: Simple backup of critical files.
+    include:
+    - "$HOME"
+    - /$BUP_DESTINATION/db.dump
+    exclude:
+    - "$HOME"
+    - "$HOME/backups"
+    lastrun: '2021-11-14T03:06:45 +0000'
+    destination: "$HOME/backups"
+    history: 2
+    tarcmd:
+    - tar
+    - cJvf
+    - "${BUP_FILENAME}.tar.xz"
+    pre_cmds:
+    - - dumpdatabase
+      - /$BUP_DESTINATION/db.dump
+    post_cmds:
+    - - ls
+      - $BUP_FILENAME.tar.xz
+    - - rm
+      - /$BUP_DESTINATION/db.dump
+```
 
-## Contributing
+This defines a single profile, "default". You can list
+defined profiles by running `bup -l`. You run the default backup profile
+simply by running `bup`. Other profiles must be selected on the command
+line using `bup -p <profile name>`.
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/bup. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/bup/blob/main/CODE_OF_CONDUCT.md).
+The default profile, as defined above, excludes both the `$HOME` and
+`$HOME/backups` directories. Remove `$HOME` from the excludes list to actually
+backup your entire home directory.
 
-## License
+## Incremental Backups
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+By calling `bup -t incremental` an incremental backup will be run.
+This runs `tar` with the `--newer` option included with the time
+specified in the profile's `lastrun` setting. `Tar` will then exclude
+files older than that time.
 
-## Code of Conduct
+## Backup History
 
-Everyone interacting in the Bup project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/bup/blob/main/CODE_OF_CONDUCT.md).
+Before a new backup is run, previous backups are removed. If you run many 
+uncompleted backups, you will eventually remove all good backups from your
+backup history. This behavior is desired because backups should nearly always
+succeed and we want to ensure that disk pressure is kep lower than higher.
+Higher disk pressure might cause a backup to fail.

data/buprc

@@ -1,15 +1,21 @@
 ---
+default_profile: default
 profiles:
   default:
-    description: "Simple backup of critical files."
+    description: Simple backup of critical files.
     include:
     - "$HOME"
     exclude:
     - "$HOME"
     - "$HOME/backups"
-    lastrun: '2021-11-11T18:45:33 +0000'
+    lastrun: '2021-11-14T03:06:45 +0000'
     destination: "$HOME/backups"
     history: 2
     tarcmd:
     - sudo
     - tar
+    - cJvf
+    - "${BUP_FILENAME}.tar.xz"
+    post_cmds:
+    - - ls
+      - $BUP_FILENAME.tar.xz

data/exe/bup

@@ -6,6 +6,7 @@ require "optparse"
 
 configfile = File.join(ENV["HOME"], ".buprc")
 config = Bup::Config.new
+profile = nil
 
 OptParse.new do |opt|
   opt.banner = "#{$PROGRAM_NAME} #{Bup::VERSION}"
@@ -15,21 +16,32 @@ OptParse.new do |opt|
   end
 
   opt.on("-p", "--profile=profile", String, "Profile name.") do |p|
-    config.runtime["profile"] = p
+    profile = p
   end
 
-  opt.on("-t", "--type=type", String, "Type of backup.") do |t|
-    config.runtime["type"] = t == "incremental" ? t : "full"
+  opt.on("-i", "--incremental", String, "Set the type of backup to incremental.") do 
+    config.runtime["type"] = "incremental"
+  end
+
+  opt.on("-f", "--full", String, "Set the type of backup to full.") do 
+    config.runtime["type"] = "full"
   end
 
   opt.on("-l", "--list", "List profiles and exit.") do
-    config.runtime['action'] = 'list'
+    config.runtime["action"] = "list"
   end
 end.parse!
 
 config.load(configfile) if File.exist?(configfile)
 
-case config.runtime["action"] 
+if profile
+  config.runtime["profile"] = profile
+elsif config.config["default_profile"]
+  config.runtime["profile"] = config.config["default_profile"]
+end
+
+
+case config.runtime["action"]
 when "list"
   config.config["profiles"].each do |key, profile|
     puts "#{key} - #{profile["description"] || ""}"
@@ -38,4 +50,4 @@ else
   tar = Bup::Tar.new(config)
   tar.call
   config.save(configfile)
-end
+end

data/lib/bup/config.rb

@@ -5,12 +5,13 @@ require "yaml"
 require "date"
 
 module Bup
+  # Configuration management.
   class Config
-    @@timeformat = "%Y-%m-%dT%H:%M:%S %z"
 
     attr_accessor :config, :runtime
 
     def initialize
+      @timeformat = "%Y-%m-%dT%H:%M:%S %z"
       @runtime = {
         "profile" => "default",
         "type" => "full"
@@ -45,14 +46,14 @@ module Bup
 
     # Return the last run time of the backup or nil if there is none.
     def lastrun(name)
-      DateTime.strptime(profile(name)["lastrun"] || "", @@timeformat)
+      DateTime.strptime(profile(name)["lastrun"] || "", @timeformat)
     rescue Date::Error
       nil
     end
 
     # Return the last run time of the backup or nil if there is none.
     def set_lastrun(name, date)
-      profile(name)["lastrun"] = date.strftime(@@timeformat)
+      profile(name)["lastrun"] = date.strftime(@timeformat)
     end
 
     def profile(name)

data/lib/bup/tar.rb

@@ -1,7 +1,5 @@
 # frozen_string_literal: true
 
-require "English"
-require "English"
 require "tempfile"
 require "open3"
 require "date"
@@ -13,9 +11,9 @@ module Bup
     end
 
     def expand_string(str)
-      while str =~ /\${?([a-zA-Z0-9]+)}?/
-        all = $LAST_MATCH_INFO[0]
-        nm = $LAST_MATCH_INFO[1]
+      while str =~ /\${?([a-zA-Z0-9_]+)}?/
+        all = $~[0]
+        nm = $~[1]
         str = str.sub(all, ENV[nm] || "")
       end
 
@@ -34,100 +32,130 @@ module Bup
     end
 
     # Prepare the destination directory.
-    def prepare_destination(profile_name)
-      destination = expand_string(@config.profile(profile_name)["destination"] || ".")
-      suffix = ".tar.xz"
+    def prepare_destination(profile_name, profile)
+      destination = profile["destination"] || "."
+
       type = @config.runtime["type"] || "full"
 
       FileUtils.mkdir_p(destination) unless File.exist?(destination)
 
       filename_base = "#{profile_name}-#{type}"
       filename = File.join(destination,
-                           "#{filename_base}-#{DateTime.now.new_offset(0).strftime("%Y%m%d%H%M%S")}#{suffix}")
+                           "#{filename_base}-#{DateTime.now.new_offset(0).strftime("%Y%m%d%H%M%S")}")
 
       history = @config.profile(profile_name)["history"] || 0
 
-      if type == "full" && history.positive?
+      if type == "full" 
+        if history.positive?
+          prune_history(profile_name, destination, filename_base, history)
+        elsif history == 0
+          clear_history(profile_name, destination)
+        end
+      end
 
-        oldest_kept_file = nil
+      ENV["BUP_DESTINATION"] = destination
+      ENV["BUP_FILENAME"] = filename
+      filename
+    end
 
-        # Keep some full backups and remove the others.
-        # We capture the oldest full backup and get rid of preceeding incrementals.
-        Dir["#{destination}/#{filename_base}*"].sort.reverse.each_with_index do |fullbackup, idx|
-          if idx < history
-            oldest_kept_file = fullbackup
-          else
-            File.delete(fullbackup)
-          end
-        end
+    def clear_history(profile_name, destination)
+      Dir["#{destination}/#{profile_name}*"].each do |backup|
+        File.delete(backup)
+      end
+    end
 
-        # Remove all incremental files that are older than the oldest kept full backup.
-        if oldest_kept_file
-          remove_before = File.stat(oldest_kept_file).ctime
-          Dir["#{destination}/#{profile_name}*"].each do |backupfile|
-            File.delete(backupfile) if File.stat(backupfile).ctime < remove_before
-          end
+    def prune_history(profile_name, destination, filename_base, history)
+      oldest_kept_file = nil
+
+      # Keep some full backups and remove the others.
+      # We capture the oldest full backup and get rid of preceeding incrementals.
+      Dir["#{destination}/#{filename_base}*"].sort.reverse.each_with_index do |fullbackup, idx|
+        if idx < history
+          oldest_kept_file = fullbackup
+        else
+          File.delete(fullbackup)
         end
       end
 
-      filename
+      # Remove all incremental files that are older than the oldest kept full backup.
+      if oldest_kept_file
+        remove_before = File.stat(oldest_kept_file).ctime
+        Dir["#{destination}/#{profile_name}*"].each do |backupfile|
+          File.delete(backupfile) if File.stat(backupfile).ctime < remove_before
+        end
+      end
     end
 
+    # Run tar.
     def call(profile_name = nil)
       profile_name ||= @config.runtime["profile"]
       profile = @config.config["profiles"][profile_name]
 
       raise "Missing profile #{profile_name}." unless profile
 
-      args = @config.profile(profile_name)["tarcmd"].dup || ["tar"]
+      args = profile["tarcmd"].dup || ["tar", "cJvf", "${BUP_FILENAME}.tar.xz"]
 
-      args << "cJvf"
-
-      filename = prepare_destination(profile_name)
-      args << filename
+      prepare_destination(profile_name, profile)
 
       if @config.runtime["type"] == "incremental"
         lastrun = @config.lastrun(profile_name)
         args += ["--newer", lastrun.strftime("%Y-%m-%d %H:%M:%S %z")] if lastrun
       end
 
+      run_pre_cmds profile
+
       @config.update_lastrun(profile_name)
 
       tf = build_exclude_file(profile["exclude"] || [])
 
-      args += ["--exclude-from", tf.path]
-      args += (@config.profile(profile_name)["include"] || ["."]).map do |str|
-        expand_string(str)
+      # Build and run tar command.
+      begin
+        args += ["--exclude-from", tf.path]
+        args += (profile["include"] || ["."])
+        args = args.map do |str|
+          expand_string(str)
+        end
+
+        # Run tar.
+        run_cmd(*args)
+      ensure
+        tf.unlink
       end
 
-      begin
-        Open3.popen3(*args) do |stdin, stdout, stderr, wait_thr|
-          stdin.close
-          while wait_thr.status
-            r, w, e = IO.select([stdout, stderr])
-            r.each do |stream|
-              begin
-                print stream.read_nonblock(1024)
-              rescue EOFError
-              end
-            end
-          end
+      run_post_cmds profile
+    end
 
-          wait_thr.join
-          begin
-            print stdout.read_nonblock(1024)
-          rescue EOFError
-          end
+    def run_pre_cmds(profile)
+      # Before we update the last_run, execute any pre_cmds.
+      (profile["pre_cmds"] || []).each do |args|
+        run_cmd(*args.map { |c| expand_string(c) })
+      end
+    end
 
-          begin
-            print stderr.read_nonblock(1024)
-          rescue EOFError
-          end
+    def run_post_cmds(profile)
+      (profile["post_cmds"] || []).each do |args|
+        run_cmd(*args.map { |c| expand_string(c) })
+      end
+    end
 
-          STDOUT.flush
+    # Exec and run a command in a standard way.
+    def run_cmd(*args)
+      Open3.popen3(*args) do |stdin, stdout, stderr, wait_thr|
+        stdin.close
+        x = nil
+        while wait_thr.status
+          r, _w, _e = IO.select([stdout, stderr])
+          r.each do |stream|
+            print x if (x = stream.read_nonblock(1024, exception: false))
+          end
         end
-      ensure
-        tf.unlink
+
+        wait_thr.join
+        print x if (x = stdout.read_nonblock(1024, exception: false))
+
+        print x if (x = stderr.read_nonblock(1024, exception: false))
+
+        $stdout.flush
       end
     end
   end

data/lib/bup/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Bup
-  VERSION = "0.2.0"
+  VERSION = "0.6.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: bup
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Sam Baskinger
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-11-14 00:00:00.000000000 Z
+date: 2021-11-29 00:00:00.000000000 Z
 dependencies: []
 description: Backup tool driver that connects tar, gnupg, and other tools to accomplish
   backups.