thin_out_backups 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 71d687dd4b0eb09b150c37d7d4567d134eb21cc6
+  data.tar.gz: 10a0c8c8eb6fa2a090fff6569720ced62fafa997
+SHA512:
+  metadata.gz: ed27167bae6ab29f0466127064500f964288dce9e71fc7d6836623b871b437368014d0800084b5106131f567d7be4ffdcab994f02ed91d382b2e091ff214da92
+  data.tar.gz: e89661fc8e6f5e2e41226f8f746c573775cb7dae69434a9ad8ba75465a3a3c97b7b269062806a08c828b88b45cb65983a1883422f10bb730f4a46b08d187f1d9

data/.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/.rspec

@@ -0,0 +1 @@
+--color

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in thin_out_backups.gemspec
+gemspec

data/License

@@ -0,0 +1,22 @@
+Copyright (c) 2012 Tyler Rick
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,2 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"

data/Readme.md

@@ -0,0 +1,43 @@
+# thin_out_backups
+
+Quickly and safely thin out a backups directory that's taking up too much hard disk space!
+
+`thin_out_backups` will keep the specified number of backups in each frequency category (weekly,
+daily, etc.) and delete the rest, keep the space requirements of your backups directory fairly
+constant over time. 
+
+The files that you are thinning out don't have to be backups, but that is probably the most common
+use case.
+
+## Installation
+
+    $ gem install thin_out_backups
+
+## Usage
+
+    $ thin_out_backups
+
+
+
+## License
+
+Copyright 2008, 2012 Tyler Rick
+
+Released under the MIT license. See License file.
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+
+## Other names considered
+
+* thin_out_backup_dir
+* sparsen_dir
+* sparsify_dir
+* [rm_extra_copies](https://github.com/TylerRick/command-line/blob/master/bin/rm_extra_copies) (previously published under this name)
+* prune_backups
+* trim_dir

data/bin/thin_out_backups

@@ -0,0 +1,180 @@
+#!/usr/bin/env ruby
+# vim: textwidth=150
+
+=begin To Do
+Make it possible to run on a capistrano releases directory to only keep the n latest releases and delete the rest (just like cap deploy:cleanup). So you don't end up with a pile of 200 releases in that directory...
+
+Add --duplicates/--dups option, that when present deletes all duplicate files in a directory
+
+Add --max-size option: keeps deleting until the du -s #{dir} reports that it is beneath that threshold.
+What order should it delete? Oldest first?
+After each deletion, checks if it's reached (below) max size yet (just check filesize before deleting and then subtract that from a running total -- faster than doing a du over and over).
+Before any other deletes, it should search for/delete duplicates. If --duplicates option present, will do the search even if already below max size.
+=end
+
+require 'getoptlong'
+#require 'rubygems'
+#require 'facets'
+require 'thin_out_backups'
+
+def help
+  puts <<End
+Synopsis:
+---------
+
+thin_out_backups [options] dirs...
+
+Example:
+thin_out_backups --daily 5 --monthly 12 /path/to/backups
+
+
+thin_out_backups will keep the specified number of time-stamped copies from each "time bucket" and remove all older copies.
+
+So even if you don't actually make any weekly backups, as long as you make daily or hourly backups, you will still *have* weekly backups which
+you can keep. This will let you keep, for example, *all* the weekly backups but only 6 days worth of daily backups.
+
+When you specify --weekly=3, it means go back for 3 weeks and keep one backup from each week.
+So it will keep the most recent file from the current week, the most recent file from the previous week, etc.
+
+Use a * instead of a number to go back all the way to the oldest [week] represented in that directory and keep 1 copy from each [week] visited
+along the way. Note that it usually only makes sense to do this for the largest time interval being used. (It wouldn't make any sense to say
+--hourly=* --daily=3 because the --daily=3 will cause no files to be kept that wouldn't already be kept due to the --hourly.)
+
+The same file may satisfy more than 1 quota. So if you set them all to 1, they will all be satisfied by the same file: the most recent backup in the
+directory. For that reason, 2 is probably the minimum you'd want to set any of these to.
+
+
+Options:
+--------
+  --help                   Print this message
+  --minutely/--hourly/--daily/--weekly/--monthly/--yearly quota/frequency  
+                           Keep n number of [daily] backups or * to keep all
+  --<custom-interval>=<n>
+  --force                  Don't ask for confirmation before deleting. Use this if you put this command in a crontab and you are very sure you know 
+                           what it will do.
+  --now                    Use a different date/time as now.
+  --get-time-from filename Takes the time from the filename. It must be present in the filename and formatted in %Y%m%dT%H%M format.
+  --get-time-from file_system
+                           Uses the mtime of the file.
+  --no-color               Use a different date/time as now.
+  --last/--latest          Take the last (latest) candidate in each time interval (default). 
+  --first/--earliest       Take the first (earliest) candidate in each time interval (not default).
+
+By default, days start at the beginning of day, weeks to beginning of week, etc.
+
+To do: 
+  --identical              Compares the md5 digest of each pair of files and removes any files that have the same (and therefore probably are identical) -- except for the oldest/newest one.
+  --consec-identical       Same as --identical except only checks and removes consecutive/neighboring files to see if they are identical.
+  --align=<?>
+      add option to align the time ranges (days, etc.) from "now" rather than at the beginning of the day, etc. So "day" would actually be "the most
+recent 24-hour period, beginning exactly 24 hours ago and ending right now".
+
+
+Format of quota/frequency
+-------------------------
+
+TODO: allow crontab-style specifiers like --hourly=2/4 to keep 2 4-hourly backups, or
+--hourly=*/12 to keep *all* 12-hourly backups (= keep 2 backups per day).
+
+
+How your files/directories must be organized
+--------------------------------------------
+
+Each directory is expected to contain only one type of backup -- with many "copies" of that backup accumulated over time -- not a mix of backups.
+
+These backups may be either files or directories. (rm -rf will be used to remove them)
+
+Within each directory, this program will take the n most recently created files in each "time bucket" and retain them while pruning/deleting *all* 
+other files in the directory.
+
+WARNING: If you have other stuff in that directory that you want to keep, be warned: it WILL be deleted by this script!
+
+TODO: let you specify a glob pattern for files that WILL be pruned; and/or add an --ignore option to specify file patterns to ignore = NOT subject to pruning = not delete
+
+For the moment, I require that the timestamp be in the filename itself, in "%Y%m%dT%H%M%S" format.
+TODO: add option to use the timestamp recorded in the file system rather than the timestamp from within the filename.
+
+Background
+----------
+
+This is useful when you have a cron job that continuously creates backups and dumps them in a certain directory and you want to keep that directory
+from becoming inordinately large. (Or quickly get it down to a smaller size now that it's grown extremely large and you've run out of room on the disk.)
+
+The idea is that you are more likely to want/need a recent backup than an older backup. Probably because you hope to become aware of whatever problem
+necessitates looking at/using/restoring from the backup very soon after the problem arises.
+
+So you want to keep a higher density of backups from recent times than you do from older times. The high density of recent backups ensures that you 
+have a higher likelihood of having a backup from very soon before the data became corrupt/whatever and needed to be restored.
+
+For example, if you have backups for every hour in the last 24 hours, and you discover that something got deleted/corrupted/etc. at 23:15 last night,
+then if you pull the most recent backup, say the 23:00 backup, it will be at most an hour before the time of the problem. So the amount of data that
+is lost is at most 1 hour's worth.
+
+As you go further back in time, however, you are less likely to need any backups from that time. But you may want to keep them around for historical
+or statistical or "just in case there is a subtle problem that we don't find out about until 6 months later and we need to be able to go back and
+determine how it happened and restore a certain chunk of data from that old pre-problem backup."
+
+Well, this tool will help you to keep around SOME old copies, without keep around as many as you keep of the more recent ones.
+
+End
+  exit 0
+end
+
+
+
+quotas = {}
+
+help if ARGV.empty?
+
+opts = GetoptLong.new(
+    [ '--minutely',                     GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--hourly',                       GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--daily',                        GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--weekly',                       GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--monthly',                      GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--yearly',                       GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--get-time-from',                GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--last',  '--latest',            GetoptLong::NO_ARGUMENT ],
+    [ '--first', '--earliest',          GetoptLong::NO_ARGUMENT ],
+    [ '--force', '-f',                  GetoptLong::NO_ARGUMENT ],
+    [ '--no-color',                     GetoptLong::NO_ARGUMENT ],
+    [ '--time-format',                  GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--pattern',                      GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--exclude',                      GetoptLong::REQUIRED_ARGUMENT ],
+    [ '--now',                          GetoptLong::REQUIRED_ARGUMENT ]
+)
+
+opts.each do | opt, arg |
+  case opt
+
+  when '--help'
+    help
+
+  when '--force', '-f'
+    ThinOutBackups::Command.force = true
+
+  when '--no-color'
+    ThinOutBackups::Command.color = false
+
+  when *ThinOutBackups::Command.options.map {|o| "--#{o.to_s.gsub(/_/, '-')}"}
+    name = opt.gsub(/^--/, '').to_sym
+    ThinOutBackups::Command.send("#{name}=", arg)
+
+  when *ThinOutBackups::Command.allowed_bucket_names.map {|o| "--#{o}"}
+    name = opt.gsub(/^--/, '').to_sym
+    if arg == '*'
+      #quotas[name] = :all
+      quotas[name] = '*'
+    else
+      quotas[name] = arg.to_i
+    end
+  end
+end
+
+dirs = ARGV
+raise "Must specify at least one directory" if dirs.empty?
+dirs.each do |dir|
+  command = ThinOutBackups::Command.new(dir, quotas)
+  command.run
+end
+

data/lib/thin_out_backups/time_fixes.rb

@@ -0,0 +1,131 @@
+# Until facets gets patched proper, we'll monkey patch it here:
+# TODO: Is this still needed?
+#require '/home/tyler/dev/ruby/facets/lib/core/facets/time/hence'
+class Time
+
+  if defined?(::ActiveSupport)
+
+    alias_method :in, :since
+    alias_method :hence, :since
+
+  else
+
+    # Returns a new Time representing the time
+    # a number of time-units ago.
+    #
+    def ago(number, units=:seconds)
+      time =(
+        case units.to_s.downcase.to_sym
+        when :years
+          set(:year => (year - number))
+        when :months
+          y = ((month - number) / 12).to_i
+          #puts "(#{month} - #{number}) / 12 == #{y}"
+
+          new_month = ((month - number - 1) % 12) + 1
+          y += 1 if new_month > month
+          #puts y
+
+          set(:year => (year - y), :month => new_month)
+        when :weeks
+          self - (number * 604800)
+        when :days
+          self - (number * 86400)
+        when :hours
+          self - (number * 3600)
+        when :minutes
+          self - (number * 60)
+        when :seconds, nil
+          self - number
+        else
+          raise ArgumentError, "unrecognized time units -- #{units}"
+        end
+      )
+      dst_adjustment(time)
+    end
+    #
+    # Returns a new Time representing the time
+    # a number of time-units hence.
+
+    def hence(number, units=:seconds)
+      time =(
+        case units.to_s.downcase.to_sym
+        when :years
+          set( :year=>(year + number) )
+        when :months
+          y = ((month + number - 1) / 12).to_i
+          m = ((month + number - 1) % 12) + 1
+          set(:year => (year + y), :month => m)
+        when :weeks
+          self + (number * 604800)
+        when :days
+          self + (number * 86400)
+        when :hours
+          self + (number * 3600)
+        when :minutes
+          self + (number * 60)
+        when :seconds
+          self + number
+        else
+          raise ArgumentError, "unrecognized time units -- #{units}"
+        end
+      )
+      dst_adjustment(time)
+    end
+
+    alias_method :in, :hence
+    alias_method :since, :hence
+
+    # Adjust DST
+    #
+    # TODO: Can't seem to get this to pass ActiveSupport tests.
+    # Even though it is essentially identical to the
+    # ActiveSupport code (see Time#since in time/calculations.rb).
+    # It handles all but 4 tests.
+    def dst_adjustment(time)
+      self_dst = self.dst? ? 1 : 0
+      time_dst = time.dst? ? 1 : 0
+      seconds  = (self - time).abs
+      if (seconds >= 86400 && self_dst != time_dst)
+        time + ((self_dst - time_dst) * 60 * 60)
+      else
+        time
+      end
+    end
+
+  end
+
+end
+
+
+class DateTime
+  def to_time
+    Time.mktime(year, month, day, hour, min, sec)
+  end
+end
+
+class Time
+  def to_s
+    #strftime("%Y%m%dT%H%M%S")
+    strftime("%Y-%m-%d %H:%M")
+  end
+  def to_s_full
+    strftime("%Y-%m-%d %H:%M:%S")
+  end
+
+  # /usr/lib/ruby/gems/1.8/gems/activesupport-2.1.2/lib/active_support/core_ext/time/calculations.rb
+
+    def beginning_of_day
+      change(:hour => 0)
+    end
+    alias :midnight :beginning_of_day
+
+    # Returns a new Time representing the "start" of this week (Sunday, 0:00)
+    def beginning_of_week
+      days_to_sunday = self.wday
+      self.ago(days_to_sunday, :days).midnight
+    end
+    alias :sunday :beginning_of_week
+
+end
+

data/lib/thin_out_backups/version.rb

@@ -0,0 +1,3 @@
+module ThinOutBackups
+  Version = "0.0.1"
+end

data/lib/thin_out_backups.rb

@@ -0,0 +1,285 @@
+require "thin_out_backups/version"
+
+# Tested by: spec/thin_out_backups_spec.rb
+
+require 'fileutils'
+require 'pathname'
+require 'delegate'
+
+#require 'rubygems'
+require 'facets/time'
+require 'colored'
+require 'quality_extensions/module/attribute_accessors'
+require 'thin_out_backups/time_fixes'
+
+class ThinOutBackups::Command
+  #---------------------------------------------------------------------------------------------------------------------------------------------------
+
+  @@allowed_bucket_names = [:minutely, :hourly, :daily, :weekly, :monthly, :yearly]
+  mattr_reader :allowed_bucket_names
+
+  #---------------------------------------------------------------------------------------------------------------------------------------------------
+  # Options
+  @@options = [:get_time_from, :ignore_files ,:verbosity, :time_format, :now, :force, :no_color]
+  mattr_reader :options
+  mattr_accessor *@@options
+
+  @@get_time_from = :filename
+  def self.get_time_from=(new)
+    @@get_time_from = new.to_sym
+    raise "Unknown value for #{ThinOutBackups::Command.get_time_from}" unless @@get_time_from.in?([:filename, :file_system])
+  end
+
+  @@ignore_files = nil
+  @@verbosity = 1
+  @@force = false
+  @@color = true
+
+  @@now = Time.now
+  def self.now=(new)
+    time = DateTime.strptime('2008-11-12 07:45:00', '%Y-%m-%d %H:%M:%S').to_time
+    @@now = new
+    puts "Using alternate now: #{@@now}"
+  end
+
+  @@time_format = /(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})?/
+  @@time_format_parts = [:Y,:m,:d, :H,:M,:S]
+  # TODO: Maybe use something like this for interpreting time, rather than a regexp? DateTime.strptime("27/Nov/2007:15:01:43 -0800", "%d/%b/%Y:%H:%M:%S %z")
+  def self.time_format=(new)
+    # TODO: accept format strings such as 'H:M:S d.m.Y.' and 'Y-m-d H:M:S'
+    # TODO: do error checking
+  end
+
+
+  #---------------------------------------------------------------------------------------------------------------------------------------------------
+
+  class Bucket
+    attr_reader :parent, :name, :quota, :keepers
+    @@quota_format = %r[(\d+|\*)(/\d+)?]
+
+    def initialize(parent, name, quota)
+      @parent = parent
+      @name = name
+      (
+      raise "Invalid quota '#{quota}'" unless quota.is_a?(Fixnum) || quota =~ @@quota_format
+      @quota = quota
+      )
+      @keepers = []
+    end
+
+    def unit
+      {
+        :minutely => :minutes,
+        :hourly   => :hours,
+        :daily    => :days,
+        :weekly   => :weeks,
+        :monthly  => :months,
+        :yearly   => :years,
+      }[@name.to_sym]
+    end
+
+    def start_time
+      start_time = parent.now.dup
+      if parent.align_at_beginning_of_time_interval
+        beginning_of_interval = 
+          case unit
+          when :minutes
+            start_time.change(:sec => 0)
+          when :hours
+            start_time.change(:min => 0)
+          when :days
+            start_time.change(:hour => 0)
+          when :weeks
+            start_time.beginning_of_week
+          when :months
+            start_time.change(             :day => 1, :hour => 0)
+          when :years
+            start_time.change(:month => 1, :day => 1, :hour => 0)
+          else
+            raise "unexpected unit #{unit}"
+          end
+        # We actually want to use the *next* interval (in the future) as our start_time because we will be using this as our max and going backwards in time...
+        beginning_of_interval.hence(1, unit)
+      else
+        start_time
+      end
+    end
+
+    def keep(keeper)
+      @keepers << keeper
+    end
+
+    def still_need
+      if keep_all?
+        1 # it has insatiable hunger for keepers that can never be satisfied ... always just 1 more ...
+      else
+        @quota - @keepers.size
+      end
+    end
+
+    def keep_all?; quota =~ /\*/ end
+    
+    def satisfied?
+      if keep_all?
+        false # it has insatiable hunger for keepers that can never be satisfied
+      else
+        still_need == 0
+      end
+    end
+  end
+
+  class File < DelegateClass(::Pathname)
+    attr_reader :filename, :file
+
+    def initialize(filename)
+      super(Pathname.new(filename))
+    end
+
+    def full_path
+      dirname.to_s + '/' + filename
+    end
+
+    def filename
+      basename.to_s
+    end
+    def to_s
+      filename
+    end
+
+    def time
+      if ThinOutBackups::Command.get_time_from == :filename
+        if filename =~ ThinOutBackups::Command.time_format
+          y,m,d, h,i,s = $1,$2,$3, $4,$5,$6
+          Time.mktime(y,m,d, h,i,s)
+        else
+          nil
+        end
+      elsif ThinOutBackups::Command.get_time_from == :file_system
+        file.mtime
+      else
+        raise "Unknown value for #{ThinOutBackups::Command.get_time_from}"
+      end
+    end
+
+    def has_time?; !!time end
+    def ignored?
+      !has_time? or
+      ThinOutBackups::Command.ignore_files && filename =~ ThinOutBackups::Command.ignore_files
+    end
+  end
+
+  attr_reader :align_at_beginning_of_time_interval, :files_with_times
+  attr_accessor :dir
+
+  def initialize(dir, quotas)
+    @align_at_beginning_of_time_interval = true
+    @dir = dir
+    @buckets = {}
+    @@allowed_bucket_names.each do |name|
+      quota = quotas[name]
+      @buckets[name] = Bucket.new(self, name, quota) unless quota.nil?
+    end
+
+    puts "Processing #{@dir}/*".magenta
+    files = Dir["#{@dir}/*"].map { |filename|
+      file = File.new(filename)
+    }
+    @files_with_times = files.
+      reject {|file| !file.has_time?}.
+      sort { |a, b|
+             a.time <=> b.time
+           }.
+      reverse
+
+  end
+
+  def bucket_remaining(bucket_name, decr = nil)
+    send("#{bucket_name}=", send("#{bucket_name}") - decr) if decr
+    send "#{bucket_name}"
+  end
+
+  def buckets
+    @buckets.values
+  end
+  def bucket(name)
+    @buckets[name] or raise "unknown bucket '#{name}'"
+  end
+
+  def now
+    Time.now
+  end
+
+  def delete_non_keepers
+    #raise "Didn't find any files to keep?!" unless keepers.any?
+    files_with_times.each do |file|
+      if (buckets = buckets_with_file(file)).any?
+        puts "#{file.full_path}: in buckets: #{buckets.map(&:name).join(', ')}".green
+      else
+        puts "#{file.full_path}: delete".red
+      end
+    end
+
+    if @@force == false
+      print "Continue with deletions? (yes or no) >".magenta
+      response = STDIN.gets
+      (puts "Aborting"; return) unless response.chomp.downcase == 'yes'
+    end
+
+    files_with_times.each do |file|
+      if (buckets = buckets_with_file(file)).any?
+        #
+      else
+        file.unlink
+      end
+    end
+  end
+
+  def buckets_with_file(file)
+    buckets.find_all {|bucket| bucket.keepers.include?(file)}
+  end
+
+  def delete(files)
+    puts "Deleting files: #{files.join(', ')}..."
+  end
+
+  def earliest_file_time
+    files_with_times.last.time
+  end
+
+  def run
+    raise "Must keep at least 1 file from at least one time bucket" if buckets.empty?
+    (puts "Found no files with times! Aborting."; return) if files_with_times.empty?
+
+    # Fill each bucket until its quota is met
+    buckets.each do |bucket|
+      puts "Trying to fill bucket '#{bucket.name}' (quota: #{bucket.quota})...".magenta
+
+      time_max = bucket.start_time
+      time_min = time_max.ago(1, bucket.unit)
+
+      #puts "Earliest_file_time: #{earliest_file_time}"
+      while time_max > earliest_file_time
+        print "Checking range (#{time_min} .. #{time_max})... ".yellow if verbosity >= 1
+        new_keeper = files_with_times.detect {|file|
+          #print "#{file}? "
+          time_min <= file.time &&
+                      file.time < time_max
+        }
+        if new_keeper
+          puts "found keeper #{new_keeper}".green if verbosity >= 1
+          bucket.keep new_keeper
+        else
+          #puts "found no keepers".red if verbosity >= 1
+          puts "" if verbosity >= 1
+        end
+
+        time_max = time_min
+        #puts "Stepping back from #{time_min} by 1 #{bucket.unit} => #{time_min.ago(1, bucket.unit)}"
+        time_min = time_min.ago(1, bucket.unit)
+        (puts 'Filled quota!'.green; break) if bucket.satisfied?
+      end
+    end
+
+    delete_non_keepers
+  end
+end

data/spec/thin_out_backups_spec.rb

@@ -0,0 +1,137 @@
+require 'tmpdir'
+#require 'rubygems'
+require 'rspec'
+require 'facets'
+require_relative '../lib/thin_out_backups'
+
+$now = Time.utc(2008,11,12, 7,45,19)
+
+describe Time, "#beginning_of_week" do
+  it "should return a Sunday" do
+    Time.utc(2008,11,12).beginning_of_week.should == Time.utc(2008,11,9)
+  end
+end
+
+describe ThinOutBackups::Command::Bucket, "time interval alignment" do
+
+  def sample_quotas
+    {
+      :minutely => 1,
+      :hourly => 3,
+      :daily => 1,
+      :weekly => '*',
+      :monthly => 1,
+      :yearly => '*'
+    }
+  end
+
+  before do
+    @command = ThinOutBackups::Command.new('bogus_dir', sample_quotas)
+    @now = $now
+    @command.stub!(:now).and_return(@now)
+  end
+
+  it "should use the time specified by our test" do
+    @command.now.should == @now
+  end
+
+  it "hour interval should start on the hour, etc." do
+    @command.bucket(:minutely).start_time.should == Time.utc(2008,11,12, 7,46,0)
+    @command.bucket(:hourly).  start_time.should == Time.utc(2008,11,12, 8,0,0)
+    @command.bucket(:daily).   start_time.should == Time.utc(2008,11,13, 0,0,0)
+    @command.bucket(:weekly).  start_time.should == Time.utc(2008,11,16, 0,0,0)
+    @command.bucket(:monthly). start_time.should == Time.utc(2008,12, 1, 0,0,0)
+    @command.bucket(:yearly).  start_time.should == Time.utc(2009, 1, 1, 0,0,0)
+  end
+
+end
+
+
+$command = <<End
+thin_out_backups --force --daily=3 --weekly=3 --monthly=* \
+                --now='#{$now.to_s_full}'\
+                spec/test_dir/db_dumps \
+                spec/test_dir/maildir
+End
+describe ThinOutBackups::Command, "when calling `#{$command}`" do
+  before do
+    Pathname.new("spec/test_dir/").rmtree rescue nil
+
+    dir='spec/test_dir/db_dumps/'
+    system "mkdir -p #{dir}"
+    files = %w[
+    db_dump_2008-08-08T0303.sql
+    db_dump_2008-09-01T0303.sql
+    db_dump_2008-09-10T0303.sql
+    db_dump_2008-10-15T0303.sql
+    db_dump_2008-10-16T0303.sql
+    db_dump_2008-10-17T0303.sql
+    db_dump_2008-10-18T0303.sql
+    db_dump_2008-10-19T0303.sql
+    db_dump_2008-10-20T0303.sql
+    db_dump_2008-10-21T0303.sql
+    db_dump_2008-10-22T0303.sql
+    db_dump_2008-10-23T0303.sql
+    db_dump_2008-10-24T0303.sql
+    db_dump_2008-10-25T0303.sql
+    db_dump_2008-10-26T0303.sql
+    db_dump_2008-10-27T0303.sql
+    db_dump_2008-10-28T0303.sql
+    db_dump_2008-10-29T0303.sql
+    db_dump_2008-10-30T0303.sql
+    db_dump_2008-10-31T0303.sql
+    db_dump_2008-11-01T0303.sql
+    db_dump_2008-11-02T0303.sql
+    db_dump_2008-11-03T0303.sql
+    db_dump_2008-11-04T0303.sql
+    db_dump_2008-11-05T0303.sql
+    db_dump_2008-11-06T0303.sql
+    db_dump_2008-11-07T0303.sql
+    db_dump_2008-11-08T0303.sql
+    db_dump_2008-11-09T0303.sql
+    db_dump_2008-11-10T0303.sql
+    db_dump_2008-11-11T0303.sql
+    db_dump_2008-11-12T0303.sql
+    ]
+    files.each do |file|
+      Dir.getwd
+      #puts %(Dir.getwd=#{(Dir.getwd).inspect})
+      #puts %("touch #{dir}/#{file}"=#{("touch #{dir}/#{file}").inspect})
+      system "touch #{dir}/#{file}"
+    end
+
+    dir='spec/test_dir/maildir/'
+    system "mkdir -p #{dir}"
+    subdirs = %w[
+    2008-11-09T0303
+    2008-11-10T0303
+    2008-11-11T0303
+    ]
+    subdirs.each do |subdir|
+      system "mkdir -p #{dir}/#{subdir}"
+      system "touch    #{dir}/#{subdir}/inbox"
+      system "touch    #{dir}/#{subdir}/some_other_folder"
+    end
+
+    puts %($command=#{($command).inspect})
+    #system $command
+    # TODO: also capture output of command and check it against expected
+  end
+
+  it "keeps/removes the correct files" do
+    Dir['spec/test_dir/db_dumps/*'].should =~
+     ["spec/test_dir/db_dumps/db_dump_2008-11-12T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-11-08T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-10-31T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-11-10T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-08-08T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-11-01T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-09-10T0303.sql",
+      "spec/test_dir/db_dumps/db_dump_2008-11-11T0303.sql"]
+  end
+
+  after do
+    #Pathname.new("spec/test_dir/").rmtree rescue nil
+  end
+end
+

data/thin_out_backups.gemspec

@@ -0,0 +1,23 @@
+# -*- encoding: utf-8 -*-
+require File.expand_path('../lib/thin_out_backups/version', __FILE__)
+
+Gem::Specification.new do |gem|
+  gem.authors       = ["Tyler Rick"]
+  gem.email         = ["github.com@tylerrick.com"]
+  gem.summary       = %q{Thin out a directory full of backups, only keeping a specified number from each category (weekly, daily, etc.), and deleting the rest.}
+  gem.description   = gem.summary
+  gem.homepage      = ""
+
+  gem.files         = `git ls-files`.split($\)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.name          = "thin_out_backups"
+  gem.require_paths = ["lib"]
+  gem.version       = ThinOutBackups::Version
+
+  gem.add_dependency 'facets'
+  gem.add_dependency 'colored'
+  gem.add_dependency 'quality_extensions'
+
+  gem.add_development_dependency 'rspec'
+end

metadata

@@ -0,0 +1,116 @@
+--- !ruby/object:Gem::Specification
+name: thin_out_backups
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Tyler Rick
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-03-27 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: facets
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: colored
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: quality_extensions
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Thin out a directory full of backups, only keeping a specified number
+  from each category (weekly, daily, etc.), and deleting the rest.
+email:
+- github.com@tylerrick.com
+executables:
+- thin_out_backups
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- .rspec
+- Gemfile
+- License
+- Rakefile
+- Readme.md
+- bin/thin_out_backups
+- lib/thin_out_backups.rb
+- lib/thin_out_backups/time_fixes.rb
+- lib/thin_out_backups/version.rb
+- spec/thin_out_backups_spec.rb
+- thin_out_backups.gemspec
+homepage: ''
+licenses: []
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.0.0
+signing_key: 
+specification_version: 4
+summary: Thin out a directory full of backups, only keeping a specified number from
+  each category (weekly, daily, etc.), and deleting the rest.
+test_files:
+- spec/thin_out_backups_spec.rb
+has_rdoc: