smooth_s3 0.2.0 → 0.2.1

data/CHANGELOG

@@ -1,3 +1,6 @@
+### 0.2.1 ###
+  - Added :only and :except options to the 2 directory_sync methods. You can pass a Regexp or an array of Regexps.
+
 ### 0.2.0 ###
   - Refactoring run
   - Connection errors and failed uploads are now all properly rescued by appropriate logic.
@@ -9,4 +12,4 @@
   - Removed dependency to jeweler (oops!)
 
 ### 0.1.0 ###
-  - First release
+  - First release

data/README.textile

@@ -12,6 +12,20 @@ Smooth S3 is a user-friendly superset of the S3 gem geared towards file system b
 
 The goal with Smooth S3 is to facilitate and simplify your S3 uploads. It is a library focused on the file system, so no integration with MySQL, third-party services or anything like that. Nothing prevents you from doing a _mysqldump_ and uploading the results in the same script using Smooth S3 though ;)
 
+h2. Why Use Smooth S3?
+
+Aren't the AWS/S3 and S3 gems already good enough? Well yes, they are, but all they do is wrap the AWS S3 API. They do an awesome job at it, mind you, but if you've had to write more than a few apps and/or scripts where you need to use S3, you know that there is quite a bit of boilerplate code required even for the simplest upload operations.
+
+There was an opportunity to make all of this way simpler (or, _puts on sunglasses_, smoother), and out of that opportunity Smooth S3 was born. For the people who like bullet points, here are a few reasons why you should probably use it:
+
+* **Smart defaults** - No more boilerplate code. Once the service is initialized (a one-liner, of course), you only need to specify bucket and file(s) and they will be uploaded with a single method call on the service. The bucket doesn't exist? No problem. As long as the name you provided is unique in the S3 global namespace, it will be created. Want to overwrite existing files in the bucket? Add a bang(!) to your method. You get the gist of it.
+
+* **Few methods, lots of power** - On top of service initialization, the library currently only offers 4 methods (that could technically, and may eventually be reduced to 2) so it is pretty darn simple to use. Options are used to arm that simplicity with some power. There is a good chance that by combining options, you will get your uploads to behave the way you want with SmoothS3. If not, fork away or suggest one in the issues! :)
+
+* **No raising, unless absolutely necessary** - The established S3 libraries are pretty trigger-happy when it comes to raising exceptions. We all love Fail Fast™ but there are times when it's simply not the right approach, such as with backups. Sure, you will eventually have rescued everything but by then, your codebase will have tripled. Smooth S3 only raises when it absolutely has no choice to do so (ex: Wrong AWS credentials). Failed uploads are retried a few times, then skipped with a log message if they are still a no-go (permissions etc.). Bucket and object operations are also retried in case of a random HTTP failures and won't throw exceptions if you look for a non-existant one.
+
+* **Dogfood is being eaten** - Smooth S3 is being used in production to back up 100+ GBs of unique data daily over here at "Needium":http://needium.com. Most options that are added are derived from our direct needs, so you can be certain that the library will remain supported and new features will keep being added.
+
 h2. Installation
 
 This library has been developed against and designed for MRI 1.9.2 in a UNIX environment. It should be compatible with 1.8.6+, as well as with other Ruby implementations, but no guarantees. Probably not compatible with Windows environments.
@@ -29,7 +43,7 @@ Once that is done you can use the following methods:
 
 * upload() - Regular file upload. Can specify multiple files at once. Directory structure is not preserved, only the file name.
 
-* sync_directory() - Uploads the entire content of a directory and its subdirectories. Preserves folder directory structure inside S3.
+* directory_sync() - Uploads the entire content of a directory and its subdirectories. Preserves folder directory structure inside S3.
  
 * timestamped_upload() - Like a regular upload. Files uploaded this way have a timestamp added in front of their names. Default timestamp format: YYYYmmddHHMMSS  
 
@@ -60,11 +74,11 @@ require 'smooth_s3'
 @service.upload!("my_test_bucket", ["here.rb", "../parent.rb"], :prefix => "prefix/to/files")
 
 
-# sync_directory()
+# directory_sync()
 # Params: bucket_name, directory(, :prefix)
 
-@service.sync_directory("my_test_bucket", "../reports")
-@service.sync_directory!("my_test_bucket", "data", :prefix => "prefix/to/dir")
+@service.directory_sync("my_test_bucket", "../reports")
+@service.directory_sync!("my_test_bucket", "data", :prefix => "prefix/to/dir")
 
 
 # timestamped_upload()
@@ -85,7 +99,7 @@ require 'smooth_s3'
 
 h2. Credits
 
-* Jakub Kuźma for his great S3 library, often overshadowed by the popular but old AWS-S3.
+* Jakub Kuźma for his great S3 library, often overshadowed by AWS-S3.
 
 h2. LICENSE
 

data/VERSION

@@ -1 +1 @@
-0.2.0
+0.2.1

data/lib/smooth_s3.rb

@@ -11,5 +11,5 @@ require 'smooth_s3/uploader.rb'
 require 'smooth_s3/error.rb'
 
 module SmoothS3
-  VERSION = "0.2.0"
-end
+  VERSION = "0.2.1"
+end

data/lib/smooth_s3/bucket.rb

@@ -31,14 +31,14 @@ module SmoothS3
       Service.new_buckets[service.aws_key] << new_bucket
     end
 
-    def self.store_file(file, remote_file, bucket, service, prefix, overwrite)
+    def self.store_file(file, remote_file, bucket, service, options)
       b = service.refresh.buckets[bucket]
 
-      if prefix
-        remote_file = prefix =~ /\/$/ ? (prefix + remote_file) : prefix + "/" + remote_file
+      if options[:prefix]
+        remote_file = options[:prefix] =~ /\/$/ ? (options[:prefix] + remote_file) : options[:prefix] + "/" + remote_file
       end
 
-      unless overwrite == true
+      unless options[:overwrite] == true
         if Bucket.file_exists?(remote_file, bucket, service)
           puts "'#{remote_file}' already exists on S3 bucket named '#{bucket}'. Use the bang(!) version of the method to overwrite."
           return

data/lib/smooth_s3/uploader.rb

@@ -8,7 +8,7 @@ module SmoothS3
       valid_files = Uploader.validate_files(files)
       valid_files.each do |vf|
         remote_file_name = options[:timestamped] ? (options[:timestamp] + "_" + vf.split("/")[-1]) : vf.split("/")[-1]
-        Bucket.store_file(vf, remote_file_name, bucket, service, options[:prefix], options[:overwrite])
+        Bucket.store_file(vf, remote_file_name, bucket, service, options)
       end
     end
 
@@ -20,10 +20,10 @@ module SmoothS3
       [:overwrite, :timestamped].each {|s| options[s] = false unless options[s]}
       Bucket.select(bucket, service)
 
-      valid_files = Uploader.validate_files_in_directory(directory)
+      valid_files = Uploader.validate_files_in_directory(directory, options)
       valid_files.each do |vf|
         remote_file_name = options[:timestamped] ? (options[:timestamp] + "_" + vf[1]) : vf[1]
-        Bucket.store_file(vf[0], remote_file_name, bucket, service, options[:prefix], options[:overwrite])
+        Bucket.store_file(vf[0], remote_file_name, bucket, service, options)
       end
     end
 
@@ -78,7 +78,7 @@ module SmoothS3
         valid_files
       end
 
-      def self.validate_files_in_directory(directory)
+      def self.validate_files_in_directory(directory, options)
         valid_files = []
 
         begin
@@ -92,7 +92,27 @@ module SmoothS3
           raise SmoothS3::Error, "'#{directory}' is not a valid directory."
         end
 
-        valid_files
+        Uploader.filter_files(valid_files, options)
+      end
+
+      def self.filter_files(files, options)
+        [:except, :only].each { |o| options[o] = options[o].class == Regexp ? [options[o]] : options[o] }
+        except, only = [], []
+
+        files.each do |f|
+          options[:except].each {|r| except << f if f[0] =~ r} if options[:except]
+          options[:only].each {|r| only << f if f[0] =~ r} if options[:only]
+        end
+
+        if options[:except] && options[:only]
+          filtered_files = only - except
+        elsif options[:except]
+          filtered_files = files - except
+        elsif options[:only]
+          filtered_files = only
+        else
+          filtered_files = files
+        end
       end
 
       def self.calculate_timestamp(options)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: smooth_s3
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2011-12-08 00:00:00.000000000Z
+date: 2011-12-16 00:00:00.000000000Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: s3
-  requirement: &2153519260 !ruby/object:Gem::Requirement
+  requirement: &2153165680 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -21,7 +21,7 @@ dependencies:
         version: 0.3.9
   type: :runtime
   prerelease: false
-  version_requirements: *2153519260
+  version_requirements: *2153165680
 description: A user-friendly superset of the S3 gem geared towards file system backup
   operations. Simplifies standard actions such as basic uploads, for example allowing
   multiple files to be uploaded in one operation and adds new functionality such as