s3_private_direct_upload 0.1.6

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2012 Wayne
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,274 @@
+# S3DirectUpload
+
+Easily generate a form that allows you to upload directly to Amazon S3.
+Multi file uploading supported by jquery-fileupload.
+
+Code extracted from Ryan Bates' [gallery-jquery-fileupload](https://github.com/railscasts/383-uploading-to-amazon-s3/tree/master/gallery-jquery-fileupload).
+
+## Installation
+Add this line to your application's Gemfile:
+
+    gem 's3_direct_upload'
+
+Then add a new initalizer with your AWS credentials:
+
+**config/initializers/s3_direct_upload.rb**
+```ruby
+S3DirectUpload.config do |c|
+  c.access_key_id = ""       # your access key id
+  c.secret_access_key = ""   # your secret access key
+  c.bucket = ""              # your bucket name
+  c.region = nil             # region prefix of your bucket url (optional), eg. "s3-eu-west-1"
+  c.url = nil                # S3 API endpoint (optional), eg. "https://#{c.bucket}.s3.amazonaws.com/"
+end
+```
+
+Make sure your AWS S3 CORS settings for your bucket look something like this:
+```xml
+<CORSConfiguration>
+    <CORSRule>
+        <AllowedOrigin>http://0.0.0.0:3000</AllowedOrigin>
+        <AllowedMethod>GET</AllowedMethod>
+        <AllowedMethod>POST</AllowedMethod>
+        <AllowedMethod>PUT</AllowedMethod>
+        <MaxAgeSeconds>3000</MaxAgeSeconds>
+        <AllowedHeader>*</AllowedHeader>
+    </CORSRule>
+</CORSConfiguration>
+```
+In production the AllowedOrigin key should be your domain.
+
+Add the following js and css to your asset pipeline:
+
+**application.js.coffee**
+```coffeescript
+#= require s3_direct_upload
+```
+
+**application.css**
+```css
+//= require s3_direct_upload_progress_bars
+```
+
+## Usage
+Create a new view that uses the form helper `s3_uploader_form`:
+```ruby
+<%= s3_uploader_form callback_url: model_url, callback_param: "model[image_url]", id: "myS3Uploader" do %>
+  <%= file_field_tag :file, multiple: true %>
+<% end %>
+```
+
+Note: Its required that the file_field_tag is named 'file'.
+
+Then in your application.js.coffee, call the S3Uploader jQuery plugin on the element you created above:
+```coffeescript
+jQuery ->
+  $("#myS3Uploader").S3Uploader()
+```
+
+Optionally, you can also place this template in the same view for the progress bars:
+```js+erb
+<script id="template-upload" type="text/x-tmpl">
+<div id="file-{%=o.unique_id%}" class="upload">
+  {%=o.name%}
+  <div class="progress"><div class="bar" style="width: 0%"></div></div>
+</div>
+</script>
+```
+
+## Options for form helper
+* `callback_url:` url that is POST'd to after file is uploaded to S3. If you don't specify this option, no callback to the server will be made after the file has uploaded to S3.
+* `callback_method:` Defaults to POST. Use PUT and remove the multiple option from your file field to update a model.
+* `callback_param:` parameter value for the POST in which the key will be the URL of the file on S3. If for example this is set to "model[image_url]" then the data posted would be `model[image_url] : http://bucketname.s3.amazonws.com/filename.ext`
+* `key:` key on s3. defaults to `"uploads/{timestamp}-{unique_id}-#{SecureRandom.hex}/${filename}"`. `{timestamp}` and `{unique_id}` are special substitution params that will be populated by the javascript with values for the current upload. `${filename}` is a special s3 param that will be populated with the original upload file name. Needs to be at least `"${filename}"`. It is highly recommended to use both `{unique_id}`, which will prevent collisions when uploading files with the same name (such as from a mobile device, where every photo is named image.jpg), and a server-generated random value such as `#{SecureRandom.hex}`, which adds further collision protection with other uploaders.
+* `key_starts_with:` constraint on key on s3. Defaults to `uploads/`. if you change the `key` option, make sure this starts with what you put there. If you set this as a blank string the upload path to s3 can be anything - not recommended!
+* `acl:` acl for files uploaded to s3, defaults to "public-read"
+* `max_file_size:` maximum file size, defaults to 500.megabytes
+* `id:` html id for the form, its recommended that you give the form an id so you can reference with the jQuery plugin.
+* `class:` optional html class for the form.
+* `data:` Optional html data
+
+### Persisting the S3 url
+It is recommended that you persist the image_url that is sent back from the POST request (to the url given to the `post` option and as the key given in the `as` option). So to access your files later.
+
+One way to do this is to make sure you have `resources model` in your routes file, and add the `image_url` (or whatever you would like to name it) attribute to your model, and then make sure you have the create action in your controller for that model.
+
+You could then have your create action render a javascript file like this:
+**create.js.erb**
+```ruby
+<% if @model.new_record? %>
+  alert("Failed to upload model: <%= j @model.errors.full_messages.join(', ').html_safe %>");
+<% else %>
+  $("#container").append("<%= j render(@model) %>");
+<% end %>
+```
+So that javascript code would be executed after the model instance is created, without a page refresh. See [@rbates's gallery-jquery-fileupload](https://github.com/railscasts/383-uploading-to-amazon-s3/tree/master/gallery-jquery-fileupload)) for an example of that method.
+
+Note: the POST request to the rails app also includes the following parameters `filesize`, `filetype`, `filename` and `filepath`.
+
+### Advanced Customizations
+Feel free to override the styling for the progress bars in s3_direct_upload_progress_bars.css, look at the source for inspiration.
+
+Also feel free to write your own js to interface with jquery-file-upload. You might want to do this to do custom validations on the files before it is sent to S3 for example.
+To do this remove `s3_direct_upload` from your application.js and include the necessary jquery-file-upload scripts in your asset pipeline (they are included in this gem automatically):
+```cofeescript
+#= require jquery-fileupload/basic
+#= require jquery-fileupload/vendor/tmpl
+```
+Use the javascript in `s3_direct_upload` as a guide.
+
+
+## Options for S3Upload jQuery Plugin
+
+* `path:` manual path for the files on your s3 bucket. Example: `path/to/my/files/on/s3`
+  Note: Your path MUST start with the option you put in your form builder for `key_starts_with`, or else you will get S3 permission errors. The file path in your s3 bucket will be `path + key`.
+* `additional_data:` You can send additional data to your rails app in the persistence POST request. This would be accessible in your params hash as  `params[:key][:value]`
+  Example: `{key: value}`
+* `remove_completed_progress_bar:` By default, the progress bar will be removed once the file has been successfully uploaded. You can set this to `false` if you want to keep the progress bar.
+* `remove_failed_progress_bar:` By default, the progress bar will not be removed when uploads fail. You can set this to `true` if you want to remove the progress bar.
+* `before_add:` Callback function that executes before a file is added to the queue. It is passed file object and expects `true` or `false` to be returned. This could be useful if you would like to validate the filenames of files to be uploaded for example. If true is returned file will be uploaded as normal, false will cancel the upload.
+* `progress_bar_target:` The jQuery selector for the element where you want the progress bars to be appended to. Default is the form element.
+* `click_submit_target:` The jQuery selector for the element you wish to add a click handler to do the submitting instead of submiting on file open.
+
+### Example with all options.
+```coffeescript
+jQuery ->
+  $("#myS3Uploader").S3Uploader
+    path: 'path/to/my/files/on/s3'
+    additional_data: {key: 'value'}
+    remove_completed_progress_bar: false
+    before_add: myCallBackFunction() # must return true or false if set
+    progress_bar_target: $('.js-progress-bars')
+    click_submit_target: $('.submit-target')
+```
+### Example with single file upload bar without script template
+
+This demonstrates how to use progress_bar_target and allow_multiple_files (only works with false option - single file) to show only one progress bar without script template.
+
+```coffeescript
+jQuery ->
+  $("#myS3Uploader").S3Uploader
+    progress_bar_target: $('.js-progress-bars')
+    allow_multiple_files: false
+```
+
+Target for progress bar
+
+```html
+<div class="upload js-progress-bars">
+  <div class="progress">
+    <div class="bars"> </div>
+  </div>
+</div>
+```
+
+
+
+
+### Public methods
+You can change the settings on your form later on by accessing the jQuery instance:
+
+```coffeescript
+jQuery ->
+  v = $("#myS3Uploader").S3Uploader()
+  ...
+  v.path("new/path/") #only works when the key_starts_with option is blank. Not recommended.
+  v.additional_data("newdata")
+```
+
+### Javascript Events Hooks
+
+#### First upload started
+`s3_uploads_start` is fired once when any batch of uploads is starting.
+```coffeescript
+$('#myS3Uploader').bind 's3_uploads_start', (e) ->
+  alert("Uploads have started")
+```
+
+#### Successfull upload
+When a file has been successfully uploaded to S3, the `s3_upload_complete` is triggered on the form. A `content` object is passed along with the following attributes :
+
+* `url`       The full URL to the uploaded file on S3.
+* `filename`  The original name of the uploaded file.
+* `filepath`  The path to the file (without the filename or domain)
+* `filesize`  The size of the uploaded file.
+* `filetype`  The type of the uploaded file.
+
+This hook could be used for example to fill a form hidden field with the returned S3 url :
+```coffeescript
+$('#myS3Uploader').bind "s3_upload_complete", (e, content) ->
+  $('#someHiddenField').val(content.url)
+```
+
+#### Failed upload
+When an error occured during the transferm the `s3_upload_failed` is triggered on the form with the same `content` object is passed for the successful upload with the addition of the `error_thrown` attribute. The most basic way to handle this error would be to display an alert message to the user in case the upload fails :
+```coffeescript
+$('#myS3Uploader').bind "s3_upload_failed", (e, content) ->
+  alert("#{content.filename} failed to upload : #{content.error_thrown}")
+```
+
+#### All uploads completed
+When all uploads finish in a batch an `s3_uploads_complete` event will be triggered on `document`, so you could do something like:
+```coffeescript
+$(document).bind 's3_uploads_complete', ->
+    alert("All Uploads completed")
+```
+
+#### Rails AJAX Callbacks
+
+In addition, the regular rails ajax callbacks will trigger on the form with regards to the POST to the server.
+
+```coffeescript
+$('#myS3Uploader').bind "ajax:success", (e, data) ->
+  alert("server was notified of new file on S3; responded with '#{data}")
+```
+
+## Cleaning old uploads on S3
+You may be processing the files upon upload and reuploading them to another
+bucket or directory. If so you can remove the originali files by running a
+rake task.
+
+First, add the fog gem to your `Gemfile` and run `bundle`:
+```ruby
+  require 'fog'
+```
+
+Then, run the rake task to delete uploads older than 2 days:
+```
+  $ rake s3_direct_upload:clean_remote_uploads
+  Deleted file with key: "uploads/20121210T2139Z_03846cb0329b6a8eba481ec689135701/06 - PCR_RYA014-25.jpg"
+  Deleted file with key: "uploads/20121210T2139Z_03846cb0329b6a8eba481ec689135701/05 - PCR_RYA014-24.jpg"
+  $
+```
+
+Optionally customize the prefix used for cleaning (default is `uploads/#{2.days.ago.strftime('%Y%m%d')}`):
+**config/initalizers/s3_direct_upload.rb**
+```ruby
+S3DirectUpload.config do |c|
+  # ...
+  c.prefix_to_clean = "my_path/#{1.week.ago.strftime('%y%m%d')}"
+end
+```
+
+Alternately, if you'd prefer for S3 to delete your old uploads automatically, you can do
+so by setting your bucket's
+[Lifecycle Configuration](http://docs.aws.amazon.com/AmazonS3/latest/UG/LifecycleConfiguration.html).
+
+## Contributing / TODO
+This is just a simple gem that only really provides some javascript and a form helper.
+This gem could go all sorts of ways based on what people want and how people contribute.
+Ideas:
+* More specs!
+* More options to control file types, ability to batch upload.
+* More convention over configuration on rails side
+* Create generators.
+* Model methods.
+* Model method to delete files from s3
+
+
+## Credit
+This gem is basically a small wrapper around code that [Ryan Bates](http://github.com/rbates) wrote for [Railscast#383](http://railscasts.com/episodes/383-uploading-to-amazon-s3). Most of the code in this gem was extracted from [gallery-jquery-fileupload](https://github.com/railscasts/383-uploading-to-amazon-s3/tree/master/gallery-jquery-fileupload).
+
+Thank you Ryan Bates!
+
+This code also uses the excellecnt [jQuery-File-Upload](https://github.com/blueimp/jQuery-File-Upload), which is included in this gem by its rails counterpart [jquery-fileupload-rails](https://github.com/tors/jquery-fileupload-rails)

data/app/assets/javascripts/s3_direct_upload.js.coffee

@@ -0,0 +1,153 @@
+#= require jquery-fileupload/basic
+#= require jquery-fileupload/vendor/tmpl
+
+$ = jQuery
+
+$.fn.S3Uploader = (options) ->
+
+  # support multiple elements
+  if @length > 1
+    @each ->
+      $(this).S3Uploader options
+
+    return this
+
+  $uploadForm = this
+
+  settings =
+    path: ''
+    additional_data: null
+    before_add: null
+    remove_completed_progress_bar: true
+    remove_failed_progress_bar: false
+    progress_bar_target: null
+    click_submit_target: null
+    allow_multiple_files: true
+
+  $.extend settings, options
+
+  current_files = []
+  forms_for_submit = []
+  if settings.click_submit_target
+    settings.click_submit_target.click ->
+      form.submit() for form in forms_for_submit
+      false
+
+  setUploadForm = ->
+    $uploadForm.fileupload
+
+      add: (e, data) ->
+        file = data.files[0]
+        file.unique_id = Math.random().toString(36).substr(2,16)
+
+        unless settings.before_add and not settings.before_add(file)
+          current_files.push data
+          if $('#template-upload').length > 0
+            data.context = $($.trim(tmpl("template-upload", file)))
+            $(data.context).appendTo(settings.progress_bar_target || $uploadForm)
+          else if !settings.allow_multiple_files
+            data.context = settings.progress_bar_target
+          if settings.click_submit_target
+            if settings.allow_multiple_files
+              forms_for_submit.push data
+            else
+              forms_for_submit = [data]
+          else
+            data.submit()
+
+      start: (e) ->
+        $uploadForm.trigger("s3_uploads_start", [e])
+
+      progress: (e, data) ->
+        if data.context
+          progress = parseInt(data.loaded / data.total * 100, 10)
+          data.context.find('.bar').css('width', progress + '%')
+
+      done: (e, data) ->
+        content = build_content_object $uploadForm, data.files[0], data.result
+
+        callback_url = $uploadForm.data('callback-url')
+        if callback_url
+          content[$uploadForm.data('callback-param')] = content.url
+
+          $.ajax
+            type: $uploadForm.data('callback-method')
+            url: callback_url
+            data: content
+            beforeSend: ( xhr, settings )       -> $uploadForm.trigger( 'ajax:beforeSend', [xhr, settings] )
+            complete:   ( xhr, status )         -> $uploadForm.trigger( 'ajax:complete', [xhr, status] )
+            success:    ( data, status, xhr )   -> $uploadForm.trigger( 'ajax:success', [data, status, xhr] )
+            error:      ( xhr, status, error )  -> $uploadForm.trigger( 'ajax:error', [xhr, status, error] )
+
+        data.context.remove() if data.context && settings.remove_completed_progress_bar # remove progress bar
+        $uploadForm.trigger("s3_upload_complete", [content])
+
+        current_files.splice($.inArray(data, current_files), 1) # remove that element from the array
+        $uploadForm.trigger("s3_uploads_complete", [content]) unless current_files.length
+
+      fail: (e, data) ->
+        content = build_content_object $uploadForm, data.files[0], data.result
+        content.error_thrown = data.errorThrown
+
+        data.context.remove() if data.context && settings.remove_failed_progress_bar # remove progress bar
+        $uploadForm.trigger("s3_upload_failed", [content])
+
+      formData: (form) ->
+        data = form.serializeArray()
+        fileType = ""
+        if "type" of @files[0]
+          fileType = @files[0].type
+        data.push
+          name: "content-type"
+          value: fileType
+
+        key = $uploadForm.data("key").replace('{timestamp}', new Date().getTime()).replace('{unique_id}', @files[0].unique_id)
+
+        # substitute upload timestamp and unique_id into key
+        data[1].value = settings.path + key
+
+        # IE <= 9 doesn't have XHR2 hence it can't use formData
+        # replace 'key' field to submit form
+        unless 'FormData' of window
+          $uploadForm.find("input[name='key']").val(settings.path + key)
+        data
+
+  build_content_object = ($uploadForm, file, result) ->
+    content = {}
+    if result # Use the S3 response to set the URL to avoid character encodings bugs
+      content.url      = $(result).find("Location").text()
+      content.filepath = $('<a />').attr('href', content.url)[0].pathname
+    else # IE <= 9 return a null result object so we use the file object instead
+      domain           = $uploadForm.attr('action')
+      content.filepath = $uploadForm.find('input[name=key]').val().replace('/${filename}', '')
+      content.url      = domain + content.filepath + '/' + encodeURIComponent(file.name)
+
+    content.filename   = file.name
+    content.filesize   = file.size if 'size' of file
+    content.filetype   = file.type if 'type' of file
+    content.unique_id  = file.unique_id if 'unique_id' of file
+    content.relativePath = build_relativePath(file) if has_relativePath(file)
+    content = $.extend content, settings.additional_data if settings.additional_data
+    content
+
+  has_relativePath = (file) ->
+    file.relativePath || file.webkitRelativePath
+
+  build_relativePath = (file) ->
+    file.relativePath || (file.webkitRelativePath.split("/")[0..-2].join("/") + "/" if file.webkitRelativePath)
+
+  #public methods
+  @initialize = ->
+    # Save key for IE9 Fix
+    $uploadForm.data("key", $uploadForm.find("input[name='key']").val())
+
+    setUploadForm()
+    this
+
+  @path = (new_path) ->
+    settings.path = new_path
+
+  @additional_data = (new_data) ->
+    settings.additional_data = new_data
+
+  @initialize()

data/app/assets/stylesheets/s3_direct_upload_progress_bars.css.scss

@@ -0,0 +1,17 @@
+.upload {
+  border-top: solid 1px #CCC;
+  width: 400px;
+  padding-top: 10px;
+  margin-top: 10px;
+
+  .progress {
+    margin-top: 8px;
+    border: solid 1px #555;
+    border-radius: 3px;
+    -moz-border-radius: 3px;
+    .bar {
+      height: 10px;
+      background: #3EC144;
+    }
+  }
+}

data/lib/s3_direct_upload/config_aws.rb

@@ -0,0 +1,18 @@
+require "singleton"
+
+module S3DirectUpload
+  class Config
+    include Singleton
+
+    ATTRIBUTES = [:access_key_id, :secret_access_key, :bucket, :prefix_to_clean, :region, :url]
+
+    attr_accessor *ATTRIBUTES
+  end
+
+  def self.config
+    if block_given?
+      yield Config.instance
+    end
+    Config.instance
+  end
+end

data/lib/s3_direct_upload/engine.rb

@@ -0,0 +1,4 @@
+module S3DirectUpload
+  class Engine < ::Rails::Engine 
+  end
+end

data/lib/s3_direct_upload/form_helper.rb

@@ -0,0 +1,96 @@
+module S3DirectUpload
+  module UploadHelper
+    def s3_uploader_form(options = {}, &block)
+      uploader = S3Uploader.new(options)
+      form_tag(uploader.url, uploader.form_options) do
+        uploader.fields.map do |name, value|
+          hidden_field_tag(name, value)
+        end.join.html_safe + capture(&block)
+      end
+    end
+
+    class S3Uploader
+      def initialize(options)
+        @key_starts_with = options[:key_starts_with] || "uploads/"
+        @options = options.reverse_merge(
+          aws_access_key_id: S3DirectUpload.config.access_key_id,
+          aws_secret_access_key: S3DirectUpload.config.secret_access_key,
+          bucket: S3DirectUpload.config.bucket,
+          region: S3DirectUpload.config.region || "s3",
+          url: S3DirectUpload.config.url,
+          ssl: true,
+          expiration: 10.hours.from_now.utc.iso8601,
+          max_file_size: 500.megabytes,
+          callback_method: "POST",
+          callback_param: "file",
+          key_starts_with: @key_starts_with,
+          key: key
+        )
+      end
+
+      def form_options
+        {
+          id: @options[:id],
+          class: @options[:class],
+          method: "post",
+          authenticity_token: false,
+          multipart: true,
+          data: {
+            callback_url: @options[:callback_url],
+            callback_method: @options[:callback_method],
+            callback_param: @options[:callback_param]
+          }.reverse_merge(@options[:data] || {})
+        }
+      end
+
+      def fields
+        {
+          :key => @options[:key] || key,
+          :acl => @options[:acl],
+          "AWSAccessKeyId" => @options[:aws_access_key_id],
+          :policy => policy,
+          :signature => signature,
+          :success_action_status => "201",
+          'X-Requested-With' => 'xhr'
+        }
+      end
+
+      def key
+        @key ||= "#{@key_starts_with}{timestamp}-{unique_id}-#{SecureRandom.hex}/${filename}"
+      end
+
+      def url
+        @options[:url] || "http#{@options[:ssl] ? 's' : ''}://#{@options[:region]}.amazonaws.com/#{@options[:bucket]}/"
+      end
+
+      def policy
+        Base64.encode64(policy_data.to_json).gsub("\n", "")
+      end
+
+      def policy_data
+        {
+          expiration: @options[:expiration],
+          conditions: [
+            ["starts-with", "$utf8", ""],
+            ["starts-with", "$key", @options[:key_starts_with]],
+            ["starts-with", "$x-requested-with", ""],
+            ["content-length-range", 0, @options[:max_file_size]],
+            ["starts-with","$content-type", @options[:content_type_starts_with] ||""],
+            {bucket: @options[:bucket]},
+            {acl: @options[:acl]},
+            {success_action_status: "201"}
+          ] + (@options[:conditions] || [])
+        }
+      end
+
+      def signature
+        Base64.encode64(
+          OpenSSL::HMAC.digest(
+            OpenSSL::Digest::Digest.new('sha1'),
+            @options[:aws_secret_access_key], policy
+          )
+        ).gsub("\n", "")
+      end
+    end
+  end
+end

data/lib/s3_direct_upload/railtie.rb

@@ -0,0 +1,7 @@
+module S3DirectUpload
+  class Railtie < Rails::Railtie
+    initializer "railtie.configure_rails_initialization" do |app|
+      app.middleware.use JQuery::FileUpload::Rails::Middleware
+    end
+  end
+end

data/lib/s3_direct_upload/version.rb

@@ -0,0 +1,3 @@
+module S3DirectUpload
+  VERSION = "0.1.6"
+end

data/lib/s3_direct_upload.rb

@@ -0,0 +1,13 @@
+require 's3_direct_upload/version'
+require 'jquery-fileupload-rails' if defined?(Rails)
+
+require 'base64'
+require 'openssl'
+require 'digest/sha1'
+
+require 's3_direct_upload/config_aws'
+require 's3_direct_upload/form_helper'
+require 's3_direct_upload/engine' if defined?(Rails)
+require 's3_direct_upload/railtie' if defined?(Rails)
+
+ActionView::Base.send(:include, S3DirectUpload::UploadHelper) if defined?(ActionView::Base)

data/lib/tasks/s3_direct_upload.rake

@@ -0,0 +1,57 @@
+namespace :s3_direct_upload do
+  desc "Removes old uploads from specified s3 bucket/directory -- Useful when uploads are processed into another directory"
+  task :clean_remote_uploads => :environment do
+    require 'thread'
+    require 'fog'
+
+    s3     = Fog::Storage::AWS.new(aws_access_key_id: S3DirectUpload.config.access_key_id, aws_secret_access_key: S3DirectUpload.config.secret_access_key)
+    bucket = S3DirectUpload.config.bucket
+    prefix = S3DirectUpload.config.prefix_to_clean || "uploads/#{2.days.ago.strftime('%Y%m%d')}"
+
+    queue         = Queue.new
+    semaphore     = Mutex.new
+    threads       = []
+    thread_count  = 20
+    total_listed  = 0
+    total_deleted = 0
+
+    threads << Thread.new do
+      Thread.current[:name] = "get files"
+      # Get all the files from this bucket. Fog handles pagination internally.
+      s3.directories.get("#{bucket}").files.all({prefix: prefix}).each do |file|
+        queue.enq(file)
+        total_listed += 1
+      end
+      # Add a final EOF message to signal the deletion threads to stop.
+      thread_count.times { queue.enq(:EOF) }
+    end
+
+    # Delete all the files in the queue until EOF with N threads.
+    thread_count.times do |count|
+      threads << Thread.new(count) do |number|
+        Thread.current[:name] = "delete files(#{number})"
+        # Dequeue until EOF.
+        file = nil
+        while file != :EOF
+          # Dequeue the latest file and delete it. (Will block until it gets a new file.)
+          file = queue.deq
+          unless file == :EOF
+            file.destroy
+            puts %Q{Deleted file with key: "#{file.key}"}
+          end
+          # Increment the global synchronized counter.
+          semaphore.synchronize {total_deleted += 1}
+        end
+      end
+    end
+
+    # Wait for the threads to finish.
+    threads.each do |t|
+      begin
+        t.join
+      rescue RuntimeError => e
+        puts "Failure on thread #{t[:name]}: #{e.message}"
+      end
+    end
+  end
+end

metadata

@@ -0,0 +1,121 @@
+--- !ruby/object:Gem::Specification
+name: s3_private_direct_upload
+version: !ruby/object:Gem::Version
+  version: 0.1.6
+  prerelease: 
+platform: ruby
+authors:
+- Wayne Hoover
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-07-16 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '3.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '3.2'
+- !ruby/object:Gem::Dependency
+  name: coffee-rails
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 3.2.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 3.2.1
+- !ruby/object:Gem::Dependency
+  name: sass-rails
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 3.2.5
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 3.2.5
+- !ruby/object:Gem::Dependency
+  name: jquery-fileupload-rails
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 0.4.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 0.4.1
+description: Direct Upload to Amazon S3 With CORS and jquery-file-upload
+email:
+- w@waynehoover.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- lib/s3_direct_upload/config_aws.rb
+- lib/s3_direct_upload/engine.rb
+- lib/s3_direct_upload/form_helper.rb
+- lib/s3_direct_upload/railtie.rb
+- lib/s3_direct_upload/version.rb
+- lib/s3_direct_upload.rb
+- lib/tasks/s3_direct_upload.rake
+- app/assets/javascripts/s3_direct_upload.js.coffee
+- app/assets/stylesheets/s3_direct_upload_progress_bars.css.scss
+- LICENSE
+- README.md
+homepage: ''
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.23
+signing_key: 
+specification_version: 3
+summary: Gives a form helper for Rails which allows direct uploads to s3. Based on
+  RailsCast#383. Makes files private on S3
+test_files: []