s3_relay 0.0.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: dcccfe11a546093475043762594d3df9af34b075
+  data.tar.gz: 79acd9bdd1b6b6a6877341a08b9ec5c923fb3f3d
+SHA512:
+  metadata.gz: 504414953661b8a7fc6bc069fe90eef08005f1fdd0d5a3918a585aa132f684a831bee1c286475c722d129d331890b0545eb0d5e601c9fd3f3a35e94c94a265e2
+  data.tar.gz: 94578b8eec4bbe0a24855359d0098aaac6bbbebcf8e239d9a3b811ab98fe973eef0b37d625be722d5cf7dd56e55a99c5f76ac8adc2300f3ecebd7375784b87f5

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2014 Kenny Johnston
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,209 @@
+# s3_relay
+
+Enables direct file uploads to Amazon S3 and provides a flexible pattern for
+your Rails app to asynchronously ingest the files.
+
+## Overview
+
+This Rails engine allows you to quickly implement direct uploads to Amazon S3
+from your Rails 3.1+ / 4.x application.  It does not depend on any specific file
+upload libraries, UI frameworks or AWS gems, like other solutions tend to.
+
+It works by utilizing Amazon S3's
+[Cross-Origin Resource Sharing](http://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html)
+to permit browser-based uploads directly to S3 with presigned URLs generated by
+this gem with your application's API credentials.  As each file is uploaded,
+the gem persists detail about the uploaded file in your application's database.
+This table should be thought of much like a queue - think
+[DelayedJob](https://github.com/collectiveidea/delayed_job) for your
+uploaded-but-not-yet-ingested file uploads.
+
+How (and if) you choose to import each uploaded file into your processing
+library of choice is completely up to you.  The gem tracks the state of each
+upload so that you may used the provided `.pending` scope and `mark_imported!`
+method to fetch, process (via your background processor of choice), then
+mark-off each upload record whose file has been successfully ingested by your
+app.
+
+## Features
+
+* Files can be uploaded before or after your parent object has been saved.
+* File upload fields can be placed inside or outside of your parent object's
+form.
+* File uploads display completion progress.
+* Boilerplate styling can be used or easily replaced.
+* All uploads are set to private by default, however support for other ACLs
+is in the plans.
+* All uploads are marked for [Server-Side Encryption by AWS](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingEncryption.html) so that they are encrypted upon
+write to disk.  S3 then decrypts and streams the files as they are downloaded.
+* Models can have multiple types of uploads and specify for each if only one
+file is permitted or if multiple are.
+* Multiple files can upload concurrently to S3.
+
+## Technology & Requirements
+
+Uploads are made possible by use of the `FormData` object, defined in
+[XMLHttpRequest Level 2](http://dev.w3.org/2006/webapi/XMLHttpRequest-2/).
+Many people are broadly referring to this as being provided by HTML5, but
+technically it's part of the aforementioned spec that browsers have been
+adhering to for a couple of major versions now.  Even IE, wuh?
+
+The latest versions of all of the following are ideal, but here are the gem's
+minimum requirements:
+
+* Ruby 1.9.3+
+* Rails 3.1+
+* Modern versions of Chrome, Safari, FireFox or IE 10+
+  * Note: Progress bars are currently disabled in IE
+  * Note: IE <= 9 users will be instructed to upgrade their browser upon
+  selecting a file
+
+## Demo
+
+See a demo application using `s3_relay` [here](https://github.com/kjohnston/s3_relay-demo).
+
+## Configuring CORS
+
+Edit your S3 bucket's CORS Configuration to resemble the following:
+
+```
+<CORSConfiguration>
+  <CORSRule>
+    <AllowedOrigin>*</AllowedOrigin>
+    <AllowedMethod>POST</AllowedMethod>
+    <AllowedHeader>Content-Type</AllowedHeader>
+    <AllowedHeader>origin</AllowedHeader>
+  </CORSRule>
+</CORSConfiguration>
+```
+
+Note: The example above is a starting point for development.  Obviously, you
+don't want to permit requests from any domain to upload to your S3 bucket.
+Please see the [AWS Documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html)
+to learn how to lock it down further.
+
+## Installation
+
+* Add `gem "s3_relay"` to your Gemfile and run `bundle`.
+* Add migrations to your app with `rake s3_relay:install:migrations db:migrate`.
+* Add `mount S3Relay::Engine => "/s3_relay"` to the top of your routes file.
+* Add `require s3_relay` to your JavaScript manifest.
+* [Optional] Add `require s3_relay` to your Style Sheet manifest.
+* Add the following environment variables to your app:
+
+```
+S3_RELAY_ACCESS_KEY_ID="abc123"
+S3_RELAY_SECRET_ACCESS_KEY="xzy456"
+S3_RELAY_REGION="us-west-2"
+S3_RELAY_BUCKET="some-s3-bucket"
+S3_RELAY_ACL="private"
+```
+
+## Use
+
+### Add upload definitions to your model
+
+```ruby
+class Product < ActiveRecord::Base
+  s3_relay :icon_upload
+  s3_relay :photo_uploads, has_many: true
+end
+```
+
+### Restricting uploads to authenticated users
+
+If your app's file uploads need to be restricted to logged in users, simply
+override the following method in your application controller to call any
+authentication method you're currently using.
+
+```ruby
+def authenticate_for_s3_relay
+  authenticate_user!  # Devise example
+end
+```
+
+### Add virtual attributes to your controller's Strong Parameters config
+
+```ruby
+product_params = params.require(:product)
+  .permit(:name, :new_icon_upload_uuid, new_photo_uploads_uuids: [])
+
+@product = Product.new(product_params)
+```
+
+### Add file upload fields to your views
+
+```erb
+<%= s3_relay_field @product, :icon_upload %>
+<%= s3_relay_field @product, :photo_uploads, multiple: true %>
+```
+
+### Process uploads asynchronously
+
+Use your background job processor of choice to process uploads pending
+ingestion (and image processing) by your app.
+
+Say you're using [Resque](https://github.com/resque/resque) and [CarrierWave](https://github.com/carrierwaveuploader/carrierwave), you could define a job class:
+
+```ruby
+class ProductPhotoImporter
+  @queue = :photo_import
+
+  def self.perform(product_id)
+    @product = Product.find(id)
+
+    @product.photo_uploads.pending.each do |upload|
+      @product.photos.create(remote_file_url: upload.private_url)
+      upload.mark_processed!
+    end
+  end
+end
+```
+
+Then, enqueue a job from your controller, callback, service object, etc.
+whenever there may be new uploads to process:
+
+```ruby
+Resque.enqueue(ProductPhotoImporter, product.id)
+```
+
+### Restricting the objects uploads can be associated with
+
+Remember the time when that guy found a way to a submit Github form in such a
+way that it linked a new SSH key he provided to DHH's user record?  No bueno.
+Don't let your users attach files to objects they don't have access to.
+
+You can prevent this by defining a method in ApplicationController that
+filters out the parent object params passed during upload creation if your logic
+finds that the user doesn't have access to the parent object in question.  Ex:
+
+```ruby
+def order_file_uploads_params(parent)
+  if parent.user == current_user
+    # Yep, that's your order, you can add files to it
+    { parent: parent }
+  else
+    # Nope, you're trying to add a file to someone else's order, or whatever
+    { }
+  end
+end
+```
+
+## Contributing
+
+1. [Fork it](https://github.com/kjohnston/s3_relay/fork)
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. [Create a Pull Request](https://github.com/kjohnston/s3_relay/pull/new)
+
+## Contributors
+
+Many thanks go to the following who have contributed to making this gem even better:
+
+[your name here]
+
+## License
+
+* Freely distributable and licensed under the [MIT license](http://kjohnston.mit-license.org/license.html).
+* Copyright (c) 2014 Kenny Johnston

data/Rakefile

@@ -0,0 +1,31 @@
+begin
+  require "bundler/setup"
+rescue LoadError
+  puts "You must `gem install bundler` and `bundle install` to run rake tasks"
+end
+
+require "rdoc/task"
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = "rdoc"
+  rdoc.title    = "S3Relay"
+  rdoc.options << "--line-numbers"
+  rdoc.rdoc_files.include("README.rdoc")
+  rdoc.rdoc_files.include("lib/**/*.rb")
+end
+
+APP_RAKEFILE = File.expand_path("../test/dummy/Rakefile", __FILE__)
+load "rails/tasks/engine.rake"
+
+Bundler::GemHelper.install_tasks
+
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "lib"
+  t.libs << "test"
+  t.pattern = "test/**/*_test.rb"
+  t.verbose = false
+end
+
+task default: :test

data/app/assets/javascripts/s3_relay.coffee

@@ -0,0 +1,112 @@
+displayFailedUpload = (progressColumn=null) ->
+  if progressColumn
+    progressColumn.text("File could not be uploaded")
+  else
+    alert("File could not be uploaded")
+
+saveUrl = (container, uuid, filename, contentType, publicUrl) ->
+  privateUrl = null
+
+  $.ajax
+    type: "POST"
+    url: "/s3_relay/uploads"
+    async: false
+    data:
+      parent_type: container.data("parentType")
+      parent_id: container.data("parentId")
+      association: container.data("association")
+      uuid: uuid
+      filename: filename
+      content_type: contentType
+      public_url: publicUrl
+    success: (data, status, xhr) ->
+      privateUrl = data.private_url
+    error: (xhr) ->
+      console.log xhr.responseText
+
+  return privateUrl
+
+uploadFiles = (container) ->
+  fileInput = $("input.s3r-field", container)
+  files = fileInput.get(0).files
+  uploadFile(container, file) for file in files
+  fileInput.val("")
+
+uploadFile = (container, file) ->
+  fileName = file.name
+
+  $.ajax
+    type: "GET"
+    url: "/s3_relay/uploads/new"
+    async: false
+    success: (data, status, xhr) ->
+      formData = new FormData()
+      xhr = new XMLHttpRequest()
+      endpoint = data.endpoint
+
+      formData.append("AWSAccessKeyID", data.awsaccesskeyid)
+      formData.append("x-amz-server-side-encryption", data.x_amz_server_side_encryption)
+      formData.append("key", data.key)
+      formData.append("success_action_status", data.success_action_status)
+      formData.append("acl", data.acl)
+      formData.append("policy", data.policy)
+      formData.append("signature", data.signature)
+      formData.append("content-type", file.type)
+      formData.append("file", file)
+
+      uuid = data.uuid
+
+      uploadList = $(".s3r-upload-list", container)
+      uploadList.prepend("<tr id='#{uuid}'><td><div class='s3r-file-url'>#{fileName}</div></td><td class='s3r-progress'><div class='s3r-bar' style='display: none;'><div class='s3r-meter'></div></div></td></tr>")
+      fileColumn = $(".s3r-upload-list ##{uuid} .s3r-file-url", container)
+      progressColumn = $(".s3r-upload-list ##{uuid} .s3r-progress", container)
+      progressBar = $(".s3r-bar", progressColumn)
+      progressMeter = $(".s3r-meter", progressColumn)
+
+      xhr.upload.addEventListener "progress", (ev) ->
+        if ev.position
+          percentage = ((ev.position / ev.totalSize) * 100.0).toFixed(0)
+          progressBar.show()
+          progressMeter.css "width", "#{percentage}%"
+        else
+          progressColumn.text("Uploading...")  # IE can't get position
+
+      xhr.onreadystatechange = (ev) ->
+        if xhr.readyState is 4
+          progressColumn.text("")  # IE can't get position
+          progressBar.remove()
+
+          if xhr.status == 201
+            contentType = file.type
+            publicUrl = $("Location", xhr.responseXML).text()
+            privateUrl = saveUrl(container, uuid, fileName, contentType, publicUrl)
+
+            if privateUrl == null
+              displayFailedUpload(progressColumn)
+            else
+              fileColumn.html("<a href='#{privateUrl}'>#{fileName}</a>")
+
+              virtualAttr = "#{container.data('parentType')}[new_#{container.data('association')}_uuids]"
+              hiddenField = "<input type='hidden' name='#{virtualAttr}[]' value='#{uuid}' />"
+              container.append(hiddenField)
+
+          else
+            displayFailedUpload(progressColumn)
+            console.log $("Message", xhr.responseXML).text()
+
+      xhr.open "POST", endpoint, true
+      xhr.send formData
+    error: (xhr) ->
+      displayFailedUpload()
+      console.log xhr.responseText
+
+jQuery ->
+
+  $(document).on "change", ".s3r-field", ->
+    $this = $(this)
+
+    if !!window.FormData
+      uploadFiles($this.parent())
+    else
+      $this.hide()
+      $this.parent().append("<p>Your browser can't handle file uploads, please switch to <a href='http://google.com/chrome'>Google Chrome</a>.</p>")

data/app/assets/stylesheets/s3_relay.css

@@ -0,0 +1,31 @@
+.s3r-upload-list {
+  border: 1px solid #ccc;
+  border-collapse: collapse;
+  margin: 15px 0;
+}
+
+.s3r-upload-list td {
+  border-bottom: 1px solid #ccc;
+  padding: .5em;
+  width: 180px;
+}
+
+.s3r-file-url {
+  overflow: hidden;
+  text-overflow: ellipsis;
+  white-space: nowrap;
+  width: 180px;
+}
+
+.s3r-bar {
+  background-color: #eee;
+  border-radius: 3px;
+  height: 15px;
+  width: 180px;
+}
+
+.s3r-meter {
+  background-color: #43ac6a;
+  border-radius: 3px;
+  height: 15px;
+}

data/app/controllers/s3_relay/uploads_controller.rb

@@ -0,0 +1,65 @@
+class S3Relay::UploadsController < ApplicationController
+
+  before_filter :authenticate
+  skip_before_filter :verify_authenticity_token
+
+  def new
+    render json: S3Relay::UploadPresigner.new.form_data
+  end
+
+  def create
+    @upload = S3Relay::Upload.new(upload_attrs)
+
+    if @upload.save
+      render json: { private_url: @upload.private_url }, status: 201
+    else
+      render json: { errors: @upload.errors }, status: 422
+    end
+  end
+
+  protected
+
+  def authenticate
+    if respond_to?(:authenticate_for_s3_relay)
+      authenticate_for_s3_relay
+    end
+  end
+
+  def parent_attrs
+    type = params[:parent_type].try(:classify)
+    id   = params[:parent_id]
+
+    return {} unless type.present? && id.present? &&
+      parent = type.constantize.find_by_id(id)
+
+    begin
+      public_send(
+        "#{type.underscore.downcase}_#{params[:association]}_params",
+        parent
+      )
+    rescue NoMethodError
+      { parent: parent }
+    end
+  end
+
+  def upload_attrs
+    attrs = {
+      upload_type:  params[:association].try(:classify),
+      uuid:         params[:uuid],
+      filename:     params[:filename],
+      content_type: params[:content_type]
+    }
+
+    attrs.merge!(parent_attrs)
+    attrs.merge!(user_attrs)
+  end
+
+  def user_attrs
+    if respond_to?(:current_user) && (id = current_user.try(:id))
+      { user_id: id }
+    else
+      {}
+    end
+  end
+
+end