webpacker_uploader 0.1.0 → 0.5.0

data/README.md

@@ -1,5 +1,9 @@
 # WebpackerUploader
 
+[![Ruby tests](https://img.shields.io/github/workflow/status/tlatsas/webpacker_uploader/Ruby%20tests?style=flat-square)](https://github.com/tlatsas/webpacker_uploader/actions)
+[![RuboCop](https://img.shields.io/github/workflow/status/tlatsas/webpacker_uploader/RuboCop?label=rubocop&style=flat-square)](https://github.com/tlatsas/webpacker_uploader/actions)
+[![Gem](https://img.shields.io/gem/v/webpacker_uploader?style=flat-square)](https://rubygems.org/gems/webpacker_uploader)
+
 Webpacker uploader provides an easy way to upload your assets to AWS S3.
 It knows which files to upload by reading the `manifest.json` file.
 
@@ -11,14 +15,14 @@ S3 + CDN, outside of your Rails application.
 Add this line to your application's Gemfile:
 
 ```ruby
-gem "webpacker_uploader"
+gem "webpacker_uploader", require: false
 ```
 
-And then execute:
+and run:
 
     $ bundle install
 
-Or install it yourself as:
+or:
 
     $ gem install webpacker_uploader
 
@@ -30,18 +34,109 @@ gem "aws-sdk-s3", require: false
 
 ## Usage
 
+Usually, in a Rake task you would add the following to upload the assets to AWS S3:
+
+```ruby
+require "webpacker_uploader"
+require "webpacker_uploader/providers/aws"
+
+provider_options = {
+  credentials: { profile_name: "staging" },
+  region: "eu-central-1",
+  bucket: "application-assets-20200929124523451600000001"
+}
+
+provider = WebpackerUploader::Providers::Aws.new(provider_options)
+WebpackerUploader.upload!(provider)
+```
+
+Currently, the only provider implemented is the AWS S3 provider.
+
+### AWS S3 provider
+
+The AWS S3 provider credentials can be configured in three ways.
+Passing a named profile name:
+
+```ruby
+provider_options = {
+  credentials: { profile_name: "staging" },
+  region: "eu-central-1",
+  bucket: "application-assets-20200929124523451600000001"
+}
+
+provider = WebpackerUploader::Providers::Aws.new(provider_options)
+```
+
+passing the access and the secret keys directly:
+
+```ruby
+provider_options = {
+  credentials: { access_key_id: "AKIAIOSFODNN7EXAMPLE", secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY" },
+  region: "eu-central-1",
+  bucket: "application-assets-20200929124523451600000001"
+}
+
+provider = WebpackerUploader::Providers::Aws.new(provider_options)
+```
+
+or using an EC2 instance profile:
+
 ```ruby
-    require 'webpacker_uploader'
-    require 'webpacker_uploader/providers/aws'
+provider_options = {
+  credentials: { instance_profile: true },
+  region: "eu-central-1",
+  bucket: "application-assets-20200929124523451600000001"
+}
 
-    provider_options = {
-      credentials: { profile_name: "staging" },
-      region: "eu-central-1",
-      bucket: "application-assets-20200929124523451600000001"
-    }
+provider = WebpackerUploader::Providers::Aws.new(provider_options)
+```
+
+### Prefix remote files
+
+Uploaded files can be prefixed by setting the `prefix` parameter during upload:
+
+```ruby
+WebpackerUploader.upload!(provider, prefix: "assets")
+```
+
+This will prefix all remote file paths with `assets` so instead of storing `packs/application-dd6b1cd38bfa093df600.css` it
+will store `assets/packs/application-dd6b1cd38bfa093df600.css`.
+
+### Configuration
+
+WebpackerUploader currently supports the following configuration options:
+
+| option               | description                                                  | default value                         |
+|----------------------|--------------------------------------------------------------|---------------------------------------|
+| ignored_extensions   | Which files uploader should skip based on the file extension | []                                    |
+| logger               | The logger to use for logging                                | ActiveSupport::Logger.new(STDOUT)     |
+| log_output           | Log output as we upload files                                | true                                  |
+| public_manifest_path | The webpack manifest path                                    | Webpacker.config.public_manifest_path |
+| public_path          | The webpack public output path                               | Webpacker.config.public_path          |
 
-    provider = WebpackerUploader::Providers::Aws.new(provider_options)
-    WebpackerUploader.upload!(provider)
+It can be configured using a block:
+
+```ruby
+WebpackerUploader.configure do |config|
+  config.ignored_extensions = [".png", ".jpg", ".webp"]
+  config.log_output = false
+  config.public_manifest_path = "path/to/manifest.json"
+  config.public_path = "path/to/public/dir"
+end
+```
+
+or directly:
+
+```ruby
+WebpackerUploader.config.log_output = false
+```
+
+The uploader used to skip `.map` files by default. This has changed (see [CHANGELOG.md](https://github.com/tlatsas/webpacker_uploader/blob/main/CHANGELOG.md))
+and everything is uploaded by default now. To retain the previous functionality use:
+
+```ruby
+# skip uploading map files
+WebpackerUploader.config.ignored_extensions = [".map"]
 ```
 
 ## Development

data/Rakefile

@@ -1,6 +1,11 @@
+# frozen_string_literal: true
+
 require "bundler/gem_tasks"
 require "rake/testtask"
 
+require "yard"
+YARD::Rake::YardocTask.new
+
 Rake::TestTask.new(:test) do |t|
   t.libs << "test"
   t.libs << "lib"

data/lib/webpacker_uploader.rb

@@ -1,23 +1,35 @@
 # frozen_string_literal: true
 
 require "active_support/core_ext/object/blank"
-require "active_support/logger"
-require "active_support/tagged_logging"
 
 module WebpackerUploader
   extend self
 
+  # @private
   def instance=(instance)
     @instance = instance
   end
 
+  # @private
   def instance
     @instance ||= WebpackerUploader::Instance.new
   end
 
-  delegate :logger, :logger=, :upload!, to: :instance
+  # @!attribute [rw] config
+  #   @see Instance#config
+  #   @!scope class
+  # @!method configure
+  #   @see Instance#configure
+  #   @!scope class
+  # @!method upload!
+  #   @return [void]
+  #   @see Instance#upload!
+  #   @!scope class
+  delegate :configure, :config, :upload!, to: :instance
 end
 
+require "webpacker_uploader/configuration"
 require "webpacker_uploader/instance"
 require "webpacker_uploader/manifest"
+require "webpacker_uploader/mime"
 require "webpacker_uploader/version"

data/lib/webpacker_uploader/configuration.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require "active_support/logger"
+require "active_support/tagged_logging"
+
+# This is the class which holds the configuration options.
+#
+# Options are set and retrieved using `WebpackerUploader.config`
+# and `WebpackerUploader.configure`.
+class WebpackerUploader::Configuration
+  # @return [Array] the file extentions ignored by the uploader.
+  attr_accessor :ignored_extensions
+
+  # @return [ActiveSupport::Logger] the logger to use.
+  attr_accessor :logger
+
+  # @return [Boolean] whether or not to log operations.
+  attr_accessor :log_output
+
+  # @return [Pathname] the path to manifest.json, defaults to Webpacker public manifest path.
+  attr_reader :public_manifest_path
+
+  # @return [Pathname] the public root path, defaults to Webpacker public root path.
+  attr_reader :public_path
+
+  alias_method :log_output?, :log_output
+
+  def initialize
+    @ignored_extensions = []
+    @logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
+    @log_output = true
+    @public_manifest_path = ::Webpacker.config.public_manifest_path
+    @public_path = ::Webpacker.config.public_path
+  end
+
+  def public_manifest_path=(path)
+    @public_manifest_path = Pathname.new(path)
+  end
+
+  def public_path=(path)
+    @public_path = Pathname.new(path)
+  end
+end

data/lib/webpacker_uploader/instance.rb

@@ -1,36 +1,72 @@
 # frozen_string_literal: true
 
-require "mime-types"
-
 class WebpackerUploader::Instance
-  # TODO make this configurable
-  IGNORE_EXTENSION = %w[.map].freeze
-  public_constant :IGNORE_EXTENSION
-
-  cattr_accessor(:logger) { ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT)) }
+  attr_writer :config
 
+  # @private
   def manifest
     @manifest ||= WebpackerUploader::Manifest.new
   end
 
-  def upload!(provider)
-    manifest.assets.each do |name, path|
-      path = path[1..-1]
-      file_path = Rails.root.join("public", path)
+  # Returns the global WebpackerUploader::Configuration object.
+  # Use this to set and retrieve specific configuration options.
+  #
+  # @return [WebpackerUploader::Configuration]
+  #
+  # @example Disable log output
+  #   WebpackerUploader.config.log_output = false
+  #
+  # @example Get the list of excluded file extension
+  #   puts WebpackerUploader.config.ignored_extensions
+  #
+  # @see WebpackerUploader::Configuration
+  def config
+    @config ||= WebpackerUploader::Configuration.new
+  end
 
-      if name.end_with?(*IGNORE_EXTENSION)
-        logger.info("Skipping: #{file_path}")
-      else
-        logger.info("Processing: #{file_path}")
-        provider.upload!(path, file_path, content_type_for(path)) unless name.end_with?(*IGNORE_EXTENSION)
-      end
-    end
+  # Yields the global configuration to a block. Use this to
+  # configure the base gem features.
+  #
+  # @example
+  #   WebpackerUploader.configure do |config|
+  #     config.ignored_extensions = [".png", ".jpg", ".webp"]
+  #     config.log_output = false
+  #     config.public_manifest_path = "path/to/manifest.json"
+  #     config.public_path = "path/to/public/dir"
+  #   end
+  #
+  # @see WebpackerUploader::Configuration
+  def configure
+    yield config
   end
 
-  private
+  # Uploads assets using the supplied provider. Currently only AWS S3 is implemented.
+  #
+  # @return [void]
+  # @param provider [WebpackerUploader::Providers::Aws] A provider to use for file uploading.
+  # @param prefix [String] Used to prefix the remote file paths.
+  def upload!(provider, prefix: nil)
+    manifest.assets.each do |name, js_path|
+      path = js_path[1..-1]
+
+      remote_path =
+        if prefix.nil?
+          path
+        else
+          "#{prefix}/#{path}"
+        end
 
-    def content_type_for(file)
-      fallback = MIME::Types.type_for(file).first.content_type
-      Rack::Mime.mime_type(File.extname(file), fallback)
+      file_path = config.public_path.join(path)
+
+      if name.end_with?(*config.ignored_extensions)
+        config.logger.info("Skipping #{file_path}") if config.log_output?
+      else
+        content_type = WebpackerUploader::Mime.mime_type(path)
+
+        config.logger.info("Processing #{file_path} as #{content_type}") if config.log_output?
+
+        provider.upload!(remote_path, file_path, content_type)
+      end
     end
+  end
 end

data/lib/webpacker_uploader/manifest.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-class WebpackerUploader::Manifest
+class WebpackerUploader::Manifest # @private
   attr_reader :assets
 
   def initialize
@@ -10,8 +10,8 @@ class WebpackerUploader::Manifest
   private
 
     def load
-      if ::Webpacker.config.public_manifest_path.exist?
-        JSON.parse(::Webpacker.config.public_manifest_path.read).except("entrypoints")
+      if WebpackerUploader.config.public_manifest_path.exist?
+        JSON.parse(WebpackerUploader.config.public_manifest_path.read).except("entrypoints")
       else
         {}
       end

data/lib/webpacker_uploader/mime.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+require "mime-types"
+
+module WebpackerUploader::Mime
+  # Returns the mime type for the given file in the filesystem.
+  # If it's unable to detect the mime type, it returns +application/octet-stream+
+  # as a fallback.
+  #
+  # @param file_path [String] A file path in the local filesystem.
+  # @return [String] The file mime type.
+  def mime_type(file_path)
+    fallback = MIME::Types.type_for(file_path).first&.content_type || "application/octet-stream"
+    Rack::Mime.mime_type(File.extname(file_path), fallback)
+  end
+  module_function :mime_type
+end

data/lib/webpacker_uploader/providers/aws.rb

@@ -3,29 +3,76 @@
 require "aws-sdk-s3"
 
 module WebpackerUploader
+  # Namespace for the upload providers.
   module Providers
+    # AWS provider uploads files to AWS S3. It uses the +aws-sdk-s3+ gem.
     class Aws
-      attr_reader :client
-      attr_reader :resource
-      attr_reader :bucket
+      attr_reader :client, :resource, :bucket # @private
 
+      # @param [Hash] options
+      #   * :region (String) The S3 region name.
+      #   * :bucket (String) The S3 bucket name.
+      #   * :credentials (Hash) credential options for the AWS provider:
+      #     * :profile_name (String) use a named profile configured in ~/.aws/credentials
+      #     * :instance_profile (Boolean) use an instance profile from an EC2
+      #     * :access_key_id (String) the AWS credentials access id.
+      #     * :secret_access_key (String) the AWS credentials secret access key.
+      #
+      # @example Initialize the using a named profile:
+      #
+      #   provider_options = {
+      #     credentials: { profile_name: "staging" },
+      #     region: "eu-central-1",
+      #     bucket: "application-assets-20200929124523451600000001"
+      #   }
+      #   provider = WebpackerUploader::Providers::Aws.new(provider_options)
+      #
+      # @example Initialize using IAM keys
+      #
+      #   provider_options = {
+      #     credentials: { access_key_id: "KEY_ID", secret_access_key: "ACCESS_KEY" },
+      #     region: "eu-central-1",
+      #     bucket: "application-assets-20200929124523451600000001"
+      #   }
+      #   provider = WebpackerUploader::Providers::Aws.new(provider_options)
+      #
+      # @example Initialize using an EC2 instance profile
+      #
+      #   provider_options = {
+      #     credentials: { instance_profile: true },
+      #     region: "eu-central-1",
+      #     bucket: "application-assets-20200929124523451600000001"
+      #   }
+      #   provider = WebpackerUploader::Providers::Aws.new(provider_options)
       def initialize(options)
-        @client = ::Aws::S3::Client.new(credentials: credentials(options[:credentials]), region: options[:region])
+        @client = ::Aws::S3::Client.new(client_options(options))
         @resource = ::Aws::S3::Resource.new(client: @client)
         @bucket = @resource.bucket(options[:bucket])
       end
 
+      # Uploads a file to AWS S3.
+      #
+      # @param object_key [String] Is the remote path name for the S3 object.
+      # @param file [Pathname] Path of the local file.
+      # @param content_type [String] The content type that will be set to the S3 object.
+      # @return [void]
       def upload!(object_key, file, content_type = "")
         object = @bucket.object(object_key)
         object.upload_file(file, content_type: content_type)
       end
 
       private
+        def client_options(options)
+          opts = { region: options[:region] }
+          opts[:profile] = options[:profile_name] if options.key? :profile_name
+          opts[:credentials] = credentials(options) if credentials(options)
+          opts
+        end
 
         def credentials(options)
-          if options[:profile_name].present?
-            ::Aws::SharedCredentials.new(profile_name: options[:profile_name])
-          else
+          if options.key?(:instance_profile) && options[:instance_profile]
+            ::Aws::InstanceProfileCredentials.new
+          elsif options.key?(:access_key_id)
             ::Aws::Credentials.new(options[:access_key_id], options[:secret_access_key])
           end
         end