webpacker_uploader 0.2.1 → 0.6.0

data/README.md

@@ -3,6 +3,7 @@
 [![Ruby tests](https://img.shields.io/github/workflow/status/tlatsas/webpacker_uploader/Ruby%20tests?style=flat-square)](https://github.com/tlatsas/webpacker_uploader/actions)
 [![RuboCop](https://img.shields.io/github/workflow/status/tlatsas/webpacker_uploader/RuboCop?label=rubocop&style=flat-square)](https://github.com/tlatsas/webpacker_uploader/actions)
 [![Gem](https://img.shields.io/gem/v/webpacker_uploader?style=flat-square)](https://rubygems.org/gems/webpacker_uploader)
+[![Yard Docs](http://img.shields.io/badge/yard-docs-blue.svg?style=flat-square)](https://www.rubydoc.info/gems/webpacker_uploader)
 
 Webpacker uploader provides an easy way to upload your assets to AWS S3.
 It knows which files to upload by reading the `manifest.json` file.
@@ -15,14 +16,14 @@ S3 + CDN, outside of your Rails application.
 Add this line to your application's Gemfile:
 
 ```ruby
-gem "webpacker_uploader"
+gem "webpacker_uploader", require: false
 ```
 
-And then execute:
+and run:
 
     $ bundle install
 
-Or install it yourself as:
+or:
 
     $ gem install webpacker_uploader
 
@@ -34,6 +35,8 @@ gem "aws-sdk-s3", require: false
 
 ## Usage
 
+Usually, in a Rake task you would add the following to upload the assets to AWS S3:
+
 ```ruby
 require "webpacker_uploader"
 require "webpacker_uploader/providers/aws"
@@ -89,18 +92,6 @@ provider_options = {
 provider = WebpackerUploader::Providers::Aws.new(provider_options)
 ```
 
-### Ignore files
-
-The uploader can be configured to skip certain files based on the file extension.
-By default `.map` files are excluded. This can be configured through the `ignored_extensions` attribute.
-In order to upload everything pass an empty array.
-
-```ruby
-# skip uploading images
-WebpackerUploader.ignored_extensions = [".png", ".jpg", ".webp"]
-WebpackerUploader.upload!(provider)
-```
-
 ### Prefix remote files
 
 Uploaded files can be prefixed by setting the `prefix` parameter during upload:
@@ -112,6 +103,43 @@ WebpackerUploader.upload!(provider, prefix: "assets")
 This will prefix all remote file paths with `assets` so instead of storing `packs/application-dd6b1cd38bfa093df600.css` it
 will store `assets/packs/application-dd6b1cd38bfa093df600.css`.
 
+### Configuration
+
+WebpackerUploader currently supports the following configuration options:
+
+| option               | description                                                  | default value                         |
+|----------------------|--------------------------------------------------------------|---------------------------------------|
+| ignored_extensions   | Which files uploader should skip based on the file extension | []                                    |
+| logger               | The logger to use for logging                                | ActiveSupport::Logger.new(STDOUT)     |
+| log_output           | Log output as we upload files                                | true                                  |
+| public_manifest_path | The webpack manifest path                                    | Webpacker.config.public_manifest_path |
+| public_path          | The webpack public output path                               | Webpacker.config.public_path          |
+
+It can be configured using a block:
+
+```ruby
+WebpackerUploader.configure do |config|
+  config.ignored_extensions = [".png", ".jpg", ".webp"]
+  config.log_output = false
+  config.public_manifest_path = "path/to/manifest.json"
+  config.public_path = "path/to/public/dir"
+end
+```
+
+or directly:
+
+```ruby
+WebpackerUploader.config.log_output = false
+```
+
+The uploader used to skip `.map` files by default. This has changed (see [CHANGELOG.md](https://github.com/tlatsas/webpacker_uploader/blob/main/CHANGELOG.md))
+and everything is uploaded by default now. To retain the previous functionality use:
+
+```ruby
+# skip uploading map files
+WebpackerUploader.config.ignored_extensions = [".map"]
+```
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies.
@@ -120,6 +148,19 @@ interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`.
 
+## Integration tests
+
+This gem also provides an integration test suite. It runs using [localstack](https://github.com/localstack/localstack/).
+To run the integration tests, install docker on your system and spin up a localstack container running the s3 service.
+
+```shell
+docker compose -f "integration/docker-compose.yml" up --detach
+```
+
+To run the tests, use `rake test:integration`.
+
+To stop the container once done run: `docker-compose -f "integration/docker-compose.yml" down` 
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/tlatsas/webpacker_uploader.

data/Rakefile

@@ -1,10 +1,23 @@
+# frozen_string_literal: true
+
 require "bundler/gem_tasks"
 require "rake/testtask"
 
+require "yard"
+YARD::Rake::YardocTask.new
+
 Rake::TestTask.new(:test) do |t|
   t.libs << "test"
   t.libs << "lib"
   t.test_files = FileList["test/**/*_test.rb"]
 end
 
+namespace :test do
+  desc "Run integration tests using localstack"
+  Rake::TestTask.new(:integration) do |t|
+    t.libs << "integration"
+    t.test_files = FileList["integration/**/*_test.rb"]
+  end
+end
+
 task default: :test

data/lib/webpacker_uploader.rb

@@ -1,23 +1,35 @@
 # frozen_string_literal: true
 
+require "active_support/core_ext/module"
 require "active_support/core_ext/object/blank"
-require "active_support/logger"
-require "active_support/tagged_logging"
 
 module WebpackerUploader
   extend self
 
+  # @private
   def instance=(instance)
     @instance = instance
   end
 
+  # @private
   def instance
     @instance ||= WebpackerUploader::Instance.new
   end
 
-  delegate :logger, :logger=, :upload!, :ignored_extensions, :ignored_extensions=, to: :instance
+  # @!attribute [rw] config
+  #   @see Instance#config
+  #   @!scope class
+  # @!method configure
+  #   @see Instance#configure
+  #   @!scope class
+  # @!method upload!
+  #   @return [void]
+  #   @see Instance#upload!
+  #   @!scope class
+  delegate :configure, :config, :upload!, to: :instance
 end
 
+require "webpacker_uploader/configuration"
 require "webpacker_uploader/instance"
 require "webpacker_uploader/manifest"
 require "webpacker_uploader/mime"

data/lib/webpacker_uploader/configuration.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require "active_support/logger"
+require "active_support/tagged_logging"
+
+# This is the class which holds the configuration options.
+#
+# Options are set and retrieved using `WebpackerUploader.config`
+# and `WebpackerUploader.configure`.
+class WebpackerUploader::Configuration
+  # @return [Array] the file extentions ignored by the uploader.
+  attr_accessor :ignored_extensions
+
+  # @return [ActiveSupport::Logger] the logger to use.
+  attr_accessor :logger
+
+  # @return [Boolean] whether or not to log operations.
+  attr_accessor :log_output
+
+  # @return [Pathname] the path to manifest.json, defaults to Webpacker public manifest path.
+  attr_reader :public_manifest_path
+
+  # @return [Pathname] the public root path, defaults to Webpacker public root path.
+  attr_reader :public_path
+
+  alias_method :log_output?, :log_output
+
+  def initialize
+    @ignored_extensions = []
+    @logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
+    @log_output = true
+    @public_manifest_path = ::Webpacker.config.public_manifest_path
+    @public_path = ::Webpacker.config.public_path
+  end
+
+  def public_manifest_path=(path)
+    @public_manifest_path = Pathname.new(path)
+  end
+
+  def public_path=(path)
+    @public_path = Pathname.new(path)
+  end
+end

data/lib/webpacker_uploader/instance.rb

@@ -1,18 +1,51 @@
 # frozen_string_literal: true
 
 class WebpackerUploader::Instance
-  cattr_accessor(:logger) { ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT)) }
-  cattr_accessor(:ignored_extensions) { %w[.map] }
+  attr_writer :config
 
+  # @private
   def manifest
     @manifest ||= WebpackerUploader::Manifest.new
   end
 
-  def upload!(provider, prefix: nil)
-    if ignored_extensions.nil?
-      raise ArgumentError, "Ignored extensions should be specified as an array"
-    end
+  # Returns the global WebpackerUploader::Configuration object.
+  # Use this to set and retrieve specific configuration options.
+  #
+  # @return [WebpackerUploader::Configuration]
+  #
+  # @example Disable log output
+  #   WebpackerUploader.config.log_output = false
+  #
+  # @example Get the list of excluded file extension
+  #   puts WebpackerUploader.config.ignored_extensions
+  #
+  # @see WebpackerUploader::Configuration
+  def config
+    @config ||= WebpackerUploader::Configuration.new
+  end
 
+  # Yields the global configuration to a block. Use this to
+  # configure the base gem features.
+  #
+  # @example
+  #   WebpackerUploader.configure do |config|
+  #     config.ignored_extensions = [".png", ".jpg", ".webp"]
+  #     config.log_output = false
+  #     config.public_manifest_path = "path/to/manifest.json"
+  #     config.public_path = "path/to/public/dir"
+  #   end
+  #
+  # @see WebpackerUploader::Configuration
+  def configure
+    yield config
+  end
+
+  # Uploads assets using the supplied provider. Currently only AWS S3 is implemented.
+  #
+  # @return [void]
+  # @param provider [WebpackerUploader::Providers::Aws] A provider to use for file uploading.
+  # @param prefix [String] Used to prefix the remote file paths.
+  def upload!(provider, prefix: nil)
     manifest.assets.each do |name, js_path|
       path = js_path[1..-1]
 
@@ -23,13 +56,14 @@ class WebpackerUploader::Instance
           "#{prefix}/#{path}"
         end
 
-      file_path = Rails.root.join("public", path)
+      file_path = config.public_path.join(path)
 
-      if name.end_with?(*ignored_extensions)
-        logger.info("Skipping #{file_path}")
+      if name.end_with?(*config.ignored_extensions)
+        config.logger.info("Skipping #{file_path}") if config.log_output?
       else
         content_type = WebpackerUploader::Mime.mime_type(path)
-        logger.info("Processing #{file_path} as #{content_type}")
+
+        config.logger.info("Processing #{file_path} as #{content_type}") if config.log_output?
 
         provider.upload!(remote_path, file_path, content_type)
       end

data/lib/webpacker_uploader/manifest.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-class WebpackerUploader::Manifest
+class WebpackerUploader::Manifest # @private
   attr_reader :assets
 
   def initialize
@@ -10,8 +10,8 @@ class WebpackerUploader::Manifest
   private
 
     def load
-      if ::Webpacker.config.public_manifest_path.exist?
-        JSON.parse(::Webpacker.config.public_manifest_path.read).except("entrypoints")
+      if WebpackerUploader.config.public_manifest_path.exist?
+        JSON.parse(WebpackerUploader.config.public_manifest_path.read).except("entrypoints")
       else
         {}
       end

data/lib/webpacker_uploader/mime.rb

@@ -3,6 +3,12 @@
 require "mime-types"
 
 module WebpackerUploader::Mime
+  # Returns the mime type for the given file in the filesystem.
+  # If it's unable to detect the mime type, it returns +application/octet-stream+
+  # as a fallback.
+  #
+  # @param file_path [String] A file path in the local filesystem.
+  # @return [String] The file mime type.
   def mime_type(file_path)
     fallback = MIME::Types.type_for(file_path).first&.content_type || "application/octet-stream"
     Rack::Mime.mime_type(File.extname(file_path), fallback)

data/lib/webpacker_uploader/providers/aws.rb

@@ -3,34 +3,105 @@
 require "aws-sdk-s3"
 
 module WebpackerUploader
+  # Namespace for the upload providers.
   module Providers
+    # AWS provider uploads files to AWS S3. It uses the +aws-sdk-s3+ gem.
     class Aws
-      attr_reader :client
-      attr_reader :resource
-      attr_reader :bucket
+      # AWS provider error class. Raised when the provided AWS client credentials options are wrong.
+      class CredentialsError < StandardError; end
 
+      # @param [Hash] options
+      #   * :region (String) The S3 region name.
+      #   * :bucket (String) The S3 bucket name.
+      #   * :credentials (Hash) credential options for the AWS provider:
+      #     * :profile_name (String) use a named profile configured in ~/.aws/credentials
+      #     * :instance_profile (Boolean) use an instance profile from an EC2
+      #     * :access_key_id (String) the AWS credentials access id.
+      #     * :secret_access_key (String) the AWS credentials secret access key.
+      #
+      # @note Any unknown options will be passed directly to the +Aws::S3::Client+ class
+      #   during initialization.
+      #
+      # @raise [CredentialsError] if the credential options Hash is not correct.
+      #
+      # @example Initialize using a named profile:
+      #
+      #   provider_options = {
+      #     credentials: { profile_name: "staging" },
+      #     region: "eu-central-1",
+      #     bucket: "application-assets-20200929124523451600000001"
+      #   }
+      #   provider = WebpackerUploader::Providers::Aws.new(provider_options)
+      #
+      # @example Initialize using IAM keys
+      #
+      #   provider_options = {
+      #     credentials: { access_key_id: "KEY_ID", secret_access_key: "ACCESS_KEY" },
+      #     region: "eu-central-1",
+      #     bucket: "application-assets-20200929124523451600000001"
+      #   }
+      #   provider = WebpackerUploader::Providers::Aws.new(provider_options)
+      #
+      # @example Initialize using an EC2 instance profile
+      #
+      #   provider_options = {
+      #     credentials: { instance_profile: true },
+      #     region: "eu-central-1",
+      #     bucket: "application-assets-20200929124523451600000001"
+      #   }
+      #   provider = WebpackerUploader::Providers::Aws.new(provider_options)
       def initialize(options)
-        @client = ::Aws::S3::Client.new(credentials: credentials(options[:credentials]), region: options[:region])
-        @resource = ::Aws::S3::Resource.new(client: @client)
-        @bucket = @resource.bucket(options[:bucket])
+        @region      = options.delete(:region)
+        @bucket_name = options.delete(:bucket)
+        @credentials = credentials(options.delete(:credentials))
+        @aws_options = options
+        @resource    = ::Aws::S3::Resource.new(client: client)
       end
 
+      # Uploads a file to AWS S3.
+      #
+      # @param object_key [String] Is the remote path name for the S3 object.
+      # @param file [Pathname] Path of the local file.
+      # @param content_type [String] The content type that will be set to the S3 object.
+      # @return [void]
       def upload!(object_key, file, content_type = "")
-        object = @bucket.object(object_key)
+        object = @resource.bucket(@bucket_name).object(object_key)
         object.upload_file(file, content_type: content_type)
       end
 
       private
-
         def credentials(options)
-          if options[:profile_name].present?
-            ::Aws::SharedCredentials.new(profile_name: options[:profile_name])
+          if options.key?(:profile_name)
+            { profile: options[:profile_name] }
           elsif options.key?(:instance_profile) && options[:instance_profile]
             ::Aws::InstanceProfileCredentials.new
-          else
+          elsif options.key?(:access_key_id) && options.key?(:secret_access_key)
             ::Aws::Credentials.new(options[:access_key_id], options[:secret_access_key])
+          else
+            raise CredentialsError, "Wrong AWS provider credentials options."
           end
         end
+
+        def profile?
+          @credentials.is_a?(Hash) && @credentials.key?(:profile)
+        end
+
+        def credentials_object?
+          !profile?
+        end
+
+        def client
+          ::Aws::S3::Client.new(client_options)
+        end
+
+        def client_options
+          opts = {}
+          opts.merge!(@aws_options)
+          opts[:region] = @region
+          opts.merge!(@credentials) if profile?
+          opts[:credentials] = @credentials if credentials_object?
+          opts
+        end
     end
   end
 end