webpacker_uploader 0.1.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # WebpackerUploader
2
2
 
3
+ [![Ruby tests](https://img.shields.io/github/workflow/status/tlatsas/webpacker_uploader/Ruby%20tests?style=flat-square)](https://github.com/tlatsas/webpacker_uploader/actions)
4
+ [![RuboCop](https://img.shields.io/github/workflow/status/tlatsas/webpacker_uploader/RuboCop?label=rubocop&style=flat-square)](https://github.com/tlatsas/webpacker_uploader/actions)
5
+ [![Gem](https://img.shields.io/gem/v/webpacker_uploader?style=flat-square)](https://rubygems.org/gems/webpacker_uploader)
6
+
3
7
  Webpacker uploader provides an easy way to upload your assets to AWS S3.
4
8
  It knows which files to upload by reading the `manifest.json` file.
5
9
 
@@ -11,14 +15,14 @@ S3 + CDN, outside of your Rails application.
11
15
  Add this line to your application's Gemfile:
12
16
 
13
17
  ```ruby
14
- gem "webpacker_uploader"
18
+ gem "webpacker_uploader", require: false
15
19
  ```
16
20
 
17
- And then execute:
21
+ and run:
18
22
 
19
23
  $ bundle install
20
24
 
21
- Or install it yourself as:
25
+ or:
22
26
 
23
27
  $ gem install webpacker_uploader
24
28
 
@@ -30,18 +34,109 @@ gem "aws-sdk-s3", require: false
30
34
 
31
35
  ## Usage
32
36
 
37
+ Usually, in a Rake task you would add the following to upload the assets to AWS S3:
38
+
39
+ ```ruby
40
+ require "webpacker_uploader"
41
+ require "webpacker_uploader/providers/aws"
42
+
43
+ provider_options = {
44
+ credentials: { profile_name: "staging" },
45
+ region: "eu-central-1",
46
+ bucket: "application-assets-20200929124523451600000001"
47
+ }
48
+
49
+ provider = WebpackerUploader::Providers::Aws.new(provider_options)
50
+ WebpackerUploader.upload!(provider)
51
+ ```
52
+
53
+ Currently, the only provider implemented is the AWS S3 provider.
54
+
55
+ ### AWS S3 provider
56
+
57
+ The AWS S3 provider credentials can be configured in three ways.
58
+ Passing a named profile name:
59
+
60
+ ```ruby
61
+ provider_options = {
62
+ credentials: { profile_name: "staging" },
63
+ region: "eu-central-1",
64
+ bucket: "application-assets-20200929124523451600000001"
65
+ }
66
+
67
+ provider = WebpackerUploader::Providers::Aws.new(provider_options)
68
+ ```
69
+
70
+ passing the access and the secret keys directly:
71
+
72
+ ```ruby
73
+ provider_options = {
74
+ credentials: { access_key_id: "AKIAIOSFODNN7EXAMPLE", secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY" },
75
+ region: "eu-central-1",
76
+ bucket: "application-assets-20200929124523451600000001"
77
+ }
78
+
79
+ provider = WebpackerUploader::Providers::Aws.new(provider_options)
80
+ ```
81
+
82
+ or using an EC2 instance profile:
83
+
33
84
  ```ruby
34
- require 'webpacker_uploader'
35
- require 'webpacker_uploader/providers/aws'
85
+ provider_options = {
86
+ credentials: { instance_profile: true },
87
+ region: "eu-central-1",
88
+ bucket: "application-assets-20200929124523451600000001"
89
+ }
36
90
 
37
- provider_options = {
38
- credentials: { profile_name: "staging" },
39
- region: "eu-central-1",
40
- bucket: "application-assets-20200929124523451600000001"
41
- }
91
+ provider = WebpackerUploader::Providers::Aws.new(provider_options)
92
+ ```
93
+
94
+ ### Prefix remote files
95
+
96
+ Uploaded files can be prefixed by setting the `prefix` parameter during upload:
97
+
98
+ ```ruby
99
+ WebpackerUploader.upload!(provider, prefix: "assets")
100
+ ```
101
+
102
+ This will prefix all remote file paths with `assets` so instead of storing `packs/application-dd6b1cd38bfa093df600.css` it
103
+ will store `assets/packs/application-dd6b1cd38bfa093df600.css`.
104
+
105
+ ### Configuration
106
+
107
+ WebpackerUploader currently supports the following configuration options:
108
+
109
+ | option | description | default value |
110
+ |----------------------|--------------------------------------------------------------|---------------------------------------|
111
+ | ignored_extensions | Which files uploader should skip based on the file extension | [] |
112
+ | logger | The logger to use for logging | ActiveSupport::Logger.new(STDOUT) |
113
+ | log_output | Log output as we upload files | true |
114
+ | public_manifest_path | The webpack manifest path | Webpacker.config.public_manifest_path |
115
+ | public_path | The webpack public output path | Webpacker.config.public_path |
42
116
 
43
- provider = WebpackerUploader::Providers::Aws.new(provider_options)
44
- WebpackerUploader.upload!(provider)
117
+ It can be configured using a block:
118
+
119
+ ```ruby
120
+ WebpackerUploader.configure do |config|
121
+ config.ignored_extensions = [".png", ".jpg", ".webp"]
122
+ config.log_output = false
123
+ config.public_manifest_path = "path/to/manifest.json"
124
+ config.public_path = "path/to/public/dir"
125
+ end
126
+ ```
127
+
128
+ or directly:
129
+
130
+ ```ruby
131
+ WebpackerUploader.config.log_output = false
132
+ ```
133
+
134
+ The uploader used to skip `.map` files by default. This has changed (see [CHANGELOG.md](https://github.com/tlatsas/webpacker_uploader/blob/main/CHANGELOG.md))
135
+ and everything is uploaded by default now. To retain the previous functionality use:
136
+
137
+ ```ruby
138
+ # skip uploading map files
139
+ WebpackerUploader.config.ignored_extensions = [".map"]
45
140
  ```
46
141
 
47
142
  ## Development
data/Rakefile CHANGED
@@ -1,6 +1,11 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "bundler/gem_tasks"
2
4
  require "rake/testtask"
3
5
 
6
+ require "yard"
7
+ YARD::Rake::YardocTask.new
8
+
4
9
  Rake::TestTask.new(:test) do |t|
5
10
  t.libs << "test"
6
11
  t.libs << "lib"
@@ -1,23 +1,35 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "active_support/core_ext/object/blank"
4
- require "active_support/logger"
5
- require "active_support/tagged_logging"
6
4
 
7
5
  module WebpackerUploader
8
6
  extend self
9
7
 
8
+ # @private
10
9
  def instance=(instance)
11
10
  @instance = instance
12
11
  end
13
12
 
13
+ # @private
14
14
  def instance
15
15
  @instance ||= WebpackerUploader::Instance.new
16
16
  end
17
17
 
18
- delegate :logger, :logger=, :upload!, to: :instance
18
+ # @!attribute [rw] config
19
+ # @see Instance#config
20
+ # @!scope class
21
+ # @!method configure
22
+ # @see Instance#configure
23
+ # @!scope class
24
+ # @!method upload!
25
+ # @return [void]
26
+ # @see Instance#upload!
27
+ # @!scope class
28
+ delegate :configure, :config, :upload!, to: :instance
19
29
  end
20
30
 
31
+ require "webpacker_uploader/configuration"
21
32
  require "webpacker_uploader/instance"
22
33
  require "webpacker_uploader/manifest"
34
+ require "webpacker_uploader/mime"
23
35
  require "webpacker_uploader/version"
@@ -0,0 +1,43 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_support/logger"
4
+ require "active_support/tagged_logging"
5
+
6
+ # This is the class which holds the configuration options.
7
+ #
8
+ # Options are set and retrieved using `WebpackerUploader.config`
9
+ # and `WebpackerUploader.configure`.
10
+ class WebpackerUploader::Configuration
11
+ # @return [Array] the file extentions ignored by the uploader.
12
+ attr_accessor :ignored_extensions
13
+
14
+ # @return [ActiveSupport::Logger] the logger to use.
15
+ attr_accessor :logger
16
+
17
+ # @return [Boolean] whether or not to log operations.
18
+ attr_accessor :log_output
19
+
20
+ # @return [Pathname] the path to manifest.json, defaults to Webpacker public manifest path.
21
+ attr_reader :public_manifest_path
22
+
23
+ # @return [Pathname] the public root path, defaults to Webpacker public root path.
24
+ attr_reader :public_path
25
+
26
+ alias_method :log_output?, :log_output
27
+
28
+ def initialize
29
+ @ignored_extensions = []
30
+ @logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
31
+ @log_output = true
32
+ @public_manifest_path = ::Webpacker.config.public_manifest_path
33
+ @public_path = ::Webpacker.config.public_path
34
+ end
35
+
36
+ def public_manifest_path=(path)
37
+ @public_manifest_path = Pathname.new(path)
38
+ end
39
+
40
+ def public_path=(path)
41
+ @public_path = Pathname.new(path)
42
+ end
43
+ end
@@ -1,36 +1,72 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "mime-types"
4
-
5
3
  class WebpackerUploader::Instance
6
- # TODO make this configurable
7
- IGNORE_EXTENSION = %w[.map].freeze
8
- public_constant :IGNORE_EXTENSION
9
-
10
- cattr_accessor(:logger) { ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT)) }
4
+ attr_writer :config
11
5
 
6
+ # @private
12
7
  def manifest
13
8
  @manifest ||= WebpackerUploader::Manifest.new
14
9
  end
15
10
 
16
- def upload!(provider)
17
- manifest.assets.each do |name, path|
18
- path = path[1..-1]
19
- file_path = Rails.root.join("public", path)
11
+ # Returns the global WebpackerUploader::Configuration object.
12
+ # Use this to set and retrieve specific configuration options.
13
+ #
14
+ # @return [WebpackerUploader::Configuration]
15
+ #
16
+ # @example Disable log output
17
+ # WebpackerUploader.config.log_output = false
18
+ #
19
+ # @example Get the list of excluded file extension
20
+ # puts WebpackerUploader.config.ignored_extensions
21
+ #
22
+ # @see WebpackerUploader::Configuration
23
+ def config
24
+ @config ||= WebpackerUploader::Configuration.new
25
+ end
20
26
 
21
- if name.end_with?(*IGNORE_EXTENSION)
22
- logger.info("Skipping: #{file_path}")
23
- else
24
- logger.info("Processing: #{file_path}")
25
- provider.upload!(path, file_path, content_type_for(path)) unless name.end_with?(*IGNORE_EXTENSION)
26
- end
27
- end
27
+ # Yields the global configuration to a block. Use this to
28
+ # configure the base gem features.
29
+ #
30
+ # @example
31
+ # WebpackerUploader.configure do |config|
32
+ # config.ignored_extensions = [".png", ".jpg", ".webp"]
33
+ # config.log_output = false
34
+ # config.public_manifest_path = "path/to/manifest.json"
35
+ # config.public_path = "path/to/public/dir"
36
+ # end
37
+ #
38
+ # @see WebpackerUploader::Configuration
39
+ def configure
40
+ yield config
28
41
  end
29
42
 
30
- private
43
+ # Uploads assets using the supplied provider. Currently only AWS S3 is implemented.
44
+ #
45
+ # @return [void]
46
+ # @param provider [WebpackerUploader::Providers::Aws] A provider to use for file uploading.
47
+ # @param prefix [String] Used to prefix the remote file paths.
48
+ def upload!(provider, prefix: nil)
49
+ manifest.assets.each do |name, js_path|
50
+ path = js_path[1..-1]
51
+
52
+ remote_path =
53
+ if prefix.nil?
54
+ path
55
+ else
56
+ "#{prefix}/#{path}"
57
+ end
31
58
 
32
- def content_type_for(file)
33
- fallback = MIME::Types.type_for(file).first.content_type
34
- Rack::Mime.mime_type(File.extname(file), fallback)
59
+ file_path = config.public_path.join(path)
60
+
61
+ if name.end_with?(*config.ignored_extensions)
62
+ config.logger.info("Skipping #{file_path}") if config.log_output?
63
+ else
64
+ content_type = WebpackerUploader::Mime.mime_type(path)
65
+
66
+ config.logger.info("Processing #{file_path} as #{content_type}") if config.log_output?
67
+
68
+ provider.upload!(remote_path, file_path, content_type)
69
+ end
35
70
  end
71
+ end
36
72
  end
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- class WebpackerUploader::Manifest
3
+ class WebpackerUploader::Manifest # @private
4
4
  attr_reader :assets
5
5
 
6
6
  def initialize
@@ -10,8 +10,8 @@ class WebpackerUploader::Manifest
10
10
  private
11
11
 
12
12
  def load
13
- if ::Webpacker.config.public_manifest_path.exist?
14
- JSON.parse(::Webpacker.config.public_manifest_path.read).except("entrypoints")
13
+ if WebpackerUploader.config.public_manifest_path.exist?
14
+ JSON.parse(WebpackerUploader.config.public_manifest_path.read).except("entrypoints")
15
15
  else
16
16
  {}
17
17
  end
@@ -0,0 +1,17 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "mime-types"
4
+
5
+ module WebpackerUploader::Mime
6
+ # Returns the mime type for the given file in the filesystem.
7
+ # If it's unable to detect the mime type, it returns +application/octet-stream+
8
+ # as a fallback.
9
+ #
10
+ # @param file_path [String] A file path in the local filesystem.
11
+ # @return [String] The file mime type.
12
+ def mime_type(file_path)
13
+ fallback = MIME::Types.type_for(file_path).first&.content_type || "application/octet-stream"
14
+ Rack::Mime.mime_type(File.extname(file_path), fallback)
15
+ end
16
+ module_function :mime_type
17
+ end
@@ -3,29 +3,76 @@
3
3
  require "aws-sdk-s3"
4
4
 
5
5
  module WebpackerUploader
6
+ # Namespace for the upload providers.
6
7
  module Providers
8
+ # AWS provider uploads files to AWS S3. It uses the +aws-sdk-s3+ gem.
7
9
  class Aws
8
- attr_reader :client
9
- attr_reader :resource
10
- attr_reader :bucket
10
+ attr_reader :client, :resource, :bucket # @private
11
11
 
12
+ # @param [Hash] options
13
+ # * :region (String) The S3 region name.
14
+ # * :bucket (String) The S3 bucket name.
15
+ # * :credentials (Hash) credential options for the AWS provider:
16
+ # * :profile_name (String) use a named profile configured in ~/.aws/credentials
17
+ # * :instance_profile (Boolean) use an instance profile from an EC2
18
+ # * :access_key_id (String) the AWS credentials access id.
19
+ # * :secret_access_key (String) the AWS credentials secret access key.
20
+ #
21
+ # @example Initialize the using a named profile:
22
+ #
23
+ # provider_options = {
24
+ # credentials: { profile_name: "staging" },
25
+ # region: "eu-central-1",
26
+ # bucket: "application-assets-20200929124523451600000001"
27
+ # }
28
+ # provider = WebpackerUploader::Providers::Aws.new(provider_options)
29
+ #
30
+ # @example Initialize using IAM keys
31
+ #
32
+ # provider_options = {
33
+ # credentials: { access_key_id: "KEY_ID", secret_access_key: "ACCESS_KEY" },
34
+ # region: "eu-central-1",
35
+ # bucket: "application-assets-20200929124523451600000001"
36
+ # }
37
+ # provider = WebpackerUploader::Providers::Aws.new(provider_options)
38
+ #
39
+ # @example Initialize using an EC2 instance profile
40
+ #
41
+ # provider_options = {
42
+ # credentials: { instance_profile: true },
43
+ # region: "eu-central-1",
44
+ # bucket: "application-assets-20200929124523451600000001"
45
+ # }
46
+ # provider = WebpackerUploader::Providers::Aws.new(provider_options)
12
47
  def initialize(options)
13
- @client = ::Aws::S3::Client.new(credentials: credentials(options[:credentials]), region: options[:region])
48
+ @client = ::Aws::S3::Client.new(client_options(options))
14
49
  @resource = ::Aws::S3::Resource.new(client: @client)
15
50
  @bucket = @resource.bucket(options[:bucket])
16
51
  end
17
52
 
53
+ # Uploads a file to AWS S3.
54
+ #
55
+ # @param object_key [String] Is the remote path name for the S3 object.
56
+ # @param file [Pathname] Path of the local file.
57
+ # @param content_type [String] The content type that will be set to the S3 object.
58
+ # @return [void]
18
59
  def upload!(object_key, file, content_type = "")
19
60
  object = @bucket.object(object_key)
20
61
  object.upload_file(file, content_type: content_type)
21
62
  end
22
63
 
23
64
  private
65
+ def client_options(options)
66
+ opts = { region: options[:region] }
67
+ opts[:profile] = options[:profile_name] if options.key? :profile_name
68
+ opts[:credentials] = credentials(options) if credentials(options)
69
+ opts
70
+ end
24
71
 
25
72
  def credentials(options)
26
- if options[:profile_name].present?
27
- ::Aws::SharedCredentials.new(profile_name: options[:profile_name])
28
- else
73
+ if options.key?(:instance_profile) && options[:instance_profile]
74
+ ::Aws::InstanceProfileCredentials.new
75
+ elsif options.key?(:access_key_id)
29
76
  ::Aws::Credentials.new(options[:access_key_id], options[:secret_access_key])
30
77
  end
31
78
  end