middleman-s3_sync 3.0.7 → 3.0.8

data/README.md

@@ -41,7 +41,85 @@ activate :s3_sync do |s3_sync|
 end
 ```
 
-You can then start synchronizing files with S3 through ```middleman s3_sync```. 
+You can then start synchronizing files with S3 through ```middleman s3_sync```.
+
+## Push All Content to S3
+
+There are situations where you might need to push the files to S3. In
+such case, you can pass the ```--force``` option:
+
+    $ middleman s3_sync --force
+
+## HTTP Caching
+
+By default, ```middleman-s3_sync``` does not set caching headers. In
+general, the default settings are sufficient. However, there are
+situations where you might want to set a different HTTP caching policy.
+This may be very helpful if you are using the ```asset_hash```
+extension.
+
+### Setting a policy based on the mime-type of a file
+
+You can set a caching policy for every files that match a certain
+mime-type. For example, setting max-age to 0 and kindly asking the
+browser to revalidate the content for HTML files would take the
+following form:
+
+```ruby
+caching_policy 'text/html', max_age: 0, must_revalidate: true
+```
+
+As a result, the following ```Cache-Control``` header would be set to ```max-age:0, must-revalidate```
+
+### Setting a Default Policy
+
+You can set the default policy by passing an options hash to ```default_caching_policy``` in your ```config.rb``` file:
+
+```ruby
+default_caching_policy max_age:(60 * 60 * 24 * 365)
+```
+
+This will apply the policy to any file that do not have a mime-type
+specific policy.
+
+### Caching Policies
+
+The [Caching Tutorial](http://www.mnot.net/cache_docs/) is a great
+introduction to HTTP caching. The caching policy code in this gem is
+based on it.
+
+The following keys can be set:
+
+| Key                | Value   | Header             | Description                                                                                                                            |
+| ---                | ----    | ------             | -----------                                                                                                                            |
+| `max_age`          | seconds | `max-age`          | Specifies the maximum amount of time that a representation will be considered fresh. This value is relative to the time of the request |
+| `s_maxage`         | seconds | `s-maxage`         | Only applies to shared (proxies) caches                                                                                                |
+| `public`           | boolean | `public`           | Marks authenticated responses as cacheable.                                                                                            |
+| `private`          | boolean | `private`          | Allows caches that are specific to one user to store the response. Shared caches (proxies) may not.                                    |
+| `no_cache`         | boolean | `no-cache`         | Forces caches to submit the request to the origin server for validation before releasing a cached copy, every time.                    |
+| `no_store`         | boolean | `no-store`         | Instructs caches not to keep a copy of the representation under any conditions.                                                        |
+| `must_revalidate`  | boolean | `must-revalidate`  | Tells the caches that they must obey any freshness information you give them about a representation.                                   |
+| `proxy_revalidate` | boolean | `proxy-revalidate` | Similar as `must-revalidate`, but only for proxies.                                                                                    |
+
+### Setting `Expires` Header
+
+You can pass the `expires` key to the `caching_policy` and
+`default_caching_policy` methods if you insist on setting the expires
+header on a results. You will need to pass it a Time object indicating
+when the resourse is set to expire.
+
+> Note that the `Cache-Control` header will take precedence over the
+> `Expires` header if both are present.
+
+### A Note About Browser Caching
+
+Browser caching is well specified. It hasn't always been the case.
+Still, even modern browsers have different behaviors if it suits it's
+developers or their employers. Specs are meant to be ignored and so they
+are (I'm looking at you Chrome!). Setting the `Cache-Control` or
+`Expires` headers are not a guarrantie that the browsers and the proxies
+that stand between them and your content will behave the way you want
+them to. YMMV.
 
 ## A Debt of Gratitude
 
@@ -50,6 +128,9 @@ The code is well structured and easy to understand and it was easy to
 extend it to add my synchronization code. My gratitude goes to @karlfreeman
 and is work on Middleman sync.
 
+Many thanks to [Gnip](http://gnip.com) and [dojo4](http://dojo4.com) for
+supporting and sponsoring work on middleman-s3_sync.
+
 ## Contributing
 
 1. Fork it

data/lib/middleman-s3_sync.rb

@@ -13,52 +13,53 @@ end
 module Middleman
   module S3Sync
     class << self
-      LONG_TIME = 315360000   # 10 years
-      DONT_CACHE = ['html']   # types of files to exclude from browser cache
-
       def sync
         puts "Gathering local files."
 
         local_files = (Dir[options.build_dir + "/**/*"] + Dir[options.build_dir + "/**/.*"])
           .reject { |f| File.directory?(f) }
           .map { |f| f.gsub(/^#{options.build_dir}\//, '') }
-        puts "Gathering remote files."
+        puts "Gathering remote files from #{options.bucket}"
         remote_files = bucket.files.map { |f| f.key }
 
-        # First pass on the set of files to work with.
-        puts "Determine files to add to S3."
-        files_to_push = local_files - remote_files
-        if options.delete
-          puts "Determine which files to delete from S3"
-          files_to_delete = remote_files - local_files
-        end
-        files_to_evaluate = local_files - files_to_push
-
-        # No need to evaluate the files that are newer on S3 than the local files.
-        puts "Determine which local files are newer than their S3 counterparts"
-        files_to_reject = []
-        Parallel.each(files_to_evaluate, :in_threads => 4) do |f|
-          print '.'
-          local_mtime = File.mtime("#{options.build_dir}/#{f}")
-          remote_mtime = s3_files.get(f).last_modified
-          files_to_reject << f if remote_mtime >= local_mtime
-        end
-
-        files_to_evaluate = files_to_evaluate - files_to_reject
+        if options.force
+          files_to_push = local_files
+        else
+          # First pass on the set of files to work with.
+          puts "Determine files to add to #{options.bucket}."
+          files_to_push = local_files - remote_files
+          if options.delete
+            puts "Determine which files to delete from #{options.bucket}"
+            files_to_delete = remote_files - local_files
+          end
+          files_to_evaluate = local_files - files_to_push
 
-        # Are the files different? Use MD5 to see
-        if (files_to_evaluate.size > 0)
-          puts "\n\nDetermine which remaining files are actually different than their S3 counterpart."
+          # No need to evaluate the files that are newer on S3 than the local files.
+          puts "Determine which local files are newer than their S3 counterparts"
+          files_to_reject = []
           Parallel.each(files_to_evaluate, :in_threads => 4) do |f|
             print '.'
-            local_md5 = Digest::MD5.hexdigest(File.read("#{options.build_dir}/#{f}"))
-            remote_md5 = s3_files.get(f).etag
-            files_to_push << f if local_md5 != remote_md5
+            local_mtime = File.mtime("#{options.build_dir}/#{f}")
+            remote_mtime = s3_files.get(f).last_modified
+            files_to_reject << f if remote_mtime >= local_mtime
+          end
+
+          files_to_evaluate = files_to_evaluate - files_to_reject
+
+          # Are the files different? Use MD5 to see
+          if (files_to_evaluate.size > 0)
+            puts "\n\nDetermine which remaining files are actually different than their S3 counterpart."
+            Parallel.each(files_to_evaluate, :in_threads => 4) do |f|
+              print '.'
+              local_md5 = Digest::MD5.hexdigest(File.read("#{options.build_dir}/#{f}"))
+              remote_md5 = s3_files.get(f).etag
+              files_to_push << f if local_md5 != remote_md5
+            end
           end
         end
 
         if files_to_push.size > 0
-          puts "\n\nReady to apply updates to S3."
+          puts "\n\nReady to apply updates to #{options.bucket}."
           files_to_push.each do |f|
             if remote_files.include?(f)
               puts "Updating #{f}"
@@ -66,11 +67,9 @@ module Middleman
               file.body = File.open("#{options.build_dir}/#{f}")
               file.public = true
               file.content_type = MIME::Types.of(f).first
-              ext = File.extname(f)[1..-1]
-
-              unless DONT_CACHE.include? ext
-                file.cache_control = "public, max-age=#{LONG_TIME}"
-                file.expires = CGI.rfc1123_date(Time.now + LONG_TIME)
+              if policy = options.caching_policy_for(file.content_type)
+                file.cache_control = policy.cache_control if policy.cache_control
+                file.expires = policy.expires if policy.expires
               end
 
               file.save
@@ -85,13 +84,11 @@ module Middleman
               }
 
               # Add cache-control headers
-              ext = File.extname(f)[1..-1]
-              unless DONT_CACHE.include? ext
-                file_hash.merge!({
-                  :cache_control => "public, max-age=#{LONG_TIME}",
-                  :expires => CGI.rfc1123_date(Time.now + LONG_TIME)
-                })
+              if policy = options.caching_policy_for(file_hash[:content_type])
+                file_hash[:cache_control] = policy.cache_control if policy.cache_control
+                file_hash[:expires] = policy.expires if policy.expires
               end
+
               file = bucket.files.create(file_hash)
             end
           end

data/lib/middleman-s3_sync/commands.rb

@@ -5,9 +5,6 @@ module Middleman
   module Cli
     class S3Sync < Thor
       include Thor::Actions
-
-      check_unknown_options!
-
       namespace :s3_sync
 
       def self.exit_on_failure?
@@ -15,17 +12,20 @@ module Middleman
       end
 
       desc "s3_sync", "Builds and push the minimum set of files needed to S3"
+      option :force, type: :boolean,
+                     desc: "Push all local files to the server",
+                     aliases: :f
       def s3_sync
         shared_inst = ::Middleman::Application.server.inst
         bucket = shared_inst.options.bucket
         if (!bucket)
           raise Thor::Error.new "You need to activate this extension."
         end
-        
-        ::Middleman::S3Sync.sync
 
-      end
+        shared_inst.options.force = options[:force] if options[:force]
 
+        ::Middleman::S3Sync.sync
+      end
     end
   end
 end

data/lib/middleman-s3_sync/extension.rb

@@ -11,8 +11,57 @@ module Middleman
       :after_build,
       :delete,
       :existing_remote_file,
-      :build_dir
+      :build_dir,
+      :force
     )
+
+      def add_caching_policy(content_type, options)
+        caching_policies[content_type.to_s] = BrowserCachePolicy.new(options)
+      end
+
+      def caching_policy_for(content_type)
+        caching_policies.fetch(content_type.to_s, caching_policies[:default])
+      end
+
+      def default_caching_policy
+        caching_policies[:default]
+      end
+
+      def caching_policies
+        @caching_policies ||= Map.new
+      end
+
+      protected
+      class BrowserCachePolicy
+        attr_accessor :policies
+
+        def initialize(options)
+          @policies = Map.from_hash(options)
+        end
+
+        def cache_control
+          policy = []
+          policy << "max-age=#{policies.max_age}" if policies.has_key?(:max_age)
+          policy << "s-maxage=#{s_maxage}" if policies.has_key?(:s_maxage)
+          policy << "public" if policies.fetch(:public, false)
+          policy << "private" if policies.fetch(:private, false)
+          policy << "no-cache" if policies.fetch(:no_cache, false)
+          policy << "no-store" if policies.fetch(:no_store, false)
+          policy << "must-revalidate" if policies.fetch(:must_revalidate, false)
+          policy << "proxy-revalidate" if policies.fetch(:proxy_revalidate, false)
+          if policy.empty?
+            nil
+          else
+            policy.join(", ")
+          end
+        end
+
+        def expires
+          if expiration = policies.fetch(:expires, nil)
+            CGI.rfc1123_date(expiration)
+          end
+        end
+      end
     end
 
     class << self
@@ -29,7 +78,7 @@ module Middleman
         app.send :include, Helpers
 
         app.after_configuration do |config|
-          options.build_dir = build_dir
+          options.build_dir ||= build_dir
         end
 
         app.after_build do |builder|
@@ -43,6 +92,14 @@ module Middleman
         def options
           ::Middleman::S3Sync.options
         end
+
+        def default_caching_policy(policy = {})
+          options.add_caching_policy(:default, policy)
+        end
+
+        def caching_policy(content_type, policy = {})
+          options.add_caching_policy(content_type, policy)
+        end
       end
     end
   end

data/lib/middleman-s3_sync/version.rb

@@ -1,5 +1,5 @@
 module Middleman
   module S3Sync
-    VERSION = "3.0.7"
+    VERSION = "3.0.8"
   end
 end

data/middleman-s3_sync.gemspec

@@ -20,6 +20,9 @@ Gem::Specification.new do |gem|
   gem.add_runtime_dependency 'middleman-core', '>= 3.0.0'
   gem.add_runtime_dependency 'fog'
   gem.add_runtime_dependency 'parallel'
+  gem.add_runtime_dependency 'map'
 
   gem.add_development_dependency 'rake'
+  gem.add_development_dependency 'pry'
+  gem.add_development_dependency 'pry-nav'
 end

metadata

@@ -2,7 +2,7 @@
 name: middleman-s3_sync
 version: !ruby/object:Gem::Version
   prerelease: 
-  version: 3.0.7
+  version: 3.0.8
 platform: ruby
 authors:
 - Frederic Jean
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-03-21 00:00:00.000000000 Z
+date: 2013-03-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   version_requirements: !ruby/object:Gem::Requirement
@@ -60,6 +60,22 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
   type: :runtime
+- !ruby/object:Gem::Dependency
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: map
+  prerelease: false
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
 - !ruby/object:Gem::Dependency
   version_requirements: !ruby/object:Gem::Requirement
     none: false
@@ -76,6 +92,38 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
+- !ruby/object:Gem::Dependency
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: pry
+  prerelease: false
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+- !ruby/object:Gem::Dependency
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: pry-nav
+  prerelease: false
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
 description: Only syncs files that have been updated to S3.
 email:
 - fred@fredjean.net
@@ -107,7 +155,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       segments:
       - 0
-      hash: -892703647475452173
+      hash: -1761006957673169609
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
@@ -116,7 +164,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       segments:
       - 0
-      hash: -892703647475452173
+      hash: -1761006957673169609
       version: '0'
 requirements: []
 rubyforge_project: