rack-cache 0.2.0 → 0.3.0

data/CHANGES

@@ -1,3 +1,61 @@
+## 0.3.0 / December 2008
+
+  * Add support for public and private cache control directives. Responses
+    marked as explicitly public are cached even when the request includes
+    an Authorization or Cookie header. Responses marked as explicitly private
+    are considered uncacheable.
+
+  * Added a "private_headers" option that dictates which request headers
+    trigger default "private" cache control processing. By default, the
+    Cookie and Authorization headers are included. Headers may be added or
+    removed as necessary to change the default private logic.
+
+  * Adhere to must-revalidate/proxy-revalidate cache control directives by
+    not assigning the default_ttl to responses that don't include freshness
+    information. This should let us begin using default_ttl more liberally
+    since we can control it using the must-revalidate/proxy-revalidate directives.
+
+  * Use the s-maxage Cache-Control value in preference to max-age when
+    present. The ttl= method now sets the s-maxage value instead of max-age.
+    Code that used ttl= to control freshness at the client needs to change
+    to set the max-age directive explicitly.
+
+  * Enable support for X-Sendfile middleware by responding to #to_path on
+    bodies served from disk storage. Adding the Rack::Sendfile component
+    upstream from Rack::Cache will result in cached bodies being served
+    directly by the web server (instead of being read in Ruby).
+
+  * BUG: MetaStore hits but EntityStore misses. This would 500 previously; now
+    we detect it and act as if the MetaStore missed as well.
+
+  * Implement low level #purge method on all concrete entity store
+    classes -- removes the entity body corresponding to the SHA1 key
+    provided and returns nil.
+
+  * Basically sane handling of HEAD requests. A HEAD request is never passed
+    through to the backend except when transitioning with pass!. This means
+    that the cache responds to HEAD requests without invoking the backend at
+    all when the cached entry is fresh. When no cache entry exists, or the
+    cached entry is stale and can be validated, the backend is invoked with
+    a GET request and the HEAD is handled right before the response
+    is delivered upstream.
+
+  * BUG: The Age response header was not being set properly when a stale
+    entry was validated. This would result in Age values that exceeded
+    the freshness lifetime in responses.
+
+  * BUG: A cached entry in a heap meta store could be unintentionally
+    modified by request processing since the cached objects were being
+    returned directly. The result was typically missing/incorrect header
+    values (e.g., missing Content-Type header). [dkubb]
+
+  * BUG: 304 responses should not include entity headers (especially
+    Content-Length). This is causing Safari/WebKit weirdness on 304
+    responses.
+
+  * BUG: The If-None-Match header was being ignored, causing the cache
+    to send 200 responses to matching conditional GET requests.
+
 ## 0.2.0 / 2008-10-24 / Initial Release
 
   * Document events and transitions in `rack/cache/config/default.rb`

data/README

@@ -3,16 +3,14 @@ Rack::Cache
 
 Rack::Cache is suitable as a quick drop-in component to enable HTTP caching for
 Rack-based applications that produce freshness (Expires, Cache-Control) and/or
-validation (Last-Modified, ETag) information.
-
-We strive to implement those portions of RFC 2616 Section 13 relevant to gateway
-(i.e., "reverse proxy") cache scenarios with a system for specifying cache
-policy:
+validation (Last-Modified, ETag) information:
 
   * Standards-based (RFC 2616)
   * Freshness/expiration based caching
-  * Validation
+  * Validation (If-Modified-Since / If-None-Match)
   * Vary support
+  * Cache-Control: public, private, max-age, s-maxage, must-revalidate,
+    and proxy-revalidate.
   * Portable: 100% Ruby / works with any Rack-enabled framework
   * Configuration language for advanced caching policies
   * Disk, memcached, and heap memory storage backends
@@ -68,10 +66,17 @@ Assuming you've designed your backend application to take advantage of HTTP's
 caching features, no further code or configuration is required for basic
 caching.
 
-Documentation
--------------
+Links
+-----
 
-http://tomayko.com/src/rack-cache/
+Documentation:
+    http://tomayko.com/src/rack-cache/
+
+Mailing List:
+    http://groups.google.com/group/rack-cache
+
+GitHub:
+    http://github.com/rtomayko/rack-cache/
 
 License
 -------

data/Rakefile

@@ -21,7 +21,7 @@ end
 desc 'Run specs with unit test style output'
 task :test => FileList['test/*_test.rb'] do |t|
   suite = t.prerequisites
-  sh "testrb -Ilib:test #{suite.join(' ')}", :verbose => false
+  sh "specrb -Ilib:test #{suite.join(' ')}", :verbose => false
 end
 
 desc 'Generate test coverage report'
@@ -72,12 +72,12 @@ desc 'Build markdown documentation files'
 task 'doc:markdown'
 FileList['doc/*.markdown'].each do |source|
   dest = "doc/#{File.basename(source, '.markdown')}.html"
-  file dest => source do |f|
+  file dest => [source, 'doc/layout.html.erb'] do |f|
     puts "markdown: #{source} -> #{dest}" if verbose
-    require 'erb'
-    require 'rdiscount'
+    require 'erb' unless defined? ERB
+    require 'rdiscount' unless defined? RDiscount
     template = File.read(source)
-    content = Markdown.new(ERB.new(template, 0, "%<>").result(binding)).to_html
+    content = Markdown.new(ERB.new(template, 0, "%<>").result(binding), :smart).to_html
     title = content.match("<h1>(.*)</h1>")[1] rescue ''
     layout = ERB.new(File.read("doc/layout.html.erb"), 0, "%<>")
     output = layout.result(binding)
@@ -87,10 +87,16 @@ FileList['doc/*.markdown'].each do |source|
   CLEAN.include dest
 end
 
+desc 'Publish documentation'
 task 'doc:publish' => :doc do
   sh 'rsync -avz doc/ gus@tomayko.com:/src/rack-cache'
 end
 
+desc 'Start the documentation development server (requires thin)'
+task 'doc:server' do
+  sh 'cd doc && thin --rackup server.ru --port 3035 start'
+end
+
 # PACKAGING =================================================================
 
 def package(ext='')

data/TODO

@@ -1,40 +1,32 @@
-## 0.3
+## 0.4
 
-  - BUG: meta store hits but entity misses
-  - BUG: HEAD request on invalid entry caches zero-length response
-  - BUG: Response body written to cache each time validation succeeds
-  - Are we doing HEAD properly?
   - liberal, conservative, sane caching configs
-  - Sample app
-  - busters.rb doc and tests
-  - no-cache.rb doc and tests
+  - Sample apps: Rack, Rails, Sinatra, Merb, etc.
+  - busters.rb and no-cache.rb doc and tests
   - Canonicalized URL for cache key:
     - sorts params by key, then value
     - urlencodes /[^ A-Za-z0-9_.-]/ host, path, and param key/value
-  - Support server-specific X-Sendfile (or similar) for delivering cached
-    bodies (file entity stores only).
-  - Sqlite3 (meta store)
+  - Custom cache keys
   - Cache invalidation on PUT, POST, DELETE.
     - Invalidate at the request URI; or, anything that's "near" the request URI.
     - Invalidate at the URI of the Location or Content-Location response header.
 
 ## Backlog
 
+  - Add missing Expires header if we have a max-age.
   - Purge/invalidate specific cache entries
   - Purge/invalidate everything
   - Maximum size of cached entity
   - Last-Modified factor: requests that have a Last-Modified header but no Expires
     header have a TTL assigned based on the last modified age of the response:
     TTL = (Age * Factor), or, 1h  = (10h * 0.1)
-  - I wonder if it would be possible to run in threaded mode but with an
-    option to lock before making requests to the backend. The idea is to be
-    able to serve requests from cache in separate threads. This should
+  - Run under multiple-threads with an option to lock before making requests
+    to the backend. The idea is to be able to serve requests from cache in
+    separate threads. This should probably be implemented as a separate
+    middleware component.
+  - Consider implementing ESI (http://www.w3.org/TR/esi-lang). This should
     probably be implemented as a separate middleware component.
+  - Sqlite3 (meta store)
   - stale-while-revalidate
   - Serve cached copies when down (see: stale-if-error) - e.g., database
     connection drops and the cache takes over what it can. 
-  - When a cache misses due to Vary, try to validate using the best match. Note
-    that you can't do this with a weak validator, so only strong etags can be
-    used.
-  - Consider implementing ESI (http://www.w3.org/TR/esi-lang). This should
-    probably be implemented as a separate middleware component.

data/doc/configuration.markdown

@@ -115,6 +115,14 @@ If no entitystore is specified, the `heap:/` store is assumed. This
 implementation has significant draw-backs so explicit configuration is
 recommended.
 
+### `private_headers`
+
+An array of request header names that cause the response to be treated with
+private cache control semantics. The default value is `['Authorization', 'Cookie']`.
+If any of these headers are present in the request, the response is considered
+private and will not be cached _unless_ the response is explicitly marked public
+(e.g., `Cache-Control: public`).
+
 <a id='machinery'></a>
 
 Configuration Machinery - Events and Transitions
@@ -184,7 +192,7 @@ cache policy into files for organization and reuse.
       # more stuff here
     end
 
-The `breakers` and `mycacheconfig` configuration files are normal Ruby source
+The `busters` and `mycacheconfig` configuration files are normal Ruby source
 files (i.e., they have a `.rb` extension) situated on the `$LOAD_PATH` - the
 `import` statement works like Ruby's `require` statement but the contents of the
 files are evaluated in the context of the configuration machinery, as if

data/doc/index.markdown

@@ -2,10 +2,10 @@ __Rack::Cache__ is suitable as a quick drop-in component to enable HTTP caching
 for [Rack][]-based applications that produce freshness (`Expires`,
 `Cache-Control`) and/or validation (`Last-Modified`, `ETag`) information.
 
-  * Standards-based ([RFC 2616][rfc] / [Section 13][s13]).
+  * Standards-based (see [RFC 2616][rfc] / [Section 13][s13]).
   * Freshness/expiration based caching
   * Validation
-  * Vary Support
+  * Vary support
   * Portable: 100% Ruby / works with any [Rack][]-enabled framework.
   * [Configuration language][config] for advanced caching policies.
   * Disk, memcached, and heap memory [storage backends][storage].
@@ -47,29 +47,35 @@ simply `require` and `use` as follows:
 
 Assuming you've designed your backend application to take advantage of HTTP's
 caching features, no further code or configuration is required for basic
-caching. More sophisticated stuff is possible with [Rack::Cache's Configuration
-Language][config].
+caching.
 
-Advanced Usage
---------------
+More
+----
 
-  * [Configuration Language Documentation][config] - How to customize cache
+  * [Configuration Language Documentation][config] - how to customize cache
     policy using the simple event-based configuration system.
 
-  * [Cache Storage Documentation][storage] - Detailed information on the various
+  * [Cache Storage Documentation][storage] - detailed information on the various
     storage implementations available in __Rack::Cache__ and how to choose the one
     that's best for your application.
 
-  * [FAQ](./faq) - Frequently Asked Questions about __Rack::Cache__.
+  * [Things Caches Do][things] - an illustrated guide to how HTTP gateway
+    caches work with pointers to other useful resources on HTTP caching.
 
-  * [GitHub Repository](http://github.com/rtomayko/rack-cache/) - Get your
+  * [GitHub Repository](http://github.com/rtomayko/rack-cache/) - get your
     fork on.
 
+  * [Mailing List](http://groups.google.com/group/rack-cache) - for hackers
+    and users (`rack-cache@groups.google.com`).
+
+  * [FAQ](./faq) - Frequently Asked Questions about __Rack::Cache__.
+
   * [RDoc API Documentation](./api/) - Mostly worthless if you just want to use
     __Rack::Cache__ in your application but mildly insightful if you'd like to
     get a feel for how the system has been put together; I recommend
     [reading the source](http://github.com/rtomayko/rack-cache/master/lib/rack/cache).
 
+
 See Also
 --------
 
@@ -99,6 +105,7 @@ and is provided under [the MIT license](./license)
 
 [config]:  ./configuration "Rack::Cache Configuration Language Documentation"
 [storage]: ./storage       "Rack::Cache Storage Documentation"
+[things]:  http://tomayko.com/writings/things-caches-do
 
 [rfc]: http://tools.ietf.org/html/rfc2616
   "RFC 2616 - Hypertext Transfer Protocol -- HTTP/1.1 [ietf.org]"

data/doc/layout.html.erb

@@ -14,6 +14,7 @@
         <p>
           <a href="./configuration" title='Configuration Language Documentation'>Config</a> |
           <a href="./storage" title='Cache Storage Documentation'>Storage</a> |
+          <a href="http://tomayko.com/writings/things-caches-do" title="Things Caches Do">Things</a> |
           <a href="./faq" title='Frequently Asked Questions'>FAQ</a> |
           <a href="./api/" title='Fucking Sucks.'>RDoc</a>
         </p>

data/doc/server.ru

@@ -0,0 +1,34 @@
+# Rackup config that serves the contents of Rack::Cache's
+# doc directory. The documentation is rebuilt on each request.
+
+# Rewrites URLs like conventional web server configs.
+class Rewriter < Struct.new(:app)
+  def call(env)
+    if env['PATH_INFO'] =~ /\/$/
+      env['PATH_INFO'] += 'index.html'
+    elsif env['PATH_INFO'] !~ /\.\w+$/
+      env['PATH_INFO'] += '.html'
+    end
+    app.call(env)
+  end
+end
+
+# Rebuilds documentation on each request.
+class DocBuilder < Struct.new(:app)
+  def call(env)
+    if env['PATH_INFO'] !~ /\.(css|js|gif|jpg|png|ico)$/
+      env['rack.errors'] << "*** rebuilding documentation (rake -s doc)\n"
+      system "rake -s doc"
+    end
+    app.call(env)
+  end
+end
+
+use Rack::CommonLogger
+use DocBuilder
+use Rewriter
+use Rack::Static, :root => File.dirname(__FILE__), :urls => ["/"]
+
+run(lambda{|env| [404,{},'<h1>Not Found</h1>']})
+
+# vim: ft=ruby

data/lib/rack/cache/config/default.rb

@@ -17,7 +17,7 @@
 #
 on :receive do
   pass! unless request.method? 'GET', 'HEAD'
-  pass! if request.header? 'Cookie', 'Authorization', 'Expect'
+  pass! if request.header? 'Expect'
   lookup!
 end
 
@@ -109,7 +109,6 @@ end
 #   * error! - return the error code specified and abandon request.
 #
 on :store do
-  entry.ttl = default_ttl if entry.ttl.nil?
   trace 'store backend response in cache (ttl: %ds)', entry.ttl
   persist!
 end

data/lib/rack/cache/core.rb

@@ -74,9 +74,9 @@ module Rack::Cache
       @triggered.include?(event)
     end
 
-  private
     # Event handlers.
     attr_reader :events
+    private :events
 
   public
     # Attach custom logic to one or more events.
@@ -101,10 +101,18 @@ module Rack::Cache
     end
 
   private
-    # Determine if the response's Last-Modified date matches the
-    # If-Modified-Since value provided in the original request.
+    # Does the request include authorization or other sensitive information
+    # that should cause the response to be considered private by default?
+    # Private responses are not stored in the cache.
+    def private_request?
+      request.header?(*private_headers)
+    end
+
+    # Determine if the #response validators (ETag, Last-Modified) matches
+    # a conditional value specified in #original_request.
     def not_modified?
-      response.last_modified_at?(original_request.if_modified_since)
+      response.etag_matches?(original_request.if_none_match) ||
+        response.last_modified_at?(original_request.if_modified_since)
     end
 
     # Delegate the request to the backend and create the response.
@@ -118,6 +126,7 @@ module Rack::Cache
   private
     def perform_receive
       @original_request = Request.new(@env.dup.freeze)
+      @env['REQUEST_METHOD'] = 'GET' if @original_request.head?
       @request = Request.new(@env)
       info "%s %s", @original_request.request_method, @original_request.fullpath
       transition(from=:receive, to=[:pass, :lookup, :error])
@@ -125,6 +134,7 @@ module Rack::Cache
 
     def perform_pass
       trace 'passing'
+      request.env['REQUEST_METHOD'] = @original_request.request_method
       fetch_from_backend
       transition(from=:pass, to=[:pass, :finish, :error]) do |event|
         if event == :pass
@@ -171,6 +181,7 @@ module Rack::Cache
         trace "cache entry valid"
         @response = entry.dup
         @response.headers.delete('Age')
+        @response.headers.delete('Date')
         @response.headers['X-Origin-Status'] = '304'
         %w[Date Expires Cache-Control Etag Last-Modified].each do |name|
           next unless value = original_response.headers[name]
@@ -189,6 +200,20 @@ module Rack::Cache
       request.env.delete('HTTP_IF_MODIFIED_SINCE')
       request.env.delete('HTTP_IF_NONE_MATCH')
       fetch_from_backend
+
+      # mark the response as explicitly private if any of the private
+      # request headers are present and the response was not explicitly
+      # declared public.
+      if private_request? && !@response.public?
+        @response.private = true
+      else
+        # assign a default TTL for the cache entry if none was specified in
+        # the response; the must-revalidate cache control directive disables
+        # default ttl assigment.
+        if default_ttl > 0 && @response.ttl.nil? && !@response.must_revalidate?
+          @response.ttl = default_ttl
+        end
+      end
       transition(from=:fetch, to=[:store, :deliver, :error])
     end
 
@@ -196,7 +221,11 @@ module Rack::Cache
       @entry = @response
       transition(from=:store, to=[:persist, :deliver, :error]) do |event|
         if event == :persist
-          trace "writing response to cache"
+          if @response.private?
+            warn 'forced to store response marked as private.'
+          else
+            trace "storing response in cache"
+          end
           metastore.store(original_request, @entry, entitystore)
           @response = @entry
           :deliver
@@ -208,14 +237,13 @@ module Rack::Cache
 
     def perform_deliver
       trace "delivering response ..."
-      if not_modified?
-        response.status = 304
-        response.body = []
-      end
+      response.not_modified! if not_modified?
+      response.body = [] if @original_request.head?
       transition(from=:deliver, to=[:finish, :error])
     end
 
     def perform_finish
+      response.headers.delete 'X-Status'
       response.to_a
     end