racknga 0.9.0 → 0.9.1

data/NEWS.ja.rdoc

@@ -1,5 +1,16 @@
 = お知らせ
 
+== 0.9.1: 2010-11-11
+
+* キャッシュの正当性チェックを強化。
+* User-Agent毎のキャッシュをサポート。
+* JSONPをサポートするミドルウェアを追加。
+* HTTPのRangeをサポートするミドルウェアを追加。
+* groongaデータベースにアクセスログを記録するミドルウェアを追加。
+* Passengerの状態を視覚化するMuninプラグインを追加。
+* ライセンスをLGPLv2.1からLGPLv2.1 or laterに変更。
+* will_paginate対応。
+
 == 0.9.0: 2010-07-04
 
 * 最初のリリース！

data/NEWS.rdoc

@@ -1,5 +1,9 @@
 = NEWS
 
+== 0.9.1: 2010-11-11
+
+* Supported will_paginate.
+
 == 0.9.0: 2010-07-04
 
 * Initial release!

data/README.ja.rdoc

@@ -6,7 +6,8 @@ racknga（らくんが）
 
 == 説明
 
-rroongaの機能を利用したRackのミドルウェアを提供します。
+rroongaの機能を利用したRackのミドルウェアとrroongaの機能は利
+用してませんが役に立つミドルウェアを提供します。
 
 * rroonga: http://groonga.rubyforge.org/
 * Rack: http://rack.rubyforge.org/
@@ -17,7 +18,11 @@ Kouhei Sutou:: <tt><kou@clear-code.com></tt>
 
 == ライセンス
 
-LGPL 2.1です。詳しくはlicense/LGPLを見てください。
+LGPL 2.1またはそれ以降のバージョンです。詳しくは
+license/lgpl-2.1.txtを見てください。
+
+（コントリビュートされたパッチなども含み、Kouhei Sutouが
+ライセンスを変更する権利を持ちます。）
 
 == 依存ソフトウェア
 
@@ -25,13 +30,14 @@ LGPL 2.1です。詳しくはlicense/LGPLを見てください。
 * rroonga
 * Rack
 
-== インストール
+=== あるとよいソフトウェア
 
-  % sudo gem install racknga
+* jpmobile
+* will_paginate
 
-== ドキュメント
+== インストール
 
-http://groonga.rubyforge.org/racknga/
+  % sudo gem install racknga
 
 == メーリングリスト
 
@@ -39,6 +45,36 @@ http://groonga.rubyforge.org/racknga/
 
 http://lists.sourceforge.jp/mailman/listinfo/groonga-dev
 
+== ドキュメント
+
+RackngaにはRackアプリケーションに有用なミドルウェアをたくさん
+提供しています。ここでは、オプションなど詳細な情報は
+http://groonga.rubyforge.org/racknga/ を参照してください。
+
+=== Racknga::Middleware::Cache
+
+...
+
+=== Racknga::Middleware::Deflater
+
+...
+
+=== Racknga::Middleware::ExceptionNotifier
+
+...
+
+=== Racknga::Middleware::JSONP
+
+...
+
+=== Racknga::Middleware::Range
+
+...
+
+=== Racknga::Middleware::Log
+
+...
+
 == 感謝
 
 * ...

data/README.rdoc

@@ -17,7 +17,10 @@ Kouhei Sutou:: <tt><kou@clear-code.com></tt>
 
 == License
 
-LGPL 2.1. See license/LGPL for details.
+LGPL 2.1 or later. See license/lgpl-2.1.txt for details.
+
+(Kouhei Sutou has a right to change the license
+inclidng contributed patches.)
 
 == Dependencies
 
@@ -25,6 +28,10 @@ LGPL 2.1. See license/LGPL for details.
 * rroonga
 * Rack
 
+== Suggested libraries
+
+* will_paginate
+
 == Install
 
   % sudo gem install racknga

data/Rakefile

@@ -105,7 +105,7 @@ Hoe.spec('racknga') do
                          ["rack"]]
   project.spec_extras = {
     :extra_rdoc_files => Dir.glob("**/*.rdoc"),
-    :licenses => ["LGPL 2.1"]
+    :licenses => ["LGPL 2.1 or later"]
   }
   project.readme_file = "README.ja.rdoc"
 

data/html/footer.html.erb

@@ -6,6 +6,11 @@
           <img src="/rubyforge.png" width="120" height="24" border="0" alt="Ruby/groongaプロジェクトはRubyForge.orgにホスティングしてもらっています。" />
         </a>
       </p>
+      <p id="sponsor-github">
+        <a href="http://github.com/ranguba/">
+          ラングバプロジェクトはGitHubにホスティングしてもらっています。
+        </a>
+      </p>
       <p id="sponsor-tango">
 	<a href="http://tango.freedesktop.org/">
           <img width="120" height="53" border="0" alt="Tango Desktop Projectのアイコンを利用しています。" src="/tango-logo.png" />

data/lib/racknga/cache_database.rb

@@ -4,7 +4,8 @@
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
-# License version 2.1 as published by the Free Software Foundation.
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
 #
 # This library is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
@@ -15,6 +16,10 @@
 # License along with this library; if not, write to the Free Software
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
+require 'fileutils'
+
+require 'groonga'
+
 module Racknga
   class CacheDatabase
     def initialize(database_path)
@@ -27,10 +32,22 @@ module Racknga
       @context["Responses"]
     end
 
-    def purge_old_responses(threshold_time_stamp=nil)
-      threshold_time_stamp ||= Time.now
+    def configurations
+      @context["Configurations"]
+    end
+
+    def configuration
+      configurations["default"]
+    end
+
+    def purge_old_responses
+      age_modulo = 2 ** 32
+      age = configuration.age
+      previous_age = (age - 1).modulo(age_modulo)
+      configuration.age = (age + 1).modulo(age_modulo)
+
       responses.each do |response|
-        response.delete if response.created_at < threshold_time_stamp
+        response.delete if response.age == previous_age
       end
     end
 
@@ -41,6 +58,7 @@ module Racknga
         create_database
       end
       ensure_tables
+      ensure_default_configuration
     end
 
     def close_database
@@ -55,12 +73,24 @@ module Racknga
                             :key_type => "ShortText") do |table|
           table.uint32("status")
           table.short_text("headers")
-          table.text("body")
+          table.text("body", :compress => :zlib)
+          table.short_text("checksum")
+          table.uint32("age")
           table.time("created_at")
         end
       end
     end
 
+    def create_configurations_table
+      Groonga::Schema.define(:context => @context) do |schema|
+        schema.create_table("Configurations",
+                            :type => :hash,
+                            :key_type => "ShortText") do |table|
+          table.uint32("age")
+        end
+      end
+    end
+
     def create_database
       FileUtils.mkdir_p(File.dirname(@database_path))
       @database = Groonga::Database.create(:path => @database_path,
@@ -68,8 +98,12 @@ module Racknga
     end
 
     def ensure_tables
-      return if responses
+      create_configurations_table
       create_responses_table
     end
+
+    def ensure_default_configuration
+      configurations.add("default")
+    end
   end
 end

data/lib/racknga/exception_mail_notifier.rb

@@ -4,7 +4,8 @@
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
-# License version 2.1 as published by the Free Software Foundation.
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
 #
 # This library is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
@@ -19,6 +20,7 @@ require 'time'
 require 'net/smtp'
 require 'etc'
 require 'socket'
+require 'nkf'
 
 require 'racknga/utils'
 

data/lib/racknga/log_database.rb

@@ -0,0 +1,108 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2010  Kouhei Sutou <kou@clear-code.com>
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+require 'fileutils'
+
+require 'groonga'
+
+module Racknga
+  class LogDatabase
+    def initialize(database_path)
+      @database_path = database_path
+      @context = Groonga::Context.new(:encoding => :none)
+      ensure_database
+    end
+
+    def entries
+      @entries ||= @context["Entries"]
+    end
+
+    def ensure_database
+      if File.exist?(@database_path)
+        @database = Groonga::Database.open(@database_path, :context => @context)
+      else
+        create_database
+      end
+      ensure_tables
+    end
+
+    def close_database
+      @database.close
+    end
+
+    def purge_old_entries(base_time=nil)
+      base_time ||= Time.now - 60 * 60 * 24
+      entries.select do |record|
+        record.time_stamp < base_time
+      end.each do |record|
+        record.key.delete
+      end
+    end
+
+    private
+    def create_tables
+      Groonga::Schema.define(:context => @context) do |schema|
+        schema.create_table("Tags",
+                            :type => :patricia_trie,
+                            :key_type => "ShortText") do |table|
+        end
+
+        schema.create_table("Paths",
+                            :type => :patricia_trie,
+                            :key_type => "ShortText") do |table|
+        end
+
+        schema.create_table("UserAgents",
+                            :type => :hash,
+                            :key_type => "ShortText") do |table|
+        end
+
+        schema.create_table("Entries") do |table|
+          table.time("time_stamp")
+          table.reference("tag", "Tags")
+          table.reference("path", "Paths")
+          table.reference("user_agent", "UserAgents")
+          table.float("runtime")
+          table.short_text("message", :compress => :zlib)
+        end
+
+        schema.change_table("Tags") do |table|
+          table.index("Entries.tag")
+        end
+
+        schema.change_table("Paths") do |table|
+          table.index("Entries.path")
+        end
+
+        schema.change_table("UserAgents") do |table|
+          table.index("Entries.user_agent")
+        end
+      end
+    end
+
+    def create_database
+      FileUtils.mkdir_p(File.dirname(@database_path))
+      @database = Groonga::Database.create(:path => @database_path,
+                                           :context => @context)
+    end
+
+    def ensure_tables
+      create_tables
+    end
+  end
+end

data/lib/racknga/middleware/cache.rb

@@ -4,7 +4,8 @@
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
-# License version 2.1 as published by the Free Software Foundation.
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
 #
 # This library is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
@@ -15,12 +16,35 @@
 # License along with this library; if not, write to the Free Software
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
+require 'digest/md5'
 require 'yaml'
 require 'racknga/cache_database'
 
 module Racknga
   module Middleware
+    class PerUserAgentCache
+      def initialize(application)
+        @application = application
+      end
+
+      def call(environment)
+        mobile = environment["rack.jpmobile"]
+        if mobile
+          last_component = mobile.class.name.split(/::/).last
+          user_agent_key = "mobile:#{last_component.downcase}"
+        else
+          user_agent_key = "pc"
+        end
+        key = environment[Cache::KEY_KEY]
+        environment[Cache::KEY_KEY] = [key, user_agent_key].join(":")
+        @application.call(environment)
+      end
+    end
+
     class Cache
+      KEY_KEY = "racknga.cache.key"
+      START_TIME_KEY = "racknga.cache.start_time"
+
       def initialize(application, options={})
         @application = application
         @options = Utils.normalize_options(options || {})
@@ -32,13 +56,15 @@ module Racknga
       def call(environment)
         request = Rack::Request.new(environment)
         return @application.call(environment) unless use_cache?(request)
-        key = "#{key_prefix(request)}:#{normalize_path(request.fullpath)}"
+        age = @database.configuration.age
+        key = environment[KEY_KEY] || request.fullpath
+        environment[START_TIME_KEY] = Time.now
         cache = @database.responses
         record = cache[key]
-        if record
-          handle_request_with_cache(cache, key, record, request)
+        if record and record.age == age
+          handle_request_with_cache(cache, key, age, record, request)
         else
-          handle_request(cache, key, request)
+          handle_request(cache, key, age, request)
         end
       end
 
@@ -55,22 +81,8 @@ module Racknga
         requeust.get? or requeust.head?
       end
 
-      def key_prefix(request)
-        if request.respond_to?(:mobile?) and request.mobile?
-          last_component = request.mobile.class.name.split(/::/).last
-          "mobile:#{last_component.downcase}"
-        else
-          "pc"
-        end
-      end
-
-      def normalize_path(path)
-        path.gsub(/&callback=jsonp\d+&_=\d+\z/, '')
-      end
-
       def skip_caching_response?(status, headers, body)
         return true if status != 200
-        return true if status != 200
 
         headers = Rack::Utils::HeaderHash.new(headers)
         content_type = headers["Content-Type"]
@@ -85,9 +97,10 @@ module Racknga
         true
       end
 
-      def handle_request(cache, key, request)
+      def handle_request(cache, key, age, request)
         status, headers, body = @application.call(request.env)
         if skip_caching_response?(status, headers, body)
+          log("skip", request)
           return [status, headers, body]
         end
 
@@ -99,23 +112,59 @@ module Racknga
           stringified_body << data
         end
         headers = headers.to_hash
+        encoded_headers = headers.to_yaml
+        encoded_body = stringified_body.force_encoding("ASCII-8BIT")
         cache[key] = {
           :status => status,
-          :headers => headers.to_yaml,
-          :body => stringified_body.force_encoding("ASCII-8BIT"),
+          :headers => encoded_headers,
+          :body => encoded_body,
+          :checksum => compute_checksum(status, encoded_headers, encoded_body),
+          :age => age,
           :created_at => now,
         }
         body = [stringified_body]
+        log("store", request)
         [status, headers, body]
       end
 
-      def handle_request_with_cache(cache, key, record, request)
-        body = record["body"]
-        return handle_request(cache, key, request) if body.nil?
+      def handle_request_with_cache(cache, key, age, record, request)
+        status = record.status
+        headers = record.headers
+        body = record.body
+        checksum = record.checksum
+        unless valid_cache?(status, headers, body, checksum)
+          log("invalid", request)
+          return handle_request(cache, key, age, request)
+        end
+
+        log("hit", request)
+        [status, YAML.load(headers), [body]]
+      end
+
+      def compute_checksum(status, encoded_headers, encoded_body)
+        md5 = Digest::MD5.new
+        md5 << status.to_s
+        md5 << ":"
+        md5 << encoded_headers
+        md5 << ":"
+        md5 << encoded_body
+        md5.hexdigest.force_encoding("ASCII-8BIT")
+      end
+
+      def valid_cache?(status, encoded_headers, encoded_body, checksum)
+        return false if status.nil? or encoded_headers.nil? or encoded_body.nil?
+        return false if checksum.nil?
+        compute_checksum(status, encoded_headers, encoded_body) == checksum
+      end
 
-        status = record["status"]
-        headers = YAML.load(record["headers"])
-        [status, headers, [body]]
+      def log(tag, request)
+        return unless Middleware.const_defined?(:Log)
+        env = request.env
+        logger = env[Middleware::Log::LOGGER_KEY]
+        return if logger.nil?
+        start_time = env[START_TIME_KEY]
+        runtime = Time.now - start_time
+        logger.log("cache-#{tag}", request.fullpath, :runtime => runtime)
       end
     end
   end

data/lib/racknga/middleware/deflater.rb

@@ -4,7 +4,8 @@
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
-# License version 2.1 as published by the Free Software Foundation.
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
 #
 # This library is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of

data/lib/racknga/middleware/exception_notifier.rb

@@ -4,7 +4,8 @@
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
-# License version 2.1 as published by the Free Software Foundation.
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
 #
 # This library is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of

data/lib/racknga/middleware/jsonp.rb

@@ -0,0 +1,77 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2010  Kouhei Sutou <kou@clear-code.com>
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+module Racknga
+  module Middleware
+    class JSONP
+      def initialize(application)
+        @application = application
+      end
+
+      def call(environment)
+        request = Rack::Request.new(environment)
+        callback = request["callback"]
+        update_cache_key(request) if callback
+        status, headers, body = @application.call(environment)
+        return [status, headers, body] unless callback
+        return [status, headers, body] unless json_response?(headers)
+        body = Writer.new(callback, body)
+        [status, headers, body]
+      end
+
+      private
+      def update_cache_key(request)
+        return unless Middleware.const_defined?(:Cache)
+        cache_key_key = Cache::KEY_KEY
+
+        path = request.fullpath
+        path, parameters = path.split(/\?/, 2)
+        if parameters
+          parameters = parameters.split(/[&;]/).reject do |parameter|
+            key, value = parameter.split(/\=/, 2)
+            key == "callback" or (key == "_" and value = /\A\d+\z/)
+          end.join("&")
+          path << "?" << parameters unless parameters.empty?
+        end
+
+        key = request.env[cache_key_key]
+        request.env[cache_key_key] = [key, path].compact.join(":")
+      end
+
+      def json_response?(headers)
+        content_type = Rack::Utils::HeaderHash.new(headers)["Content-Type"]
+        content_type == "application/json" or
+          content_type == "application/javascript" or
+          content_type == "text/javascript"
+      end
+
+      class Writer
+        def initialize(callback, body)
+          @callback = callback
+          @body = body
+        end
+
+        def each(&block)
+          block.call("#{@callback}(")
+          @body.each(&block)
+          block.call(");")
+        end
+      end
+    end
+  end
+end