rails_asset_packager 0.1.0

data/.gitignore

@@ -0,0 +1,3 @@
+pkg/*
+*.gem
+.bundle

data/CHANGELOG

@@ -0,0 +1,7 @@
+------------------------------------------------------------------------
+v0.1.0 | plainprogrammer | 2011-01-16
+
+* Converted project into Gem
+* Added task to support S3 distribution (http://www.blog.bridgeutopiaweb.com/post/package-assets-for-rails-3-and-upload-to-amazon-s3/)
+* Added handling of other static assets when pushing to S3.
+------------------------------------------------------------------------

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in rails_asset_packager.gemspec
+gemspec

data/README

@@ -0,0 +1,174 @@
+= RailsAssetPackager
+
+Rails Asset Packager provides an easy way to combine and compress stylesheets
+and Javascript files to improve performance for production environments. The
+packager also provides a convenient way to push those packed files, along with
+any other static assets to Amazon S3.
+
+== Description
+
+When it comes time to deploy your new web application, instead of 
+sending down a dozen JavaScript and CSS files full of formatting 
+and comments, this Rails plugin makes it simple to merge and 
+compress JavaScript and CSS down into one or more files, increasing 
+speed and saving bandwidth.
+
+When in development, it allows you to use your original versions 
+and retain formatting and comments for readability and debugging.
+
+This code is released under the MIT license (like Ruby). You're free 
+to rip it up, enhance it, etc. And if you make any enhancements, 
+I'd like to know so I can add them back in. Thanks!
+
+== Credit
+
+This gem is derived directly from the work of Scott Becker and Katherine G Pe
+on the AssetPackager plugin for Rails.
+
+== Key Features
+
+* Merges and compresses JavaScript and CSS when running in production.
+* Uses uncompressed originals when running in development.
+* Can generate packages on demand in production
+* Provides Rake tasks to pre-package and push static resources to Amazon S3
+
+== Components
+
+* Rake tasks for managing packages and pushing to S3
+* Helper functions for including these JavaScript and CSS files in your views.
+* YAML configuration file for mapping JavaScript and CSS files to packages.
+* Section in YAML configuration for declaring folders of static assets to be
+  pushed to Amazon S3.
+* Rake Task for auto-generating the YAML file from your existing JavaScript files.
+
+== Updates
+
+January 2011:
+* Ported to be packaged as a Gem
+* Leveraging Railties capabilities in Rails 3
+* Added S3 publishing support
+
+== How to Use:
+
+1. Add the gem to your Gemfile
+   gem "rails_asset_packager"
+
+2. Run the rake task "asset:packager:create_yml" to generate the /config/asset_packages.yml
+file the first time. You will need to reorder files under 'base' so dependencies are loaded 
+in correct order. Feel free to rename or create new file packages.
+
+IMPORTANT: JavaScript files can break once compressed if each statement doesn't end with a semi-colon.
+The minifier puts multiple statements on one line, so if the semi-colon is missing, the statement may no 
+longer makes sense and cause a syntax error.
+
+== Examples of config/asset_packages.yml
+
+Example from a fresh rails app after running the rake task. (Stylesheets is blank because a 
+default rails app has no stylesheets yet.):
+
+--- 
+javascripts: 
+- base: 
+  - prototype
+  - effects
+  - dragdrop
+  - controls
+  - application
+stylesheets: 
+- base: []
+uploads:
+- images
+
+Multiple packages:
+
+---
+javascripts:
+- base:
+  - prototype
+  - effects
+  - controls
+  - dragdrop
+  - application
+- secondary:
+  - foo
+  - bar
+stylesheets:
+- base:
+  - screen
+  - header
+- secondary:
+  - foo
+  - bar
+uploads:
+- images
+
+3. Run the rake task "asset:packager:build_all" to generate the compressed, merged versions
+for each package. Whenever you rearrange the yaml file, you'll need to run this task again. 
+
+Merging and compressing is expensive, so this is something we want to do once, not every time
+your app starts. Thats why its a rake task. You can run this task via Capistrano when deploying
+to avoid an initially slow request the first time a page is generated. 
+
+Note: The package will be generated on the fly if it doesn't yet exist, so you don't *need* 
+to run the rake task when deploying, its just recommended for speeding up initial requests.
+
+4. Run the rake task "rake asset:cache:production" to push your static assets and compressed
+packages up to Amazon S3.
+
+Note: You need to have a file at config/s3.yml that defines your Amazon S3 access credentials.
+
+5. Use the helper functions whenever including these files in your application. See below for examples.
+
+6. Potential warning: css compressor function currently removes CSS comments. This might blow
+away some CSS hackery. To disable comment removal, comment out /lib/synthesis/asset_package.rb line 176.
+
+== JavaScript Examples
+
+Example call (based on above /config/asset_packages.yml):
+  <%= javascript_include_merged :base %>
+
+In development, this generates: 
+  <script type="text/javascript" src="/javascripts/prototype.js?1228027240"></script>
+  <script type="text/javascript" src="/javascripts/effects.js?1228027240"></script>
+  <script type="text/javascript" src="/javascripts/controls.js?1228027240"></script>
+  <script type="text/javascript" src="/javascripts/dragdrop.js?1228027240"></script>
+  <script type="text/javascript" src="/javascripts/application.js?1228027240"></script>
+
+In production, this generates: 
+  <script type="text/javascript" src="/javascripts/base_packaged.js?123456789"></script>
+
+== Stylesheet Examples
+
+Example call:
+  <%= stylesheet_link_merged :base %>
+
+In development, this generates:
+  <link href="/stylesheets/screen.css?1228027240" media="screen" rel="Stylesheet" type="text/css" />
+  <link href="/stylesheets/header.css?1228027240" media="screen" rel="Stylesheet" type="text/css" />
+
+In production this generates:
+  <link href="/stylesheets/base_packaged.css?1228027240" media="screen" rel="Stylesheet" type="text/css" />
+
+== Different CSS Media
+
+All options for stylesheet_link_tag still work, so if you want to specify a different media type:
+  <%= stylesheet_link_merged :secondary, 'media' => 'print' %>
+
+== Rake tasks
+
+rake asset:packager:build_all        # Merge and compress assets
+rake asset:packager:create_yml       # Generate asset_packages.yml from existing assets
+rake asset:packager:delete_all       # Delete all asset builds
+rake asset:cache:s3                  # update s3
+rake asset:cache:production          # cache assets and update s3 for production
+
+== License
+Copyright (c) 2006-2008 Scott Becker - http://synthesis.sbecker.net
+Copyright (c) 2011 James Thompson - james@plainprograms.com
+Contact via Github for change requests, etc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,2 @@
+require 'bundler'
+Bundler::GemHelper.install_tasks

data/lib/jsmin.rb

@@ -0,0 +1,211 @@
+#!/usr/bin/ruby
+# jsmin.rb 2007-07-20
+# Author: Uladzislau Latynski
+# This work is a translation from C to Ruby of jsmin.c published by
+# Douglas Crockford.  Permission is hereby granted to use the Ruby
+# version under the same conditions as the jsmin.c on which it is
+# based.
+#
+# /* jsmin.c
+#    2003-04-21
+#
+# Copyright (c) 2002 Douglas Crockford  (www.crockford.com)
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of
+# this software and associated documentation files (the "Software"), to deal in
+# the Software without restriction, including without limitation the rights to
+# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+# of the Software, and to permit persons to whom the Software is furnished to do
+# so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# The Software shall be used for Good, not Evil.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+# Ruby 1.9 Compatibility Fix - the below isAlphanum uses Fixnum#ord to be compatible with Ruby 1.9 and 1.8.7
+# Fixnum#ord is not found by default in 1.8.6, so monkey patch it in:
+if RUBY_VERSION == '1.8.6'
+  class Fixnum; def ord; return self; end; end
+end
+
+EOF = -1
+$theA = ""
+$theB = ""
+
+# isAlphanum -- return true if the character is a letter, digit, underscore,
+# dollar sign, or non-ASCII character
+def isAlphanum(c)
+   return false if !c || c == EOF
+   return ((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') ||
+           (c >= 'A' && c <= 'Z') || c == '_' || c == '$' ||
+           c == '\\' || c[0].ord > 126)
+end
+
+# get -- return the next character from stdin. Watch out for lookahead. If
+# the character is a control character, translate it to a space or linefeed.
+def get()
+  c = $stdin.getc
+  return EOF if(!c)
+  c = c.chr
+  return c if (c >= " " || c == "\n" || c.unpack("c") == EOF)
+  return "\n" if (c == "\r")
+  return " "
+end
+
+# Get the next character without getting it.
+def peek()
+    lookaheadChar = $stdin.getc
+    $stdin.ungetc(lookaheadChar)
+    return lookaheadChar.chr
+end
+
+# mynext -- get the next character, excluding comments.
+# peek() is used to see if a '/' is followed by a '/' or '*'.
+def mynext()
+    c = get
+    if (c == "/")
+        if(peek == "/")
+            while(true)
+                c = get
+                if (c <= "\n")
+                return c
+                end
+            end
+        end
+        if(peek == "*")
+            get
+            while(true)
+                case get
+                when "*"
+                   if (peek == "/")
+                        get
+                        return " "
+                    end
+                when EOF
+                    raise "Unterminated comment"
+                end
+            end
+        end
+    end
+    return c
+end
+
+
+# action -- do something! What you do is determined by the argument: 1
+# Output A. Copy B to A. Get the next B. 2 Copy B to A. Get the next B.
+# (Delete A). 3 Get the next B. (Delete B). action treats a string as a
+# single character. Wow! action recognizes a regular expression if it is
+# preceded by ( or , or =.
+def action(a)
+    if(a==1)
+        $stdout.write $theA
+    end
+    if(a==1 || a==2)
+        $theA = $theB
+        if ($theA == "\'" || $theA == "\"")
+            while (true)
+                $stdout.write $theA
+                $theA = get
+                break if ($theA == $theB)
+                raise "Unterminated string literal" if ($theA <= "\n")
+                if ($theA == "\\")
+                    $stdout.write $theA
+                    $theA = get
+                end
+            end
+        end
+    end
+    if(a==1 || a==2 || a==3)
+        $theB = mynext
+        if ($theB == "/" && ($theA == "(" || $theA == "," || $theA == "=" ||
+                             $theA == ":" || $theA == "[" || $theA == "!" ||
+                             $theA == "&" || $theA == "|" || $theA == "?" ||
+                             $theA == "{" || $theA == "}" || $theA == ";" ||
+                             $theA == "\n"))
+            $stdout.write $theA
+            $stdout.write $theB
+            while (true)
+                $theA = get
+                if ($theA == "/")
+                    break
+                elsif ($theA == "\\")
+                    $stdout.write $theA
+                    $theA = get
+                elsif ($theA <= "\n")
+                    raise "Unterminated RegExp Literal"
+                end
+                $stdout.write $theA
+            end
+            $theB = mynext
+        end
+    end
+end
+
+# jsmin -- Copy the input to the output, deleting the characters which are
+# insignificant to JavaScript. Comments will be removed. Tabs will be
+# replaced with spaces. Carriage returns will be replaced with linefeeds.
+# Most spaces and linefeeds will be removed.
+def jsmin
+    $theA = "\n"
+    action(3)
+    while ($theA != EOF)
+        case $theA
+        when " "
+            if (isAlphanum($theB))
+                action(1)
+            else
+                action(2)
+            end
+        when "\n"
+            case ($theB)
+            when "{","[","(","+","-"
+                action(1)
+            when " "
+                action(3)
+            else
+                if (isAlphanum($theB))
+                    action(1)
+                else
+                    action(2)
+                end
+            end
+        else
+            case ($theB)
+            when " "
+                if (isAlphanum($theA))
+                    action(1)
+                else
+                    action(3)
+                end
+            when "\n"
+                case ($theA)
+                when "}","]",")","+","-","\"","\\", "'", '"'
+                    action(1)
+                else
+                    if (isAlphanum($theA))
+                        action(1)
+                    else
+                        action(3)
+                    end
+                end
+            else
+                action(1)
+            end
+        end
+    end
+end
+
+ARGV.each do |anArg|
+    $stdout.write "// #{anArg}\n"
+end
+
+jsmin

data/lib/rails_asset_packager/asset_package.rb

@@ -0,0 +1,211 @@
+module RailsAssetPackager
+  class AssetPackage
+
+    @asset_base_path    = "#{Rails.root}/public"
+    @asset_packages_yml = File.exists?("#{Rails.root}/config/asset_packages.yml") ? YAML.load_file("#{Rails.root}/config/asset_packages.yml") : nil
+  
+    # singleton methods
+    class << self
+      attr_accessor :asset_base_path,
+                    :asset_packages_yml
+
+      attr_writer   :merge_environments
+      
+      def merge_environments
+        @merge_environments ||= ["production"]
+      end
+      
+      def parse_path(path)
+        /^(?:(.*)\/)?([^\/]+)$/.match(path).to_a
+      end
+
+      def find_by_type(asset_type)
+        asset_packages_yml[asset_type].map { |p| self.new(asset_type, p) }
+      end
+
+      def find_by_target(asset_type, target)
+        package_hash = asset_packages_yml[asset_type].find {|p| p.keys.first == target }
+        package_hash ? self.new(asset_type, package_hash) : nil
+      end
+
+      def find_by_source(asset_type, source)
+        path_parts = parse_path(source)
+        package_hash = asset_packages_yml[asset_type].find do |p|
+          key = p.keys.first
+          p[key].include?(path_parts[2]) && (parse_path(key)[1] == path_parts[1])
+        end
+        package_hash ? self.new(asset_type, package_hash) : nil
+      end
+
+      def targets_from_sources(asset_type, sources)
+        package_names = Array.new
+        sources.each do |source|
+          package = find_by_target(asset_type, source) || find_by_source(asset_type, source)
+          package_names << (package ? package.current_file : source)
+        end
+        package_names.uniq
+      end
+
+      def sources_from_targets(asset_type, targets)
+        source_names = Array.new
+        targets.each do |target|
+          package = find_by_target(asset_type, target)
+          source_names += (package ? package.sources.collect do |src|
+            package.target_dir.gsub(/^(.+)$/, '\1/') + src
+          end : target.to_a)
+        end
+        source_names.uniq
+      end
+
+      def build_all
+        asset_packages_yml.keys.each do |asset_type|
+          asset_packages_yml[asset_type].each { |p| self.new(asset_type, p).build } unless asset_type == 'uploads'
+        end
+      end
+
+      def delete_all
+        asset_packages_yml.keys.each do |asset_type|
+          asset_packages_yml[asset_type].each { |p| self.new(asset_type, p).delete_previous_build } unless asset_type == 'uploads'
+        end
+      end
+
+      def create_yml
+        unless File.exists?("#{Rails.root}/config/asset_packages.yml")
+          asset_yml = Hash.new
+
+          asset_yml['javascripts'] = [{"base" => build_file_list("#{Rails.root}/public/javascripts", "js")}]
+          asset_yml['stylesheets'] = [{"base" => build_file_list("#{Rails.root}/public/stylesheets", "css")}]
+          asset_yml['uploads'] = ["images"]
+
+          File.open("#{Rails.root}/config/asset_packages.yml", "w") do |out|
+            YAML.dump(asset_yml, out)
+          end
+
+          log "config/asset_packages.yml example file created!"
+          log "Please reorder files under 'base' so dependencies are loaded in correct order."
+        else
+          log "config/asset_packages.yml already exists. Aborting task..."
+        end
+      end
+
+    end
+    
+    # instance methods
+    attr_accessor :asset_type, :target, :target_dir, :sources
+  
+    def initialize(asset_type, package_hash)
+      target_parts = self.class.parse_path(package_hash.keys.first)
+      @target_dir = target_parts[1].to_s
+      @target = target_parts[2].to_s
+      @sources = package_hash[package_hash.keys.first]
+      @asset_type = asset_type
+      @asset_path = "#{self.class.asset_base_path}/#{@asset_type}#{@target_dir.gsub(/^(.+)$/, '/\1')}"
+      @extension = get_extension
+      @file_name = "#{@target}_packaged.#{@extension}"
+      @full_path = File.join(@asset_path, @file_name)
+    end
+  
+    def package_exists?
+      File.exists?(@full_path)
+    end
+
+    def current_file
+      build unless package_exists?
+
+      path = @target_dir.gsub(/^(.+)$/, '\1/')
+      "#{path}#{@target}_packaged"
+    end
+
+    def build
+      delete_previous_build
+      create_new_build
+    end
+
+    def delete_previous_build
+      File.delete(@full_path) if File.exists?(@full_path)
+    end
+
+    private
+      def create_new_build
+        new_build_path = "#{@asset_path}/#{@target}_packaged.#{@extension}"
+        if File.exists?(new_build_path)
+          log "Latest version already exists: #{new_build_path}"
+        else
+          File.open(new_build_path, "w") {|f| f.write(compressed_file) }
+          log "Created #{new_build_path}"
+        end
+      end
+
+      def merged_file
+        merged_file = ""
+        @sources.each {|s| 
+          File.open("#{@asset_path}/#{s}.#{@extension}", "r") { |f| 
+            merged_file += f.read + "\n" 
+          }
+        }
+        merged_file
+      end
+    
+      def compressed_file
+        case @asset_type
+          when "javascripts" then compress_js(merged_file)
+          when "stylesheets" then compress_css(merged_file)
+        end
+      end
+
+      def compress_js(source)
+        jsmin_path = File.dirname(__FILE__) + '/..'
+        tmp_path = "#{Rails.root}/tmp/#{@target}_packaged"
+      
+        # write out to a temp file
+        File.open("#{tmp_path}_uncompressed.js", "w") {|f| f.write(source) }
+      
+        # compress file with JSMin library
+        `ruby #{jsmin_path}/jsmin.rb <#{tmp_path}_uncompressed.js >#{tmp_path}_compressed.js \n`
+
+        # read it back in and trim it
+        result = ""
+        File.open("#{tmp_path}_compressed.js", "r") { |f| result += f.read.strip }
+  
+        # delete temp files if they exist
+        File.delete("#{tmp_path}_uncompressed.js") if File.exists?("#{tmp_path}_uncompressed.js")
+        File.delete("#{tmp_path}_compressed.js") if File.exists?("#{tmp_path}_compressed.js")
+
+        result
+      end
+  
+      def compress_css(source)
+        source.gsub!(/\s+/, " ")           # collapse space
+        source.gsub!(/\/\*(.*?)\*\//, "")  # remove comments - caution, might want to remove this if using css hacks
+        source.gsub!(/\} /, "}\n")         # add line breaks
+        source.gsub!(/\n$/, "")            # remove last break
+        source.gsub!(/ \{ /, " {")         # trim inside brackets
+        source.gsub!(/; \}/, "}")          # trim inside brackets
+        source
+      end
+
+      def get_extension
+        case @asset_type
+          when "javascripts" then "js"
+          when "stylesheets" then "css"
+        end
+      end
+      
+      def log(message)
+        self.class.log(message)
+      end
+      
+      def self.log(message)
+        puts message
+      end
+
+      def self.build_file_list(path, extension)
+        re = Regexp.new(".#{extension}\\z")
+        file_list = Dir.new(path).entries.delete_if { |x| ! (x =~ re) }.map {|x| x.chomp(".#{extension}")}
+        # reverse javascript entries so prototype comes first on a base rails app
+        file_list.reverse! if extension == "js"
+        file_list
+      end
+   
+  end
+end

data/lib/rails_asset_packager/asset_package_helper.rb

@@ -0,0 +1,39 @@
+module RailsAssetPackager
+  module AssetPackageHelper
+    
+    def should_merge?
+      AssetPackage.merge_environments.include?(Rails.env)
+    end
+
+    def javascript_include_merged(*sources)
+      options = sources.last.is_a?(Hash) ? sources.pop.stringify_keys : { }
+
+      if sources.include?(:defaults) 
+        sources = sources[0..(sources.index(:defaults))] + 
+          ['prototype', 'effects', 'dragdrop', 'controls'] + 
+          (File.exists?("#{Rails.root}/public/javascripts/application.js") ? ['application'] : []) + 
+          sources[(sources.index(:defaults) + 1)..sources.length]
+        sources.delete(:defaults)
+      end
+
+      sources.collect!{|s| s.to_s}
+      sources = (should_merge? ? 
+        AssetPackage.targets_from_sources("javascripts", sources) : 
+        AssetPackage.sources_from_targets("javascripts", sources))
+        
+      sources.collect {|source| javascript_include_tag(source, options) }.join("\n")
+    end
+
+    def stylesheet_link_merged(*sources)
+      options = sources.last.is_a?(Hash) ? sources.pop.stringify_keys : { }
+
+      sources.collect!{|s| s.to_s}
+      sources = (should_merge? ? 
+        AssetPackage.targets_from_sources("stylesheets", sources) : 
+        AssetPackage.sources_from_targets("stylesheets", sources))
+
+      sources.collect { |source| stylesheet_link_tag(source, options) }.join("\n")    
+    end
+
+  end
+end

data/lib/rails_asset_packager/railtie.rb

@@ -0,0 +1,13 @@
+require 'rails'
+
+module RailsAssetPackager
+  class Railtie < Rails::Railtie
+    initializer "RailsAssetPackager.initialize" do
+      ActionView::Base.send :include, RailsAssetPackager::AssetPackageHelper
+    end
+    
+    rake_tasks do
+      load "tasks/asset_packager_tasks.rake"
+    end
+  end
+end

data/lib/rails_asset_packager/version.rb

@@ -0,0 +1,3 @@
+module RailsAssetPackager
+  VERSION = "0.1.0"
+end

data/lib/rails_asset_packager.rb

@@ -0,0 +1,3 @@
+require 'rails_asset_packager/asset_package'
+require 'rails_asset_packager/asset_package_helper'
+require 'rails_asset_packager/railtie' if defined?(Rails)