front-compiler 1.0.6

data/CHANGELOG

@@ -0,0 +1,14 @@
+* 1.0.6 (December 14, 2009) - Nikolay V. Nemshilov aka St.
+  * rubygem is now available
+  * console util was added
+
+* 1.0.2 (April 25, 2009) - Nikolay V. Nemshilov aka St.
+  * Tokens negotiation process update
+  * The self-build script enhancement
+
+* 1.0.0 (April 5, 2009) - Nikolay V. Nemshilov aka St.
+
+  * The Self-Build Feature was implemented
+  * Missed semicolons restoration feature added
+  * Documentation updates
+  * Tagged the source as version 1.0.0

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008 Nikolay V. Nemshilov
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README

@@ -0,0 +1,174 @@
+Welcome!
+========
+
+FrontCompiler is a simple collection of compactors for the JavaScript,
+CSS and HTML source code. It removes trailing whitespaces, comments and
+transformates the local variables to make the sourcecode shorter.
+
+For JavaScript, it can create a more compressed self-building script
+for your code, see the description below.
+
+The library can be used as a plugin for rails. See description below.
+
+
+RubyGem and Console tool
+========================
+
+FrontCompiler is now available as a rubygem at the gemcutter service
+
+gem sources -a http://gemcutter.org
+gem install front-compiler
+
+With the rubygem you'll have a console tool called 'frontcc'
+
+$ frontcc file0.js file1.js file2.js file3.js
+
+
+
+Basic Usage
+===========
+
+It's pretty simple
+
+@c = FrontCompiler.new
+
+@compact_js = @c.compact_js(File.open('src/something.js', 'r'));
+@compact_css = @c.compact_css(File.open('src/something.css', 'r'));
+@compact_html = @c.compact_html(File.open('src/something.css', 'r'));
+
+or shorter
+
+@compact_js = @c.compact_file('src/something.js');
+@compact_css = @c.compact_file('src/something.css');
+@compact_html = @c.compact_file('src/something.css');
+
+or you can compact several files at once like that
+
+@library_code = @c.compact_files(%w{
+  src/lib.js
+  src/lib/something.js
+  src/lib/another.js
+})
+
+Same for the other file-types
+
+CSS Inlining
+============
+
+With the tool you can convert your css source in some javascript
+definition so you could put the styles in the same file as your
+javascript and have -1 (or several) hits to your sever.
+
+File('public/javascripts/all_in_one.js', 'w') do |file|
+  file.write @c.compact_files(%w{
+    public/javascripts/prototype.js
+    public/javascripts/effects.js
+    public/javascripts/application.js
+  })
+
+  file.write @c.inline_css(
+    File.open('public/stylesheets/application.css').read
+  )
+end
+
+Now you have a single javascript file which contains both, javascript
+and the application stylesheets in one.
+
+NOTE: if the user have JavaScript switched off, then he won't see the
+styles.
+
+Nested CSS Handling
+===================
+
+Nested CSS is a feature when you describe your css with nested constructions
+like that
+
+div.article {
+  div.title {
+    font-weight: bold;
+    span.date {
+      color: pink;
+    }
+  }
+  div.text {
+    background: #EEE;
+  }
+}
+
+Which means the same as the following code.
+
+div.article div.title {
+  font-weight: bold;
+}
+div.article div.title span.date {
+  color: pink
+}
+div.article div.text {
+  background: #EEE;
+}
+
+Nested styles are more clean, simple and follows the DRY principle. The only
+little problem, there's no browsers which support the feature yet. But the idea
+is alive and people like it. So we have added the feature emulation in the
+project.
+
+You can create nested css descriptions and then when you compress your css
+with FrontCompiler, it will be automatically converted in a correct css source
+which the browsers can understand.
+
+
+Rails Usage
+===========
+
+The project can be used as a usual rails plugin. Just clone the
+project into your vendor/plugins/front_compiler directory and you will
+have the following methods aviable both in your controllers and
+templates
+
+
+* compact_files(list) - compacts the files fromt the given list and
+                        puts them in a single string. You can specify
+                        a list of file-names here.
+
+* compact_file(file)  - compacts the given file (can be a file-name)
+
+* compact_js(source)  - returns compacted version of the given source
+* compact_css(source)
+* compact_html(source)
+
+* inline_css(source)    - converts the css-source in javascript
+* inline_css_file(file) - converts the given css-file in a javascript
+                          source (can be a file-name)
+
+
+Self-Build Scripts
+==================
+
+FrontCompiler provides you another nice feature, the scripts self-building.
+The idea is that it will convert your code into a string, replace all long
+tokens by short replacements and then compile a javascript code which will
+replace all the tokens back on the user's browser side.
+
+This will give you extra 20-40% compression over the existing FrontCompiler
+compression.
+
+The browser side restoration happens pretty quick and almost invisible for
+the end user.
+
+As the feature actually change the code and requires the end script to be
+executed with JavaScript, it won't work if you use JSON as a media format
+without actual evaluating it as a javascript code. For this reason, to use
+the feature, you need to call it specifically.
+
+@c = FrontCompiler.new
+@c.compact_js(File.open('src/something.js', 'r')).create_self_build;
+
+If you want FrontCompiler to create self-builds by default you can just
+uncomment the call in the java_script.rb file.
+
+
+Enjoy!
+
+--
+The code released under terms of the MIT License
+Copyright (C) 2008-2009 Nikolay V. Nemshilov aka St.

data/Rakefile

@@ -0,0 +1,82 @@
+require "rake"
+require "rake/clean"
+require "rake/gempackagetask"
+require "rake/rdoctask"
+require "rake/testtask"
+require "spec/rake/spectask"
+require "fileutils"
+
+def __DIR__
+  File.dirname(__FILE__)
+end
+
+include FileUtils
+
+NAME = "front_compiler"
+
+require "lib/front_compiler"
+require "lib/front_compiler_helper"
+
+def sudo
+  ENV['FC_SUDO'] ||= "sudo"
+  sudo = windows? ? "" : ENV['FC_SUDO']
+end
+
+def windows?
+  (PLATFORM =~ /win32|cygwin/) rescue nil
+end
+
+def install_home
+  ENV['GEM_HOME'] ? "-i #{ENV['GEM_HOME']}" : ""
+end
+
+##############################################################################
+# Packaging & Installation
+##############################################################################
+CLEAN.include ["**/.*.sw?", "pkg", "lib/*.bundle", "*.gem", "doc/rdoc", ".config", "coverage", "cache"]
+
+desc "Run the specs."
+task :default => :specs
+
+task :frontcompiler => [:clean, :rdoc, :package]
+
+spec = Gem::Specification.new do |s|
+  s.name         = NAME
+  s.version      = FrontCompiler::VERSION
+  s.platform     = Gem::Platform::RUBY
+  s.author       = "Nikolay V. Nemshilov"
+  s.email        = "nemshilov@gmail.com"
+  s.homepage     = "http://st-on-it.blogspot.com"
+  s.summary      = "FrontCompiler is a simple collection of compactors for the JavaScript,
+  CSS and HTML source code. It removes trailing whitespaces, comments and
+  transformates the local variables to make the sourcecode shorter."
+  s.bindir       = "bin"
+  s.description  = s.summary
+  s.executables  = %w(  )
+  s.require_path = "lib"
+  s.files        = %w( README Rakefile init.rb install.rb uninstall.rb ) + Dir["{docs,bin,spec,lib,examples,script}/**/*"]
+
+  # rdoc
+  s.has_rdoc         = true
+  s.extra_rdoc_files = %w( README )
+  #s.rdoc_options     += RDOC_OPTS + ["--exclude", "^(app|uploads)"]
+
+  # Dependencies
+  # s.add_dependency "something"
+  # Requirements
+  s.required_ruby_version = ">= 1.8.4"
+end
+
+Rake::GemPackageTask.new(spec) do |package|
+  package.gem_spec = spec
+end
+
+desc "Run :package and install the resulting .gem"
+task :install => :package do
+  sh %{#{sudo} gem install #{install_home} --local pkg/#{NAME}-#{FrontCompiler::VERSION}.gem --no-rdoc --no-ri}
+end
+
+desc "Run :clean and uninstall the .gem"
+task :uninstall => :clean do
+  sh %{#{sudo} gem uninstall #{NAME}}
+end

data/bin/frontcc

@@ -0,0 +1,53 @@
+#!/usr/bin/env ruby
+$LOAD_PATH.unshift(File.expand_path(File.join(File.dirname(__FILE__),'..','lib')))
+
+require 'front_compiler'
+
+if ARGV.empty? || ARGV.include?('-h') || ARGV.include?('--help')
+  puts %Q{FrontCompiler the Ruby based JavaScript/CSS/HTML compiler
+(C) 2008-2009 Nikolay Nemshilov
+
+
+frontcc [options] file file file ...
+
+options:
+  -h    Show this help
+  -p    Create an albeit packed script
+
+
+The command supports any standard console */* selectors
+and will automatically recognize file-types by extensions
+}
+else
+  cc = FrontCompiler.new
+  
+  javascript = false
+  
+  # creating an overall source
+  source = ARGV.collect{ |path| Dir[path] }.flatten.collect do |path|
+    if File.exists? path
+      src = File.open(path, 'r').read
+      
+      case path.split('.').last.downcase
+        when "html"
+          throw "We don't inline HTML into JavaScript" if javascript
+          cc.compact_html(src)
+        when "css"
+          cc.send(javascript ? :inline_css : :compact_css, src)
+        when "js"
+          javascript = true
+          src =~ /;\s*\Z/ ? src : (src + ';') # <- checking the end semicolon
+          
+        else
+          throw "Unsupported kind of file '#{path}'"
+      end
+    end
+  end.compact.join("")
+  
+  if javascript
+    source = cc.compact_js(source)
+    source = source.create_self_build if ARGV.include?('-p')
+  end
+  
+  puts source
+end

data/init.rb

@@ -0,0 +1,25 @@
+require File.expand_path(File.join(File.dirname(__FILE__), 'lib', 'front_compiler'))
+
+if defined? ActionController
+  ActionController::Base.class_eval {
+    include FrontCompilerHelper
+    helper  FrontCompilerHelper
+  }
+end
+
+# Rails 2.2
+if defined? ActionView::Helpers::AssetTagHelper::AssetCollection
+  ActionView::Helpers::AssetTagHelper::JavaScriptSources.class_eval do
+    alias :original_joined_contents :joined_contents
+    def joined_contents
+      FrontCompiler.new.compact_js(original_joined_contents)
+    end
+  end
+  ActionView::Helpers::AssetTagHelper::StylesheetSources.class_eval do
+    alias :original_joined_contents :joined_contents
+    def joined_contents
+      FrontCompiler.new.compact_css(original_joined_contents)
+    end
+  end
+end
+

data/lib/front-compiler.rb

@@ -0,0 +1,5 @@
+#
+# Hook for rails when the plugin is used as a gem
+#
+require 'front_compiler'
+require File.dirname(__FILE__) + '/../init.rb'

data/lib/front_compiler/css_source/nested_styles.rb

@@ -0,0 +1,72 @@
+#
+# This module contains the nested css handling
+# it converts virtual nested css constructions to standard css constructions
+# 
+# Copyright (C) Nikolay V. Nemshilov aka St.
+#
+class FrontCompiler
+  class FrontCompiler::CssSource < FrontCompiler::SourceCode
+    module NestedStyles
+      # overloads the constructor, to convert the styles on fly
+      def initialize(src, keep_as_is=false)
+        super src
+        convert_nested_styles! unless keep_as_is
+      end
+      
+      # converts the nested styles constructions in the source
+      CSS_CONSTRUCTION_RE = /((\A|\}|;)\s*?([^\}\{;]+?)\s*?)\{.*?\}/m
+      def convert_nested_styles!
+        # loop though the blocks
+        offset = 0
+        while pos = index(CSS_CONSTRUCTION_RE, offset)
+          pos += $1.size # <- getting the actual block position
+          block = find_block("{}", pos)
+          block = block[1, block.size-1]; pos+=1 # <- remove the container
+          block_size = block.size
+          
+          parent_rules = clean_rules_from $3 # <- the block rules list
+
+          block_sub_styles = []
+          
+          # looking for a nested construction
+          while block_pos = block.index(CSS_CONSTRUCTION_RE)
+            trail_char = $2.dup
+            
+            block_start = $1.dup
+            block_start = block_start[trail_char.size, block_start.size]
+            block_start_size = block_start.size
+            
+            block_pos += trail_char.size # <- updating the sub-block position
+            
+            # update the sub-bolock rules
+            clean_rules_from($3).each do |block_rule|
+              block_start.gsub! block_rule, parent_rules.collect{ |p_rule|
+                "#{p_rule} #{block_rule}"
+              }.join(", ")
+            end
+            
+            # getting the construction body
+            block_body = find_block("{}", block_pos + block_start_size, block)
+            
+            # removing the construction out of the block
+            block[block_pos, block_start_size + block_body.size] = ''
+            block_sub_styles << block_start + block_body
+          end
+          
+          # replacing the block
+          self[pos, block_size] = block + block_sub_styles.join('')
+          
+          offset = pos + block.size - 1
+        end
+      end
+      
+    protected  
+      # creates a clean css-rules list out of the str
+      def clean_rules_from(str)
+        str.split(',').collect{ |rule|
+          rule.gsub(/\s+/, ' ').gsub(/\/\*.*?\*\//im, '').strip
+        }
+      end
+    end
+  end
+end

data/lib/front_compiler/css_source.rb

@@ -0,0 +1,42 @@
+#
+# The CSS sources compactor
+#
+# Copyright (C) Nikolay V. Nemshilov aka St.
+#
+require "front_compiler/css_source/nested_styles"
+
+class FrontCompiler::CssSource < FrontCompiler::SourceCode
+  include NestedStyles
+  
+  # removes all the comments out of the given source
+  def remove_comments!
+    string_safely do 
+      gsub!(/\/\*.*?\*\//im, '')
+    end
+  end
+  
+  # removes all the empty lines out of the source code
+  def remove_empty_lines!
+    string_safely do
+      gsub!(/\n\s*\n/m, "\n")
+    end
+  end
+  
+  # removes tailing whitespaces out of the source code
+  def remove_trailing_spaces!
+    string_safely do
+      gsub!(/\s+/im, ' ')
+      gsub!(/\s*(\+|>|\||~|\{|\}|,|\)|\(|;|:|\*)\s*/im, '\1')
+      gsub!(/;\}/, '}')
+      strip!
+    end
+  end
+  
+  # converts the source in such a way that it could be
+  # delivered with javascript in the same file
+  def to_javascript
+    "document.write(\"<style type=\\\"text/css\\\">#{
+       compact.gsub('"', '\"')
+     }</style>\");"
+  end
+end

data/lib/front_compiler/html_compactor.rb

@@ -0,0 +1,24 @@
+#
+# The HTML sources compactor
+#
+# Copyright (C) Nikolay V. Nemshilov aka St.
+#
+class FrontCompiler::HTMLCompactor
+  # applies all the compactings to the given source
+  def minimize(source)
+    source = remove_comments(source)
+    source = remove_trailing_spaces(source)
+  end
+  
+  # removes all the comments out of the code
+  def remove_comments(source)
+    source.gsub /<!--.*?-->/, ''
+  end
+  
+  # remove all the trailing spaces out of the code
+  def remove_trailing_spaces(source)
+    source.gsub! /\s+/, ' '
+    source.gsub! />\s+/, '>'
+    source.gsub  /\s+</, '<'
+  end
+end

data/lib/front_compiler/java_script/logic_compactor.rb

@@ -0,0 +1,135 @@
+#
+# This module contains the javascript logical
+# structures processor/compactor
+# 
+# This module is a part of the JavaScript class and taken out
+# just to keep the things simple
+# 
+# Copyright (C) Nikolay V. Nemshilov aka St.
+#
+class FrontCompiler
+  class JavaScript < FrontCompiler::SourceCode
+    module LogicCompactor
+      #
+      # checks and compacts the script-logic
+      #
+      def compact_logic!
+        string_safely do
+          join_multiline_defs!
+          fix_missed_semicolons!
+          simplify_constructions_of self
+        end
+      end
+      
+    protected
+      #
+      # joins constructions which split on several lines
+      #
+      MULTILINING_CHARS_RE = %w{ ? : = - + * / % ! & | < > , 
+      }.collect{ |c| Regexp.escape(c)}.join("|")
+      def join_multiline_defs!
+        gsub!(/\s+(#{MULTILINING_CHARS_RE})\s+/m, ' \1 ')
+        gsub!(/\s+(#{MULTILINING_CHARS_RE})(\S)/m, ' \1\2')
+        gsub!(/(\S)(#{MULTILINING_CHARS_RE})\s+/m, '\1\2 ')
+      
+        gsub!(/\s*(\.)\s*/m, '\1') # <- fold object member calls
+      
+        gsub!(/(\()\s+/m, '\1') # <- fold method argument defs
+        gsub!(/\s+(\))/m, '\1')
+      end
+      
+      #
+      # tries to simplify the short logical constructions
+      # which were defined as multilined but could be simplier
+      #
+      def simplify_constructions_of(src)
+        [/(if|for|while)\s*\(/im, /(else|do)\s*\s\{/im].each do |regexp|
+          offset = 0
+          while pos = src.index(regexp, offset)
+            block = name = $1.dup
+            block += find_block("()", pos + block.size, src)
+            body = find_block("{}", pos + block.size, src)
+            
+            unless body == ''
+              block += body[/\A\s*/m]; body.strip! # removing starting empty-chars
+              body_code = body[1, body.size-2]
+              
+              # checking if the code can be simplified
+              can_be_simplified = number_of_code_lines_in(body_code) == 1
+              
+              # check additional 'if' constructions restrictions
+              if can_be_simplified and name == 'if'
+                # double ifs construction check
+                can_be_simplified &= !body_code.match(/\A\s*if\s*\(/im)
+                
+                # checking the else blocks missintersections
+                if src[pos+block.size+body.size, 40].match(/\A\s*else(\s+|\{)/)
+                  can_be_simplified &= !body_code.match(/(\A|[^a-z\d_$])if\s*\(/)
+                end
+              end
+              
+              # try to simplify internal constructions
+              simplify_constructions_of body_code
+              
+              if can_be_simplified
+                check_semicolons_in body_code
+              else
+                body_code = "{#{body_code}}"
+              end
+              
+              src[pos+block.size, body.size] = body_code
+              body = body_code
+            end
+            
+            offset = pos + block.size + body.size
+          end
+        end
+      end
+      
+      # calculates the number of code-lines in the string
+      def number_of_code_lines_in(src)
+        src = src.dup
+        
+        # replacing all the method calls, blocks and lists with dummies
+        ['[]', '()', '{}'].each do |pair|
+          offset = 0
+          while pos = src.index(pair[0,1], offset)
+            offset = pos + 1
+            block = find_block(pair, 0, src[pos, src.size])
+            
+            src[pos, block.size] = pair
+          end
+        end
+        
+        # putting semicolons after try/catch/finally constructions, so they were conuted as well
+        src.gsub!(/(catch\s*\(\)|finally)\s*\{\}/, '\&;')
+        src.gsub!(/\(\)\s*\{\}/, '\&;')
+        
+        # calculating the number of the lines
+        src.split(';').collect{ |line|
+          line.strip != '' ? 1 : nil
+        }.compact.size
+      end
+    end
+    
+    # checks if there's ommited semicolons in the code
+    def check_semicolons_in(src)
+      src[/\s*\Z/m] = ";#{src[/\s*\Z/m]}" unless src.strip[-1,1].match(/[;\}]/)
+      
+      # removing bug causing semicolons after a function
+      src.gsub!(/(\A\s*function\s*.*?\(.*?\)\s*\{.*?\});(\s*\Z)/im) do
+        $1 + $2
+      end
+    end
+    
+    def fix_missed_semicolons!
+      gsub! /([^{}()|\[&;:=,\s])([ \t]*?\n\s*?(return|if|function|while|try|do)[^a-zA-Z_$])/, '\1;\2'
+      
+      # (function(){....})(..) if(...)
+      gsub! /(\}\)\([^)]*\))([ \t]*?\n\s*?(return|if|function|while|try|do)[^a-zA-Z_$])/, '\1;\2'
+      
+      # fixing wrong added semicolons before the if calls
+      gsub! /([^a-zA-Z_$](else|do));(\s+(if|while|do|try)[^a-zA-Z_$])/, '\1\3'
+    end
+  end
+end