packr 3.1.1 → 3.2.1

data/History.txt

@@ -1,7 +1,18 @@
+=== 3.2.1 / 2013-02-10
+
+* 3.2.0 has been yanked due to unexpected files in the download -- this is replacing it
+
+
+=== 3.2.0 / 2012-04-02
+
+* Added source map generation for all compression modes
+
+
 === 3.1.1 / 2011-07-19
 
 * Remove dependency on Hoe
 
+
 === 3.1.0 / 2009-02-22
 
 * Project is now a gem, not a Rails plugin
@@ -10,10 +21,12 @@
 * Changed variable protection, protected variables must be supplied
   as options to Packr#pack.
 
+  
 === 1.0.2 / 2007-12-13
 
 * Added variable protection, protecting $super for use with Prototype
 
+
 === 1.0.0 / 2007-12-04
 
 * Initial release, compatible with Packer 3.0

data/README.rdoc

@@ -1,24 +1,19 @@
-= PackR
+= Packr {<img src="https://secure.travis-ci.org/jcoglan/packr.png" alt="Build Status" />}[http://travis-ci.org/jcoglan/packr]
 
-* http://github.com/jcoglan/packr
-* http://dean.edwards.name/packer/
-* http://base2.googlecode.com
+Packr is a Ruby implementation of Dean Edwards' JavaScript compressor,
+{Packer}[http://dean.edwards.name/packer/]. It can remove comments and
+whitespace, compress variable names, obfuscate <tt>_private</tt> identifiers,
+and base-62 encode your programs. It can also generate {source
+maps}[http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/], which
+lets your browser trace log messages and errors back to the original source,
+even when running minified code.
 
+== Usage
 
-== Description
+Packr provides both a command-line interface and a Ruby API. To call it from the
+command line (use <tt>packr --help</tt> to see available options):
 
-PackR is a Ruby version of Dean Edwards' JavaScript compressor.
-
-
-== Features
-
-* Whitespace and comment removal
-* Compression of local variable names
-* Compression and obfuscation of 'private' (_underscored) identifiers
-* Base-62 encoding
-
-
-== Synopsis
+  packr my_script.js > my_script.min.js
 
 To call from within a Ruby program:
 
@@ -26,7 +21,7 @@ To call from within a Ruby program:
   
   code = File.read('my_script.js')
   compressed = Packr.pack(code)
-  File.open('my_script.min.js', 'wb') { |f| f.write(compressed) }
+  File.open('my_script.min.js', 'w') { |f| f.write(compressed) }
 
 This method takes a number of options to control compression, for example:
 
@@ -34,19 +29,83 @@ This method takes a number of options to control compression, for example:
 
 The full list of available options is:
 
+* <tt>:minify</tt> -- set to +false+ to prevent any compression; this is useful
+  if you just want to use <tt>Packr.bundle</tt> to concatenate files and
+  generate a source map
 * <tt>:shrink_vars</tt> -- set to +true+ to compress local variable names
 * <tt>:private</tt> -- set to +true+ to obfuscate 'private' identifiers, i.e.
   names beginning with a single underscore
 * <tt>:base62</tt> -- encode the program using base 62
-* <tt>:protect</tt> -- an array of variable names to protect from compression, e.g.
+* <tt>:protect</tt> -- an array of variable names to protect from compression
+* <tt>:header</tt> -- an optional string to prepend to the output, e.g. for
+  copyright comments
 
-  compressed = Packr.pack(code, :shrink_vars => true,
-                                :protect => %w[$super self])
+Here's an example using all of these:
 
-To call from the command line (use <tt>packr --help</tt> to see available
-options):
+  compressed = Packr.pack(code,
+    :shrink_vars => true,
+    :private     => true,
+    :protect     => %w[$super self],
+    :header      => '/* Copyright 2012 some guy */'
+  )
 
-  packr my_script.js > my_script.min.js
+
+=== Bundling and source maps
+
+Packr also provides an API for bundling several files together and generating a
+single output file and {source
+map}[http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/]. For
+example, given these two files:
+
+  # example_a.js
+  
+  1. // When the minified code is loaded into a browser, you should see the call to
+  2. // console.log() attributed to example_a.js:4
+  3. 
+  4. console.log('Hello from file A');
+
+and
+
+  # example_b.js
+  
+  1. var display = function(message) {
+  2.   alert(message + ' from file B');
+  3. };
+  4. 
+  5. display('Ahoy there');
+
+
+it can generate a combined output file:
+
+  # example-min.js
+  
+  /* Copyright 2012 */
+  console.log('Hello from file A');var display=function(a){alert(a+' from file B')};display('Ahoy there');
+  //@ sourceMappingURL=example-min.js.map
+
+and a source map:
+
+  # example-min.js.map
+  
+  {
+    "version": 3,
+    "file": "example-min.js",
+    "sourceRoot": "",
+    "sources": ["../example_a.js", "../example_b.js"],
+    "names": ["message"],
+    "mappings": ";AAGA,QAAQ,KAAK,MAAM,KAAK,KAAK,ICH7B,IAAI,QAAU,SAASA,GACrB,MAAMA,IAAY,KAAK,KAAK,KAG9B,SAAS,KAAK;"
+  }
+
+To do this, you use the +bundle+ method, passing an array of input files and a
+single output file, and the compression options you want to use.
+
+  Packr.bundle %w[example_a.js example_b.js], 'min/example-min.js',
+    :shrink_vars => true,
+    :header      => '/* Copyright 2012 */'
+
+On the command line, this looks like:
+
+  packr example_a.js example_b.js -o min/example-min.js -h '/* Copyright 2012 */' --shrink-vars
 
 
 == Notes
@@ -56,39 +115,26 @@ expressions. Be sure to include semicolons and braces everywhere they are
 required so that your program will work correctly when packed down to a single
 line.
 
-By far the most efficient way to serve JavaScript over the web is to use PackR
-with the --shrink-vars flag, combined with gzip compression. If you don't have
-access to your server config to set up mod_deflate, you can generate gzip files
-using (on Unix-like systems):
-
-  packr -s my-file.js | gzip > my-file.js.gz
-  
-You can then get Apache to serve the files by putting this in your .htaccess
-file:
-
-  AddEncoding gzip .gz
-  RewriteCond %{HTTP:Accept-encoding} gzip
-  RewriteCond %{HTTP_USER_AGENT} !Safari
-  RewriteCond %{REQUEST_FILENAME}.gz -f
-  RewriteRule ^(.*)$ $1.gz [QSA,L]
+By far the most efficient way to serve JavaScript over the web is to use Packr
+with the <tt>--shrink-vars</tt> flag, combined with gzip compression.
 
-If you really cannot serve gzip files, use the --base62 option to further
-compress your code. This mode is at its best when compressing large files with
-many repeated tokens.
+If you really cannot serve gzip files, use the <tt>--base62</tt> option to
+further compress your code. This mode is at its best when compressing large
+files with many repeated tokens.
 
-The --private option can be used to stop other programs calling private methods
-in your code by renaming anything beginning with a single underscore. Beware
-that you should not use this if the generated file contains 'private' methods
-that need to be accessible by other files. Also know that all the files that
-access any particular private method must be compressed together so they all get
-the same rewritten name for the private method.
+The <tt>--private</tt> option can be used to stop other programs calling private
+methods in your code by renaming anything beginning with a single underscore.
+Beware that you should not use this if the generated file contains 'private'
+methods that need to be accessible by other files. Also know that all the files
+that access any particular private method must be compressed together so they
+all get the same rewritten name for the private method.
 
 
 == License
 
 (The MIT License)
 
-Copyright (c) 2004-2011 Dean Edwards, James Coglan
+Copyright (c) 2004-2012 Dean Edwards, James Coglan
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the 'Software'), to deal in

data/bin/packr

@@ -5,76 +5,78 @@ require 'oyster'
 require File.expand_path('../../lib/packr', __FILE__)
 
 spec = Oyster.spec do
-  name    "packr -- JavaScript code compressor based on Dean Edwards' Packer"
+  name "packr -- JavaScript code compressor based on Dean Edwards' Packer"
   
   synopsis <<-EOS
-  packr [OPTIONS] INPUT_FILES > OUTPUT_FILE
+  packr [OPTIONS] INPUT_FILES -o OUTPUT_FILE
   cat INPUT_FILES | packr [OPTIONS] > OUTPUT_FILE
   EOS
   
   description <<-EOS
-  PackR is a program for compressing JavaScript programs. It can remove
-  whitespace and comments, compress local variable names, compress/obfuscate
-  private identifiers, and encode the program in base-62.
+  Packr is a program for compressing JavaScript programs. It can remove comments
+  and whitespace, compress variable names, obfuscate '_private' identifiers, and
+  base-62 encode your programs. It can also generate source maps, which lets
+  your browser trace log messages and errors back to the original source, even
+  when running minified code.
   
   When invoked from the command line, it concatenates all the code in
   INPUT_FILES (or from standard input) and compresses the code using the given
-  options, printing the result to standard output. You can pipe this output into
-  another file to save it.
+  options, printing the result to standard output. If the --output option is
+  used, the result is written to the given file along with a source map.
   EOS
   
-  flag    :'shrink-vars',   :default => true,
+  flag :minify, :default => true,
+  :desc => 'Remove unnecessary whitespace, comments and other tokens'
+  
+  flag :'shrink-vars', :default => true,
   :desc => 'Shrink local variable names inside functions'
   
-  flag    :private,         :default => false,
+  flag :private, :default => false,
   :desc => 'Obfuscate private identifiers, i.e. names beginning with a single underscore'
   
-  flag    :base62,          :default => false,
+  flag :base62, :default => false,
   :desc => 'Encode the program using base 62'
   
-  array   :protect,         :default => [],
+  array :protect, :default => [],
   :desc => 'List of variable names to protect from compression when using --shrink-vars'
   
+  string :header,
+  :desc => 'Optional string to prepend to the output, e.g. for copyright comments'
+  
+  string :output,
+  :desc => <<-EOS
+  Path to which to write compressed output; output will be written to stdout
+  unless this is passed. If this option is used with a list of INPUT_FILES then
+  a source map will also be written to OUTPUT_FILE.map.
+  EOS
+  
   notes <<-EOS
   This program is not a JavaScript parser, and rewrites your files using regular
   expressions. Be sure to include semicolons and braces everywhere they are
   required so that your program will work correctly when packed down to a single
   line.
   
-  By far the most efficient way to serve JavaScript over the web is to use PackR
-  with the --shrink-vars flag, combined with gzip compression. If you don't have
-  access to your server config to set up mod_deflate, you can generate gzip
-  files using (on Unix-like systems):
-  
-    packr -s my-file.js | gzip > my-file.js.gz
-    
-  You can then get Apache to serve the files by putting this in your .htaccess
-  file:
-  
-    AddEncoding gzip .gz
-    RewriteCond %{HTTP:Accept-encoding} gzip
-    RewriteCond %{HTTP_USER_AGENT} !Safari
-    RewriteCond %{REQUEST_FILENAME}.gz -f
-    RewriteRule ^(.*)$ $1.gz [QSA,L]
+  By far the most efficient way to serve JavaScript over the web is to use Packr
+  with the --shrink-vars flag, combined with gzip compression.
   
   If you really cannot serve gzip files, use the --base62 option to further
   compress your code. This mode is at its best when compressing large files with
   many repeated tokens.
   
-  The --private option can be used to stop other programs calling private
-  methods in your code by renaming anything beginning with a single underscore.
-  Beware that you should not use this if the generated file contains 'private'
-  methods that need to be accessible by other files. Also know that all the
-  files that access any particular private method must be compressed together so
-  they all get the same rewritten name for the private method.
+  The --private option can be used to stop other programs calling private methods
+  in your code by renaming anything beginning with a single underscore. Beware
+  that you should not use this if the generated file contains 'private' methods
+  that need to be accessible by other files. Also know that all the files that
+  access any particular private method must be compressed together so they all
+  get the same rewritten name for the private method.
   EOS
   
   author <<-EOS
-  Original JavaScript version by Dean Edwards, Ruby port by James Coglan <jcoglan@googlemail.com>
+  Original JavaScript version by Dean Edwards, Ruby port by James Coglan <jcoglan@gmail.com>
   EOS
   
   copyright <<-EOS
-  Copyright (c) 2004-2011 Dean Edwards, James Coglan. This program is free
+  Copyright (c) 2004-2012 Dean Edwards, James Coglan. This program is free
   software, distributed under the MIT license.
   EOS
 end
@@ -82,16 +84,30 @@ end
 begin
   opts = spec.parse
   
-  inputs = opts[:unclaimed]
-  code = inputs.empty? ?
-      $stdin.read :
-      inputs.map { |f| File.read(f) }.join("\n")
-  
-  $stdout.puts Packr.pack(code,
-    :shrink_vars => !!opts[:'shrink-vars'],
+  compression_options = {
+    :minify      => opts[:minify],
+    :shrink_vars => opts[:'shrink-vars'],
+    :private     => opts[:private],
     :protect     => opts[:protect],
-    :private     => !!opts[:private],
-    :base62      => !!opts[:base62])
+    :base62      => opts[:base62],
+    :header      => opts[:header]
+  }
+  
+  inputs = opts[:unclaimed]
+  output = opts[:output]
+  
+  if inputs.empty?
+    result = Packr.pack($stdin.read, compression_options)
+    if output
+      FileUtils.mkdir_p(File.expand_path('..', output))
+      File.open(output, 'w') { |f| f.write(result) }
+    else
+      $stdout.puts(result)
+    end
+  else
+    result = Packr.bundle(inputs, output, compression_options)
+    $stdout.puts(result) unless output
+  end
   
 rescue Oyster::HelpRendered
 end

data/example/compile.rb

@@ -0,0 +1,11 @@
+require 'rubygems'
+require File.expand_path('../../lib/packr', __FILE__)
+
+dir = File.expand_path('..', __FILE__)
+sources = ["#{dir}/script_a.js", "#{dir}/script_b.js"]
+
+Packr.bundle(sources => "#{dir}/min/script-min.js",
+  :shrink_vars => true,
+  :private     => true,
+  :header      => '/* Copyright 2012 some guy */'
+)

data/example/index.html

@@ -0,0 +1,12 @@
+<!doctype html>
+
+<html>
+  <head>
+    <meta http-equiv="Content-type" content="text/html; charset=utf-8">
+    <title>Source maps test</title>
+    <script type="text/javascript" src="/min/script-min.js"></script>
+  </head>
+  <body>
+    <h1>Source maps test</h1>
+  </body>
+</html>

data/example/script_a.js

@@ -0,0 +1,4 @@
+// When the minified code is loaded into a browser, you should see the call to
+// console.log() attributed to script_a.js:4
+
+console.log('Hello from file A');

data/example/script_b.js

@@ -0,0 +1,32 @@
+console.log('Hello from file B');
+
+(function(config) {
+  /* Log a few messages... */
+  for (var index = 0; index < 2; index++) {
+    console.log('Message ' + index);
+    console.log('Again! ' + index);
+  }
+  
+  var object = {
+    /**
+     * object#name -> undefined
+     * @throws Error
+     **/
+    _name: function() {
+      throw new Error('Oh noes!');
+    }
+  };
+  
+  // Find out where errors come from. We call something that throws an error
+  // and console.log() the stack trace.
+  try {
+    object._nonesuch();
+  } catch (e) {
+    console.log(e.stack);
+  }
+  
+  /*
+  Then we can something else and let the browser deal with it.
+  */
+  object._name();
+})(window);

data/lib/packr.rb

@@ -1,20 +1,29 @@
-[ '/string',
-  '/packr/map',
-  '/packr/collection',
-  '/packr/regexp_group',
-  '/packr/constants',
-  '/packr/encoder',
-  '/packr/parser',
-  '/packr/minifier',
-  '/packr/privates',
-  '/packr/shrinker',
-  '/packr/words',
-  '/packr/base62'
-].each do |path|
-  require File.dirname(__FILE__) + path
-end
+require 'erb'
+require 'fileutils'
+require 'forwardable'
+require 'set'
+require 'strscan'
 
 class Packr
+  dir = File.expand_path('../packr', __FILE__)
+  
+  autoload :Collection,  dir + '/base2/collection'
+  autoload :Map,         dir + '/base2/map'
+  autoload :RegexpGroup, dir + '/base2/regexp_group'
+  
+  autoload :Base62,      dir + '/base62'
+  autoload :Encoder,     dir + '/encoder'
+  autoload :FileSystem,  dir + '/file_system'
+  autoload :Minifier,    dir + '/minifier'
+  autoload :Parser,      dir + '/parser'
+  autoload :Privates,    dir + '/privates'
+  autoload :Shrinker,    dir + '/shrinker'
+  autoload :SourceMap,   dir + '/source_map'
+  autoload :Words,       dir + '/words'
+  
+  IGNORE = RegexpGroup::IGNORE
+  REMOVE = ""
+  SPACE  = " "
   
   DATA = Parser.new.
     put("STRING1", IGNORE).
@@ -22,6 +31,12 @@ class Packr
     put("CONDITIONAL", IGNORE). # conditional comments
     put("(OPERATOR)\\s*(REGEXP)", "\\1\\2")
   
+  module StringExtension
+    attr_accessor :source_map, :code
+    extend Forwardable
+    def_delegators :source_map, :header, :footer
+  end
+  
   def self.encode62(c)
     (c < 62 ? '' : encode62((c / 62.0).to_i)) +
         ((c = c % 62) > 35 ? (c+29).chr : c.to_s(36))
@@ -39,8 +54,11 @@ class Packr
   end
   
   def self.pack(script, options = {})
-    @packr ||= self.new
-    @packr.pack(script, options)
+    new.pack(script, options)
+  end
+  
+  def self.bundle(*options)
+    FileSystem.bundle(*options)
   end
   
   def initialize
@@ -51,12 +69,21 @@ class Packr
   end
   
   def pack(script, options = {})
-    script = @minifier.minify(script)
-    script = @shrinker.shrink(script, options[:protect]) if options[:shrink_vars]
-    script = @privates.encode(script) if options[:private]
-    script = @base62.encode(script) if options[:base62]
-    script
-  end
-  
+    minify     = (options[:minify] != false)
+    source_map = SourceMap.new(script, options)
+    script     = source_map.source_code
+    
+    if minify
+      script = @minifier.minify(script) { |sections| source_map.remove(sections) }
+    end
+    
+    script = @shrinker.shrink(script, options[:protect]) if minify && options[:shrink_vars]
+    script = @privates.encode(script) if minify && options[:private]
+    
+    source_map.update(script)
+    script = @base62.encode(script) if minify && options[:base62]
+    
+    source_map.append_metadata(script)
+  end 
 end