RubyGems - rake-pipeline-fork - Versions diffs - 0.8.0 - Mend

rake-pipeline-fork 0.8.0

Files changed (75) hide show

checksums.yaml +15 -0
data/.gitignore +18 -0
data/.rspec +1 -0
data/.travis.yml +12 -0
data/.yardopts +2 -0
data/GETTING_STARTED.md +268 -0
data/Gemfile +14 -0
data/LICENSE +20 -0
data/README.markdown +11 -0
data/README.yard +178 -0
data/Rakefile +21 -0
data/bin/rakep +4 -0
data/examples/copying_files.md +12 -0
data/examples/minifying_files.md +37 -0
data/examples/modifying_pipelines.md +67 -0
data/examples/multiple_pipelines.md +77 -0
data/lib/generators/rake/pipeline/install/install_generator.rb +70 -0
data/lib/rake-pipeline.rb +462 -0
data/lib/rake-pipeline/cli.rb +56 -0
data/lib/rake-pipeline/dsl.rb +9 -0
data/lib/rake-pipeline/dsl/pipeline_dsl.rb +246 -0
data/lib/rake-pipeline/dsl/project_dsl.rb +108 -0
data/lib/rake-pipeline/dynamic_file_task.rb +194 -0
data/lib/rake-pipeline/error.rb +17 -0
data/lib/rake-pipeline/file_wrapper.rb +182 -0
data/lib/rake-pipeline/filter.rb +249 -0
data/lib/rake-pipeline/filters.rb +4 -0
data/lib/rake-pipeline/filters/concat_filter.rb +63 -0
data/lib/rake-pipeline/filters/gsub_filter.rb +56 -0
data/lib/rake-pipeline/filters/ordering_concat_filter.rb +38 -0
data/lib/rake-pipeline/filters/pipeline_finalizing_filter.rb +21 -0
data/lib/rake-pipeline/graph.rb +178 -0
data/lib/rake-pipeline/manifest.rb +86 -0
data/lib/rake-pipeline/manifest_entry.rb +34 -0
data/lib/rake-pipeline/matcher.rb +141 -0
data/lib/rake-pipeline/middleware.rb +72 -0
data/lib/rake-pipeline/precompile.rake +8 -0
data/lib/rake-pipeline/project.rb +335 -0
data/lib/rake-pipeline/rails_plugin.rb +10 -0
data/lib/rake-pipeline/railtie.rb +34 -0
data/lib/rake-pipeline/reject_matcher.rb +29 -0
data/lib/rake-pipeline/server.rb +15 -0
data/lib/rake-pipeline/sorted_pipeline.rb +19 -0
data/lib/rake-pipeline/version.rb +6 -0
data/rails/init.rb +2 -0
data/rake-pipeline.gemspec +24 -0
data/spec/cli_spec.rb +71 -0
data/spec/concat_filter_spec.rb +37 -0
data/spec/dsl/pipeline_dsl_spec.rb +165 -0
data/spec/dsl/project_dsl_spec.rb +41 -0
data/spec/dynamic_file_task_spec.rb +119 -0
data/spec/encoding_spec.rb +106 -0
data/spec/file_wrapper_spec.rb +132 -0
data/spec/filter_spec.rb +332 -0
data/spec/graph_spec.rb +56 -0
data/spec/gsub_filter_spec.rb +87 -0
data/spec/manifest_entry_spec.rb +46 -0
data/spec/manifest_spec.rb +67 -0
data/spec/matcher_spec.rb +141 -0
data/spec/middleware_spec.rb +199 -0
data/spec/ordering_concat_filter_spec.rb +42 -0
data/spec/pipeline_spec.rb +232 -0
data/spec/project_spec.rb +295 -0
data/spec/rake_acceptance_spec.rb +738 -0
data/spec/rake_tasks_spec.rb +21 -0
data/spec/reject_matcher_spec.rb +31 -0
data/spec/sorted_pipeline_spec.rb +27 -0
data/spec/spec_helper.rb +38 -0
data/spec/support/spec_helpers/file_utils.rb +35 -0
data/spec/support/spec_helpers/filters.rb +37 -0
data/spec/support/spec_helpers/input_helpers.rb +23 -0
data/spec/support/spec_helpers/memory_file_wrapper.rb +31 -0
data/spec/support/spec_helpers/memory_manifest.rb +19 -0
data/tools/perfs +101 -0
metadata +215 -0

checksums.yaml ADDED

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    NGIzYWYzYzBiN2VjZDI5ZTAyMjMxMDYxNGI4YTQ3OTgyZGI3OGQ3OQ==
+  data.tar.gz: !binary |-
+    M2RiMzExYWU2YzE0NTc4M2UxYzQ2OGI3OGE4ZDY5ZTgyN2NkOTZhZQ==
+SHA512:
+  metadata.gz: !binary |-
+    NjY5OGRlOThhOTkyNjliM2E3YjM4MmMxNmU4ZWQ3NmE5MDRjNmIzOThkZWMx
+    ZWZkNDEzZTRkMWI1MDRkZGYyNmEyZWYxODg2MTc5NWE4YzM5YjRkNmRjMGZm
+    YjUyZGMxY2U3NDQwNDM4ZjhlYThkY2E0MjBhNmVjMzkwZmU5NmU=
+  data.tar.gz: !binary |-
+    M2RlM2ZkODQwMDNjYjE1MmU1ZGE0YjkxODkwZTExMmFkOTFiN2RjYTAzNTk2
+    MDlhY2M2ZDAwNzhlODFhNjIxOTY1MGQ0ZDRiNDliZTViYTAxNzQ2Y2I5ZDAx
+    OTk1Nzk5ZWE5ZmM2NDlmZWJjNjc3NTQxZGQwZDQ4YjExZDJkMWU=

data/.gitignore ADDED

@@ -0,0 +1,18 @@
+*.gem
+*.rbx
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+devbin

data/.rspec ADDED

	@@ -0,0 +1 @@
1	+ -c -f progress --order random -r spec_helper.rb

data/.travis.yml ADDED

@@ -0,0 +1,12 @@
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - ruby-head
+  - jruby-head
+bundler_args: --without docs
+notifications:
+  email:
+    - wycats@gmail.com
+    - dudley@steambone.org

data/.yardopts ADDED

	@@ -0,0 +1,2 @@
1	+ --readme README.yard
2	+

data/GETTING_STARTED.md ADDED

@@ -0,0 +1,268 @@
+# Rake::Pipeline Basics
+`Rake::Pipeline` provides a basic extraction over a build process. It
+doesn't have many filters out of the box, but it is very powerful. This
+guide gives you an introduction to `Rake::Pipeline`. `Rake::Pipeline`
+was originally designed for frontend development (HTML, CSS,
+Javascript). The examples assume this type of project just to create
+some context.
+## Getting Started
+`Rake::Pipeline` comes with two main functionalities out of the box:
+matching and concatentation. You can specify a match (IE: all css files)
+then run them through a concatenation filter (combine them into one
+file). There is also an order concatenation filter which allows you to
+specify order (A should come before B).
+Your pipeline is written in an `Assetfile`. The `Assetfile` uses a nice
+DSL to make things easier. The `Assetfile` should live in your project's
+root directory.
+Let's get started by writing a basic pipeline. Assume we have this
+directory structure:
+```
+/source
+  /css
+  /javascript
+/compiled
+Assetfile
+```
+The pipeline should simply concatenate all the individual JSS and CSS
+files into single files. So the JSS and CSS directories are the inputs
+and 2 files are outputs. The `source` directory is input and the output
+will go into `compiled`. Here's the `Assetfile` to do just that:
+```ruby
+# Set the Pipeline's output directory
+output "compiled"
+# input defines operations on a set of files. All files processed in
+# this block go into the output directory
+input "source" do
+  # Select all the CSS files
+  match "css/**/*.css" do
+    # concatenate all files in directory into a file named
+    # "application.css"
+    concat "application.css"
+  end
+  # Repeat for javascript files
+  match "javascript/**/*.js" do
+    concat "application.js"
+  end
+end
+```
+Now run `rakep build` from the project root to compile everything.
+Given there are files in `source/css` and `source/javascript` you will
+see files in `compiled` named `application.js` and `application.css`.
+You've just written your first pipeline!
+## Previewing Your Work
+`Rake::Pipeline` also comes with a bundled preview server. Let's add an
+HTML file to the source directory to serve the compiled site. Here's the
+HTML:
+```html
+<!-- source/index.html -->
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="utf-8">
+  <title>Rake::Pipeline Example</title>
+  <link rel="stylesheet" href="/application.css">
+</head>
+<body>
+  <script src="/application.js"></script>
+</body>
+</html>
+```
+Save that file in `source/index.html`. Now we must add some more code to
+the `Assetfile` to copy the HTML file into the output directory.
+`Rake::Pipeline` also bundles a copy filter. Note that this isn't
+actually a separate filter, but a file that concatenates itself to a
+different location. In short, `concat` is aliased to `copy` as well.
+Here's the updated `Assetfile`:
+```ruby
+# Set the Pipeline's output directory
+output "compiled"
+# input defines operations on a set of files. All files processed in
+# this block go into the output directory
+input "source" do
+  # Select all the CSS files
+  match "css/**/*.css" do
+    # concatenate all files in directory into a file named
+    # "application.css"
+    concat "application.css"
+  end
+  # Repeat for javascript files
+  match "javascript/**/*.js" do
+    concat "application.js"
+  end
+  # Explicitly select the HTML file. We don't want to copy
+  # over anything else
+  match "index.html" do
+    # copy also accepts a block. When called without any arguments it
+    # simply uses the same filename
+    copy
+  end
+end
+```
+Now we can run `rakep server` inside the projects root. You'll see some
+output in the terminal with a URL to connect to. Now you an preview your
+work as you go.
+## Writing Filters
+It's very likely that you'll need to do more than just copy and
+concatenate files. You must write your own filters to do this. Luckily,
+writing filters is pretty easy. Filters are classes that have
+`generate_output` method. This is core API requirement. There also other
+method you may implement, but this is the most important. Let's take a
+stab at writing a coffeescript filter.
+Filters are Ruby classes. They map a set of inputs to outputs and
+finally generate the output. Here is an absolute bare bones filter:
+```ruby
+# Inherit from Rake::Pipeline::Filter to get basic API implemented
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+  # This method takes the input files and does whatever is required
+  # to generate the proper output.
+  def generate_output(inputs, output)
+    inputs.each do |input|
+      output.write input.read
+    end
+  end
+end
+```
+Notice `generate_output`'s method signature: `inputs` and `output`. The
+default semantic is to take N input files and map them to one output
+file. You can overide this or do much more fancy things. This is covered
+in a different file. We could use this filter like this:
+```ruby
+output "compiled"
+input "source" do
+  match "**/*.coffee" do
+    filter CoffeeScript
+  end
+end
+```
+Now this filter doesn't do anything at the moment besides copy the
+files. Time to implement coffeescript compiling.
+```ruby
+require "coffee-script"
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+  def generate_output(inputs, output)
+    inputs.each do |input|
+      output.write CoffeeScript.compile(input.read)
+    end
+  end
+end
+```
+Great! Now the CoffeeScript files are compiled to JavaScript. However,
+you may have noticed they are compiled "in place". This means
+`source/app.coffee` will become `source/app.coffee` but as JavaScript.
+This works in our simple example, but what happens when we need to work
+with Javascript later in the pipeline or next build steps expect ".js"
+files? The filter has to customize the name. The most correct thing to
+do is make the output file has the same name except as ".js".
+This behavior is defined in the filter's initializer. This may seem odd
+to you. It was odd to me until I understood what was happening.
+`Rake::Pipeline` instantiates all the filters in order to setup how
+input files map to output files before the pipeline is compiled. This is
+how one filter can use another's outputs for inputs. This order must be
+known at compile time, so that's why it happens here. The internal API
+expects a block that takes a path and maps it to an output path.
+```ruby
+require "coffee-script"
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+  def initialize(&block)
+    &block ||= proc { |input| input.path.gsub(/\.coffee/, ".js")
+    super(&block)
+  end
+  def generate_output(inputs, output)
+    inputs.each do |input|
+      output.write CoffeeScript.compile(input.read)
+    end
+  end
+end
+```
+Let's take a fresh look at the `Assetfile` now.
+```ruby
+output "compiled"
+input "source" do
+  match "javascript/**/*.coffee" do
+    # Compile all .coffee files into.js
+    filter CoffeeScript
+  end
+  # Select the JS generated by previous filters.
+  match "javascript/**/*.js" do
+    concat "application.js"
+  end
+  match "css/**/*.css" do
+    concat "application.css"
+  end
+  match "index.html" do
+    copy
+  end
+end
+```
+Calling the filter without a block uses the default block in the filter.
+The default block that replaces ".js" with ".coffee". This is defined
+with `||=` in the initializer. Conversely you could call `filter` with a
+block and do what you want. Here's an example:
+```ruby
+# output all coffeescript files as "app.coffee.awesome"
+filter CoffeeScript do |input|
+  "#{input.path}.awesome"
+end
+```
+That covers the basics of writing filters. There is much more you can do
+with filters that are outside the scope of this guide. You can find many
+useful (as well as plenty of examples) in the
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters)
+project.
+That also concludes this guide. You should know everything you need to
+know to get started writing your own pipelines now. There is still much
+to cover though. You can find additonal information in the `examples`
+directory. If you'd like to add anything to this guide or find an error
+please open a pull request to fix it.

data/Gemfile ADDED

@@ -0,0 +1,14 @@
+source 'http://rubygems.org'
+# Specify your gem's dependencies in rake-pipeline.gemspec
+gemspec
+gem "flay"
+gem "flog"
+gem "simplecov", :require => false
+gem "pry"
+group :docs do
+  gem "yard"
+  gem "rdiscount"
+end

data/LICENSE ADDED

@@ -0,0 +1,20 @@
+Copyright (C) 2011 by LivingSocial, Inc.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.markdown ADDED

@@ -0,0 +1,11 @@
+# Rake::Pipeline
+The canonical documentation for Rake::Pipeline is hosted at
+<a href="http://rubydoc.info/github/livingsocial/rake-pipeline/master/file/README.yard">rubydoc.info</a>.
+New users are recommended to read `GETTING_STARTED.md` before anything
+else. Additional examples can be found in the `examples` directory.
+Users are also recommended to checkout
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters)
+for commonly used filters.

data/README.yard ADDED

@@ -0,0 +1,178 @@
+= Rake::Pipeline
+Rake::Pipeline is a system for packaging assets for deployment to the
+web. It uses Rake under the hood for dependency management and updating
+output files based on input changes.
+= Usage
+The easiest way to use Rake::Pipeline is via an +Assetfile+ file in the
+root of your project.
+A sample +Assetfile+ looks like this:
+  !!!ruby
+  output "public"
+  input "assets" do
+    # this block will take all JS inputs, wrap them in a closure,
+    # add some additional metadata, and concatenate them all into
+    # application.scripts.js.
+    match "*.js" do
+      filter ClosureWrapper
+      filter DataWrapper
+      concat "application.scripts.js"
+    end
+    # this block will take all HTML and CSS inputs, convert them
+    # into JavaScript
+    match "*/*.{html,css}" do
+      filter DataWrapper
+      concat "application.assets.js"
+    end
+    match "*.js" do
+      concat "application.js"
+    end
+  end
+Each +input+ block defines a collection of files, and a pipeline
+that transforms those files. Within each pipeline, you can specify
+a series of filters to describe the transformations you'd like to
+apply to the files.
+= Upgrading from Previous Versions
+The +Assetfile+ syntax has changed in version 0.6.0. In previous
+versions, each +Assetfile+ defined a single pipeline, and +input+
+statements would add input files to that pipeline. After version
+0.6.0, multiple pipelines can be defined in an +Assetfile+. The
++input+ method now takes a block, and this block defines a pipeline.
+This means that any +match+ blocks or filters must be defined
+inside an +input+ block, and no longer at the top level. For example,
+this:
+    !!!ruby
+    # Prior to 0.6.0
+    output "public"
+    input "assets"
+    match "**/*.js" do
+      concat
+    end
+would now be written as:
+    !!!ruby
+    # After 0.6.0
+    output "public"
+    input "assets" do
+      match "**/*.js" do
+        concat
+      end
+    end
+= Filters
+A filter is a simple class that inherits from
+{Rake::Pipeline::Filter}. A filter must implement a single
+method, called +generate_output+, which takes
+two parameters: a list of input files and the output file.
+Both the input and output files are {Rake::Pipeline::FileWrapper} objects.
+The most important methods on a {Rake::Pipeline::FileWrapper FileWrapper} are:
+* {Rake::Pipeline::FileWrapper#path path}: the path of the file, relative to its input root
+* {Rake::Pipeline::FileWrapper#read read}: read the contents of the file
+* {Rake::Pipeline::FileWrapper#write write(string)}: write a String to the file
+For example, a simple concatenation filter would look like:
+  !!!ruby
+  class ConcatFilter < Rake::Pipeline::Filter
+    def generate_output(inputs, output)
+      inputs.each do |input|
+        output.write input.read
+      end
+    end
+  end
+If you had a series of input files like:
+* +app/javascripts/one.js+
+* +app/javascripts/two.js+
+* +app/javascripts/three.js+
+and you specified the +ConcatFilter+ in your
++Assetfile+ like:
+  !!!ruby
+  filter ConcatFilter, "application.js"
+The filter would receive a single call to
++generate_output+ with an Array of {Rake::Pipeline::FileWrapper FileWrapper}s
+representing each of the three files, and a {Rake::Pipeline::FileWrapper FileWrapper}
+representing +application.js+.
+== Binary Data
+If your filter is operating on binary data, like images,
+rather than textual data, like source code, you can specify
+that in your filter:
+  !!!ruby
+  class ConcatFilter < Rake::Pipeline::Filter
+    processes_binary_files
+    def generate_output(inputs, output)
+      inputs.each do |input|
+        output.write input.read
+      end
+    end
+  end
+This will stop `Rake::Pipeline` from trying to interpret the
+input files as `UTF-8`, which obviously will not work on
+binary data.
+= Filters
++Rake::Pipeline+ comes with a built-in filter,
+{Rake::Pipeline::ConcatFilter}. Its implementation is the same as the
++ConcatFilter+ above. Other filters that are useful for web development
+like a +CoffeeScriptFilter+ and +SassFilter+ are available in
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters).
+= Preview Server
+To start up the preview server, run +rakep server+. This will start up
+a server that automatically recompiles files for you on the fly
+and serves up the files you need.
+This should allow you to have a single index.html file pointing
+at the same files in both development and production.
+= Compiling Assets
+To compile all assets before deployment, simply run:
+  $ rakep build
+= Encodings
+If a filter does not specify that it processes binary files,
++Rake::Pipeline+ will open all inputs and outputs as +UTF-8+.
+This means that if you have files encoded in other encodings,
+like +Latin-1+, +Rake::Pipeline+ will raise an exception. In
+this situation, you need to open the offending file in your
+text editor and re-save it as +UTF-8+.
+= Public Release Requirement
+Before publicly releasing this code, we need to properly support
+encodings other than UTF-8. That means using the
++default_external+ instead of hardcoding to UTF-8 and
+providing a mechanism for specifying the encoding of a file using
+a magic comment.