RubyGems - rake-pipeline-fork - Versions diffs - 0.8.0 - Mend

rake-pipeline-fork 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (75) hide show

checksums.yaml +15 -0
data/.gitignore +18 -0
data/.rspec +1 -0
data/.travis.yml +12 -0
data/.yardopts +2 -0
data/GETTING_STARTED.md +268 -0
data/Gemfile +14 -0
data/LICENSE +20 -0
data/README.markdown +11 -0
data/README.yard +178 -0
data/Rakefile +21 -0
data/bin/rakep +4 -0
data/examples/copying_files.md +12 -0
data/examples/minifying_files.md +37 -0
data/examples/modifying_pipelines.md +67 -0
data/examples/multiple_pipelines.md +77 -0
data/lib/generators/rake/pipeline/install/install_generator.rb +70 -0
data/lib/rake-pipeline.rb +462 -0
data/lib/rake-pipeline/cli.rb +56 -0
data/lib/rake-pipeline/dsl.rb +9 -0
data/lib/rake-pipeline/dsl/pipeline_dsl.rb +246 -0
data/lib/rake-pipeline/dsl/project_dsl.rb +108 -0
data/lib/rake-pipeline/dynamic_file_task.rb +194 -0
data/lib/rake-pipeline/error.rb +17 -0
data/lib/rake-pipeline/file_wrapper.rb +182 -0
data/lib/rake-pipeline/filter.rb +249 -0
data/lib/rake-pipeline/filters.rb +4 -0
data/lib/rake-pipeline/filters/concat_filter.rb +63 -0
data/lib/rake-pipeline/filters/gsub_filter.rb +56 -0
data/lib/rake-pipeline/filters/ordering_concat_filter.rb +38 -0
data/lib/rake-pipeline/filters/pipeline_finalizing_filter.rb +21 -0
data/lib/rake-pipeline/graph.rb +178 -0
data/lib/rake-pipeline/manifest.rb +86 -0
data/lib/rake-pipeline/manifest_entry.rb +34 -0
data/lib/rake-pipeline/matcher.rb +141 -0
data/lib/rake-pipeline/middleware.rb +72 -0
data/lib/rake-pipeline/precompile.rake +8 -0
data/lib/rake-pipeline/project.rb +335 -0
data/lib/rake-pipeline/rails_plugin.rb +10 -0
data/lib/rake-pipeline/railtie.rb +34 -0
data/lib/rake-pipeline/reject_matcher.rb +29 -0
data/lib/rake-pipeline/server.rb +15 -0
data/lib/rake-pipeline/sorted_pipeline.rb +19 -0
data/lib/rake-pipeline/version.rb +6 -0
data/rails/init.rb +2 -0
data/rake-pipeline.gemspec +24 -0
data/spec/cli_spec.rb +71 -0
data/spec/concat_filter_spec.rb +37 -0
data/spec/dsl/pipeline_dsl_spec.rb +165 -0
data/spec/dsl/project_dsl_spec.rb +41 -0
data/spec/dynamic_file_task_spec.rb +119 -0
data/spec/encoding_spec.rb +106 -0
data/spec/file_wrapper_spec.rb +132 -0
data/spec/filter_spec.rb +332 -0
data/spec/graph_spec.rb +56 -0
data/spec/gsub_filter_spec.rb +87 -0
data/spec/manifest_entry_spec.rb +46 -0
data/spec/manifest_spec.rb +67 -0
data/spec/matcher_spec.rb +141 -0
data/spec/middleware_spec.rb +199 -0
data/spec/ordering_concat_filter_spec.rb +42 -0
data/spec/pipeline_spec.rb +232 -0
data/spec/project_spec.rb +295 -0
data/spec/rake_acceptance_spec.rb +738 -0
data/spec/rake_tasks_spec.rb +21 -0
data/spec/reject_matcher_spec.rb +31 -0
data/spec/sorted_pipeline_spec.rb +27 -0
data/spec/spec_helper.rb +38 -0
data/spec/support/spec_helpers/file_utils.rb +35 -0
data/spec/support/spec_helpers/filters.rb +37 -0
data/spec/support/spec_helpers/input_helpers.rb +23 -0
data/spec/support/spec_helpers/memory_file_wrapper.rb +31 -0
data/spec/support/spec_helpers/memory_manifest.rb +19 -0
data/tools/perfs +101 -0
metadata +215 -0

checksums.yaml ADDED

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    NGIzYWYzYzBiN2VjZDI5ZTAyMjMxMDYxNGI4YTQ3OTgyZGI3OGQ3OQ==
+  data.tar.gz: !binary |-
+    M2RiMzExYWU2YzE0NTc4M2UxYzQ2OGI3OGE4ZDY5ZTgyN2NkOTZhZQ==
+SHA512:
+  metadata.gz: !binary |-
+    NjY5OGRlOThhOTkyNjliM2E3YjM4MmMxNmU4ZWQ3NmE5MDRjNmIzOThkZWMx
+    ZWZkNDEzZTRkMWI1MDRkZGYyNmEyZWYxODg2MTc5NWE4YzM5YjRkNmRjMGZm
+    YjUyZGMxY2U3NDQwNDM4ZjhlYThkY2E0MjBhNmVjMzkwZmU5NmU=
+  data.tar.gz: !binary |-
+    M2RlM2ZkODQwMDNjYjE1MmU1ZGE0YjkxODkwZTExMmFkOTFiN2RjYTAzNTk2
+    MDlhY2M2ZDAwNzhlODFhNjIxOTY1MGQ0ZDRiNDliZTViYTAxNzQ2Y2I5ZDAx
+    OTk1Nzk5ZWE5ZmM2NDlmZWJjNjc3NTQxZGQwZDQ4YjExZDJkMWU=

data/.gitignore ADDED

@@ -0,0 +1,18 @@
+*.gem
+*.rbx
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+devbin

data/.rspec ADDED

	@@ -0,0 +1 @@
1	+ -c -f progress --order random -r spec_helper.rb

data/.travis.yml ADDED

@@ -0,0 +1,12 @@
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - ruby-head
+  - jruby-head
+bundler_args: --without docs
+notifications:
+  email:
+    - wycats@gmail.com
+    - dudley@steambone.org

data/.yardopts ADDED

	@@ -0,0 +1,2 @@
1	+ --readme README.yard
2	+

data/GETTING_STARTED.md ADDED

@@ -0,0 +1,268 @@
+# Rake::Pipeline Basics
+`Rake::Pipeline` provides a basic extraction over a build process. It
+doesn't have many filters out of the box, but it is very powerful. This
+guide gives you an introduction to `Rake::Pipeline`. `Rake::Pipeline`
+was originally designed for frontend development (HTML, CSS,
+Javascript). The examples assume this type of project just to create
+some context.
+## Getting Started
+`Rake::Pipeline` comes with two main functionalities out of the box:
+matching and concatentation. You can specify a match (IE: all css files)
+then run them through a concatenation filter (combine them into one
+file). There is also an order concatenation filter which allows you to
+specify order (A should come before B).
+Your pipeline is written in an `Assetfile`. The `Assetfile` uses a nice
+DSL to make things easier. The `Assetfile` should live in your project's
+root directory.
+Let's get started by writing a basic pipeline. Assume we have this
+directory structure:
+```
+/source
+  /css
+  /javascript
+/compiled
+Assetfile
+```
+The pipeline should simply concatenate all the individual JSS and CSS
+files into single files. So the JSS and CSS directories are the inputs
+and 2 files are outputs. The `source` directory is input and the output
+will go into `compiled`. Here's the `Assetfile` to do just that:
+```ruby
+# Set the Pipeline's output directory
+output "compiled"
+# input defines operations on a set of files. All files processed in
+# this block go into the output directory
+input "source" do
+  # Select all the CSS files
+  match "css/**/*.css" do
+    # concatenate all files in directory into a file named
+    # "application.css"
+    concat "application.css"
+  end
+  # Repeat for javascript files
+  match "javascript/**/*.js" do
+    concat "application.js"
+  end
+end
+```
+Now run `rakep build` from the project root to compile everything.
+Given there are files in `source/css` and `source/javascript` you will
+see files in `compiled` named `application.js` and `application.css`.
+You've just written your first pipeline!
+## Previewing Your Work
+`Rake::Pipeline` also comes with a bundled preview server. Let's add an
+HTML file to the source directory to serve the compiled site. Here's the
+HTML:
+```html
+<!-- source/index.html -->
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="utf-8">
+  <title>Rake::Pipeline Example</title>
+  <link rel="stylesheet" href="/application.css">
+</head>
+<body>
+  <script src="/application.js"></script>
+</body>
+</html>
+```
+Save that file in `source/index.html`. Now we must add some more code to
+the `Assetfile` to copy the HTML file into the output directory.
+`Rake::Pipeline` also bundles a copy filter. Note that this isn't
+actually a separate filter, but a file that concatenates itself to a
+different location. In short, `concat` is aliased to `copy` as well.
+Here's the updated `Assetfile`:
+```ruby
+# Set the Pipeline's output directory
+output "compiled"
+# input defines operations on a set of files. All files processed in
+# this block go into the output directory
+input "source" do
+  # Select all the CSS files
+  match "css/**/*.css" do
+    # concatenate all files in directory into a file named
+    # "application.css"
+    concat "application.css"
+  end
+  # Repeat for javascript files
+  match "javascript/**/*.js" do
+    concat "application.js"
+  end
+  # Explicitly select the HTML file. We don't want to copy
+  # over anything else
+  match "index.html" do
+    # copy also accepts a block. When called without any arguments it
+    # simply uses the same filename
+    copy
+  end
+end
+```
+Now we can run `rakep server` inside the projects root. You'll see some
+output in the terminal with a URL to connect to. Now you an preview your
+work as you go.
+## Writing Filters
+It's very likely that you'll need to do more than just copy and
+concatenate files. You must write your own filters to do this. Luckily,
+writing filters is pretty easy. Filters are classes that have
+`generate_output` method. This is core API requirement. There also other
+method you may implement, but this is the most important. Let's take a
+stab at writing a coffeescript filter.
+Filters are Ruby classes. They map a set of inputs to outputs and
+finally generate the output. Here is an absolute bare bones filter:
+```ruby
+# Inherit from Rake::Pipeline::Filter to get basic API implemented
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+  # This method takes the input files and does whatever is required
+  # to generate the proper output.
+  def generate_output(inputs, output)
+    inputs.each do |input|
+      output.write input.read
+    end
+  end
+end
+```
+Notice `generate_output`'s method signature: `inputs` and `output`. The
+default semantic is to take N input files and map them to one output
+file. You can overide this or do much more fancy things. This is covered
+in a different file. We could use this filter like this:
+```ruby
+output "compiled"
+input "source" do
+  match "**/*.coffee" do
+    filter CoffeeScript
+  end
+end
+```
+Now this filter doesn't do anything at the moment besides copy the
+files. Time to implement coffeescript compiling.
+```ruby
+require "coffee-script"
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+  def generate_output(inputs, output)
+    inputs.each do |input|
+      output.write CoffeeScript.compile(input.read)
+    end
+  end
+end
+```
+Great! Now the CoffeeScript files are compiled to JavaScript. However,
+you may have noticed they are compiled "in place". This means
+`source/app.coffee` will become `source/app.coffee` but as JavaScript.
+This works in our simple example, but what happens when we need to work
+with Javascript later in the pipeline or next build steps expect ".js"
+files? The filter has to customize the name. The most correct thing to
+do is make the output file has the same name except as ".js".
+This behavior is defined in the filter's initializer. This may seem odd
+to you. It was odd to me until I understood what was happening.
+`Rake::Pipeline` instantiates all the filters in order to setup how
+input files map to output files before the pipeline is compiled. This is
+how one filter can use another's outputs for inputs. This order must be
+known at compile time, so that's why it happens here. The internal API
+expects a block that takes a path and maps it to an output path.
+```ruby
+require "coffee-script"
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+  def initialize(&block)
+    &block ||= proc { |input| input.path.gsub(/\.coffee/, ".js")
+    super(&block)
+  end
+  def generate_output(inputs, output)
+    inputs.each do |input|
+      output.write CoffeeScript.compile(input.read)
+    end
+  end
+end
+```
+Let's take a fresh look at the `Assetfile` now.
+```ruby
+output "compiled"
+input "source" do
+  match "javascript/**/*.coffee" do
+    # Compile all .coffee files into.js
+    filter CoffeeScript
+  end
+  # Select the JS generated by previous filters.
+  match "javascript/**/*.js" do
+    concat "application.js"
+  end
+  match "css/**/*.css" do
+    concat "application.css"
+  end
+  match "index.html" do
+    copy
+  end
+end
+```
+Calling the filter without a block uses the default block in the filter.
+The default block that replaces ".js" with ".coffee". This is defined
+with `||=` in the initializer. Conversely you could call `filter` with a
+block and do what you want. Here's an example:
+```ruby
+# output all coffeescript files as "app.coffee.awesome"
+filter CoffeeScript do |input|
+  "#{input.path}.awesome"
+end
+```
+That covers the basics of writing filters. There is much more you can do
+with filters that are outside the scope of this guide. You can find many
+useful (as well as plenty of examples) in the
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters)
+project.
+That also concludes this guide. You should know everything you need to
+know to get started writing your own pipelines now. There is still much
+to cover though. You can find additonal information in the `examples`
+directory. If you'd like to add anything to this guide or find an error
+please open a pull request to fix it.

data/Gemfile ADDED

@@ -0,0 +1,14 @@
+source 'http://rubygems.org'
+# Specify your gem's dependencies in rake-pipeline.gemspec
+gemspec
+gem "flay"
+gem "flog"
+gem "simplecov", :require => false
+gem "pry"
+group :docs do
+  gem "yard"
+  gem "rdiscount"
+end

data/LICENSE ADDED

@@ -0,0 +1,20 @@
+Copyright (C) 2011 by LivingSocial, Inc.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.markdown ADDED

@@ -0,0 +1,11 @@
+# Rake::Pipeline
+The canonical documentation for Rake::Pipeline is hosted at
+<a href="http://rubydoc.info/github/livingsocial/rake-pipeline/master/file/README.yard">rubydoc.info</a>.
+New users are recommended to read `GETTING_STARTED.md` before anything
+else. Additional examples can be found in the `examples` directory.
+Users are also recommended to checkout
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters)
+for commonly used filters.

data/README.yard ADDED

@@ -0,0 +1,178 @@
+= Rake::Pipeline
+Rake::Pipeline is a system for packaging assets for deployment to the
+web. It uses Rake under the hood for dependency management and updating
+output files based on input changes.
+= Usage
+The easiest way to use Rake::Pipeline is via an +Assetfile+ file in the
+root of your project.
+A sample +Assetfile+ looks like this:
+  !!!ruby
+  output "public"
+  input "assets" do
+    # this block will take all JS inputs, wrap them in a closure,
+    # add some additional metadata, and concatenate them all into
+    # application.scripts.js.
+    match "*.js" do
+      filter ClosureWrapper
+      filter DataWrapper
+      concat "application.scripts.js"
+    end
+    # this block will take all HTML and CSS inputs, convert them
+    # into JavaScript
+    match "*/*.{html,css}" do
+      filter DataWrapper
+      concat "application.assets.js"
+    end
+    match "*.js" do
+      concat "application.js"
+    end
+  end
+Each +input+ block defines a collection of files, and a pipeline
+that transforms those files. Within each pipeline, you can specify
+a series of filters to describe the transformations you'd like to
+apply to the files.
+= Upgrading from Previous Versions
+The +Assetfile+ syntax has changed in version 0.6.0. In previous
+versions, each +Assetfile+ defined a single pipeline, and +input+
+statements would add input files to that pipeline. After version
+0.6.0, multiple pipelines can be defined in an +Assetfile+. The
++input+ method now takes a block, and this block defines a pipeline.
+This means that any +match+ blocks or filters must be defined
+inside an +input+ block, and no longer at the top level. For example,
+this:
+    !!!ruby
+    # Prior to 0.6.0
+    output "public"
+    input "assets"
+    match "**/*.js" do
+      concat
+    end
+would now be written as:
+    !!!ruby
+    # After 0.6.0
+    output "public"
+    input "assets" do
+      match "**/*.js" do
+        concat
+      end
+    end
+= Filters
+A filter is a simple class that inherits from
+{Rake::Pipeline::Filter}. A filter must implement a single
+method, called +generate_output+, which takes
+two parameters: a list of input files and the output file.
+Both the input and output files are {Rake::Pipeline::FileWrapper} objects.
+The most important methods on a {Rake::Pipeline::FileWrapper FileWrapper} are:
+* {Rake::Pipeline::FileWrapper#path path}: the path of the file, relative to its input root
+* {Rake::Pipeline::FileWrapper#read read}: read the contents of the file
+* {Rake::Pipeline::FileWrapper#write write(string)}: write a String to the file
+For example, a simple concatenation filter would look like:
+  !!!ruby
+  class ConcatFilter < Rake::Pipeline::Filter
+    def generate_output(inputs, output)
+      inputs.each do |input|
+        output.write input.read
+      end
+    end
+  end
+If you had a series of input files like:
+* +app/javascripts/one.js+
+* +app/javascripts/two.js+
+* +app/javascripts/three.js+
+and you specified the +ConcatFilter+ in your
++Assetfile+ like:
+  !!!ruby
+  filter ConcatFilter, "application.js"
+The filter would receive a single call to
++generate_output+ with an Array of {Rake::Pipeline::FileWrapper FileWrapper}s
+representing each of the three files, and a {Rake::Pipeline::FileWrapper FileWrapper}
+representing +application.js+.
+== Binary Data
+If your filter is operating on binary data, like images,
+rather than textual data, like source code, you can specify
+that in your filter:
+  !!!ruby
+  class ConcatFilter < Rake::Pipeline::Filter
+    processes_binary_files
+    def generate_output(inputs, output)
+      inputs.each do |input|
+        output.write input.read
+      end
+    end
+  end
+This will stop `Rake::Pipeline` from trying to interpret the
+input files as `UTF-8`, which obviously will not work on
+binary data.
+= Filters
++Rake::Pipeline+ comes with a built-in filter,
+{Rake::Pipeline::ConcatFilter}. Its implementation is the same as the
++ConcatFilter+ above. Other filters that are useful for web development
+like a +CoffeeScriptFilter+ and +SassFilter+ are available in
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters).
+= Preview Server
+To start up the preview server, run +rakep server+. This will start up
+a server that automatically recompiles files for you on the fly
+and serves up the files you need.
+This should allow you to have a single index.html file pointing
+at the same files in both development and production.
+= Compiling Assets
+To compile all assets before deployment, simply run:
+  $ rakep build
+= Encodings
+If a filter does not specify that it processes binary files,
++Rake::Pipeline+ will open all inputs and outputs as +UTF-8+.
+This means that if you have files encoded in other encodings,
+like +Latin-1+, +Rake::Pipeline+ will raise an exception. In
+this situation, you need to open the offending file in your
+text editor and re-save it as +UTF-8+.
+= Public Release Requirement
+Before publicly releasing this code, we need to properly support
+encodings other than UTF-8. That means using the
++default_external+ instead of hardcoding to UTF-8 and
+providing a mechanism for specifying the encoding of a file using
+a magic comment.