loofah 1.0.0 → 2.19.1

data/README.rdoc

@@ -1,312 +0,0 @@
-= Loofah
-
-* http://github.com/flavorjones/loofah
-* http://loofah.rubyforge.org
-* http://rubyforge.org/projects/loofah
-
-== Description
-
-Loofah is a general library for manipulating and transforming HTML/XML
-documents and fragments. It's built on top of Nokogiri and libxml2, so
-it's fast and has a nice API.
-
-Loofah excels at HTML sanitization (XSS prevention). It includes some
-nice HTML sanitizers, which are based on HTML5lib's whitelist, so it
-most likely won't make your codes less secure. (These statements have
-not been evaluated by Netexperts.)
-
-ActiveRecord extensions for sanitization are available in the
-`loofah-activerecord` gem (see
-http://github.com/flavorjones/loofah-activerecord).
-
-== Features
-
-* Easily write custom scrubbers for HTML/XML leveraging the sweetness of Nokogiri (and HTML5lib's whitelists).
-* Common HTML sanitizing tasks are built-in:
-  * _Strip_ unsafe tags, leaving behind only the inner text.
-  * _Prune_ unsafe tags and their subtrees, removing all traces that they ever existed.
-  * _Escape_ unsafe tags and their subtrees, leaving behind lots of <tt>&lt;</tt> and <tt>&gt;</tt> entities.
-  * _Whitewash_ the markup, removing all attributes and namespaced nodes.
-* Common HTML transformation tasks are built-in:
-  * Add the _nofollow_ attribute to all hyperlinks.
-* Format markup as plain text, with or without sensible whitespace handling around block elements.
-* Replace Rails's +strip_tags+ and +sanitize+ view helper methods.
-
-== Compare and Contrast
-
-Loofah is one of two known Ruby XSS/sanitization solutions that
-guarantees well-formed and valid markup (the other is Sanitize, which
-also uses Nokogiri).
-
-Loofah works on XML, XHTML and HTML documents.
-
-Also, it's pretty fast. Here is a benchmark comparing Loofah to other
-commonly-used libraries (ActionView, Sanitize, HTML5lib and HTMLfilter):
-
-* http://gist.github.com/170193
-
-Lastly, Loofah is extensible. It's super-easy to write your own custom
-scrubbers for whatever document manipulation you need. You don't like
-the built-in scrubbers? Build your own, like a boss.
-
-== The Basics
-
-Loofah wraps Nokogiri[http://nokogiri.org] in a loving
-embrace. Nokogiri[http://nokogiri.org] is an excellent HTML/XML
-parser. If you don't know how Nokogiri[http://nokogiri.org] works, you
-might want to pause for a moment and go check it out. I'll wait.
-
-Loofah presents the following classes:
-
-* Loofah::HTML::Document and Loofah::HTML::DocumentFragment
-* Loofah::XML::Document and Loofah::XML::DocumentFragment
-* Loofah::Scrubber
-
-The documents and fragments are subclasses of the similar Nokogiri classes.
-
-The Scrubber represents the document manipulation, either by wrapping
-a block,
-
-  span2div = Loofah::Scrubber.new do |node|
-    node.name = "div" if node.name == "span"
-  end
-
-or by implementing a method.
-
-=== Side Note: Fragments vs Documents
-
-Generally speaking, unless you expect to have a DOCTYPE and a single
-root node, you don't have a *document*, you have a *fragment*. For
-HTML, another rule of thumb is that *documents* have \&lt;html\&gt;
-and \&lt;body\&gt; tags, and *fragments* usually do not.
-
-HTML fragments should be parsed with Loofah.fragment. The result won't
-be wrapped in +html+ or +body+ tags, won't have a DOCTYPE declaration,
-+head+ elements will be silently ignored, and multiple root nodes are
-allowed.
-
-XML fragments should be parsed with Loofah.xml_fragment. The result
-won't have a DOCTYPE declaration, and multiple root nodes are allowed.
-
-HTML documents should be parsed with Loofah.document. The result will
-have a DOCTYPE declaration, along with +html+, +head+ and +body+ tags.
-
-XML documents should be parsed with Loofah.xml_document. The result
-will have a DOCTYPE declaration and a single root node.
-
-=== Loofah::HTML::Document and Loofah::HTML::DocumentFragment
-
-These classes are subclasses of Nokogiri::HTML::Document and
-Nokogiri::HTML::DocumentFragment, so you get all the markup
-fixer-uppery and API goodness of Nokogiri.
-
-The module methods Loofah.document and Loofah.fragment will parse an
-HTML document and an HTML fragment, respectively.
-
-  Loofah.document(unsafe_html).is_a?(Nokogiri::HTML::Document)         # => true
-  Loofah.fragment(unsafe_html).is_a?(Nokogiri::HTML::DocumentFragment) # => true
-
-Loofah injects a +scrub!+ method, which takes either a symbol (for
-built-in scrubbers) or a Loofah::Scrubber object (for custom
-scrubbers), and modifies the document in-place.
-
-Loofah overrides +to_s+ to return HTML:
-
-  unsafe_html = "ohai! <div>div is safe</div> <script>but script is not</script>"
-
-  doc = Loofah.fragment(unsafe_html).scrub!(:strip)
-  doc.to_s    # => "ohai! <div>div is safe</div> "
-
-and +text+ to return plain text:
-
-  doc.text    # => "ohai! div is safe "
-
-Also, +to_text+ is available, which does the right thing with
-whitespace around block-level elements.
-
-  doc = Loofah.fragment("<h1>Title</h1><div>Content</div>")
-  doc.text    # => "TitleContent"           # probably not what you want
-  doc.to_text # => "\nTitle\n\nContent\n"   # better
-
-=== Loofah::XML::Document and Loofah::XML::DocumentFragment
-
-These classes are subclasses of Nokogiri::XML::Document and
-Nokogiri::XML::DocumentFragment, so you get all the markup
-fixer-uppery and API goodness of Nokogiri.
-
-The module methods Loofah.xml_document and Loofah.xml_fragment will
-parse an XML document and an XML fragment, respectively.
-
-  Loofah.xml_document(bad_xml).is_a?(Nokogiri::XML::Document)         # => true
-  Loofah.xml_fragment(bad_xml).is_a?(Nokogiri::XML::DocumentFragment) # => true
-
-=== Nodes and NodeSets
-
-Nokogiri::XML::Node and Nokogiri::XML::NodeSet also get a +scrub!+
-method, which makes it easy to scrub subtrees.
-
-The following code will apply the +employee_scrubber+ only to the
-+employee+ nodes (and their subtrees) in the document:
-
-  Loofah.xml_document(bad_xml).xpath("//employee").scrub!(employee_scrubber)
-
-And this code will only scrub the first +employee+ node and its subtree:
-
-  Loofah.xml_document(bad_xml).at_xpath("//employee").scrub!(employee_scrubber)
-
-=== Loofah::Scrubber
-
-A Scrubber wraps up a block (or method) that is run on a document node:
-
-  # change all <span> tags to <div> tags
-  span2div = Loofah::Scrubber.new do |node|
-    node.name = "div" if node.name == "span"
-  end
-
-This can then be run on a document:
-
-  Loofah.fragment("<span>foo</span><p>bar</p>").scrub!(span2div).to_s
-  # => "<div>foo</div><p>bar</p>"
-
-Scrubbers can be run on a document in either a top-down traversal (the
-default) or bottom-up. Top-down scrubbers can optionally return
-Scrubber::STOP to terminate the traversal of a subtree. Read below and
-in the Loofah::Scrubber class for more detailed usage.
-
-Here's an XML example:
-
-  # remove all <employee> tags that have a "deceased" attribute set to true
-  bring_out_your_dead = Loofah::Scrubber.new do |node|
-    if node.name == "employee" and node["deceased"] == "true"
-      node.remove
-      Loofah::Scrubber::STOP # don't bother with the rest of the subtree
-    end
-  end
-  Loofah.xml_document(File.read('plague.xml')).scrub!(bring_out_your_dead)
-
-=== Built-In HTML Scrubbers
-
-Loofah comes with a set of sanitizing scrubbers that use HTML5lib's
-whitelist algorithm:
-
-  doc.scrub!(:strip)       # replaces unknown/unsafe tags with their inner text
-  doc.scrub!(:prune)       #  removes unknown/unsafe tags and their children
-  doc.scrub!(:escape)      #  escapes unknown/unsafe tags, like this: &lt;script&gt;
-  doc.scrub!(:whitewash)   #  removes unknown/unsafe/namespaced tags and their children,
-                           #          and strips all node attributes
-
-Loofah also comes with some common transformation tasks: 
-
-  doc.scrub!(:nofollow)    #     adds rel="nofollow" attribute to links
-
-See Loofah::Scrubbers for more details and example usage.
-
-=== Chaining Scrubbers
-
-You can chain scrubbers:
-
-  Loofah.fragment("<span>hello</span> <script>alert('OHAI')</script>") \
-        .scrub!(:prune) \
-        .scrub!(span2div).to_s
-  # => "<div>hello</div> "
-
-=== Shorthand
-
-The class methods Loofah.scrub_fragment and Loofah.scrub_document are
-shorthand.
-
-  Loofah.scrub_fragment(unsafe_html, :prune)
-  Loofah.scrub_document(unsafe_html, :prune)
-  Loofah.scrub_xml_fragment(bad_xml, custom_scrubber)
-  Loofah.scrub_xml_document(bad_xml, custom_scrubber)
-
-are the same thing as (and arguably semantically clearer than):
-
-  Loofah.fragment(unsafe_html).scrub!(:prune)
-  Loofah.document(unsafe_html).scrub!(:prune)
-  Loofah.xml_fragment(bad_xml).scrub!(custom_scrubber)
-  Loofah.xml_document(bad_xml).scrub!(custom_scrubber)
-
-=== View Helpers
-
-Loofah has two "view helpers": Loofah::Helpers.sanitize and
-Loofah::Helpers.strip_tags, both of which are drop-in replacements for
-the Rails ActionView helpers of the same name.
-
-== Requirements
-
-* Nokogiri >= 1.3.3
-
-== Installation
-
-Unsurprisingly:
-
-* gem install loofah
-
-== Support
-
-The bug tracker is available here:
-
-* http://github.com/flavorjones/loofah/issues
-
-And the mailing list is on librelist:
-
-* loofah@librelist.com / http://librelist.com
-
-And the IRC channel is \#loofah on freenode.
-
-== Related Links
-
-* Nokogiri: http://nokogiri.org
-* libxml2: http://xmlsoft.org
-* html5lib: http://code.google.com/p/html5lib
-
-== Authors
-
-* {Mike Dalessio}[http://mike.daless.io] (@flavorjones[http://twitter.com/flavorjones])
-* Bryan Helmkamp
-
-Featuring code contributed by:
-
-* Aaron Patterson
-* John Barnette
-* Josh Owens
-* Paul Dix
-* Luke Melia
-
-And a big shout-out to Corey Innis for the name, and feedback on the API.
-
-== Thank You
-
-The following people have generously donated via the Pledgie[http://pledgie.com] badge on the {Loofah github page}[http://github.com/flavorjones/loofah]:
-
-* Bill Harding
-
-== Historical Note
-
-This library was formerly known as Dryopteris, which was a very bad
-name that nobody could spell properly.
-
-== License
-
-The MIT License
-
-Copyright (c) 2009, 2010 by Mike Dalessio, Bryan Helmkamp
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.

data/Rakefile

@@ -1,53 +0,0 @@
-require 'rubygems'
-gem 'hoe', '>= 2.3.0'
-require 'hoe'
-
-Hoe.plugin :git
-Hoe.plugin :gemspec
-
-Hoe.spec "loofah" do
-  developer "Mike Dalessio", "mike.dalessio@gmail.com"
-  developer "Bryan Helmkamp", "bryan@brynary.com"
-
-  self.extra_rdoc_files = FileList["*.rdoc"]
-  self.history_file     = "CHANGELOG.rdoc"
-  self.readme_file      = "README.rdoc"
-
-  extra_deps     << ["nokogiri", ">=1.3.3"]
-  extra_dev_deps << ["mocha", ">=0.9"]
-  extra_dev_deps << ["shoulda", ">=2.10"]
-  extra_dev_deps << ["rake", ">=0.8"]
-end
-
-task :gemspec do
-  system %q(rake debug_gem | grep -v "^\(in " > loofah.gemspec)
-end
-
-task :redocs => :fix_css
-task :docs => :fix_css
-task :fix_css do
-  better_css = <<-EOT
-    .method-description pre {
-      margin                    : 1em 0 ;
-    }
-
-    .method-description ul {
-      padding                   : .5em 0 .5em 2em ;
-    }
-
-    .method-description p {
-      margin-top                : .5em ;
-    }
-
-    #main ul, div#documentation ul {
-      list-style-type           : disc ! IMPORTANT ;
-      list-style-position       : inside ! IMPORTANT ;
-    }
-
-    h2 + ul {
-      margin-top                : 1em;
-    }
-  EOT
-  puts "* fixing css"
-  File.open("doc/rdoc.css", "a") { |f| f.write better_css }
-end

data/benchmark/benchmark.rb

@@ -1,149 +0,0 @@
-#!/usr/bin/env ruby
-require "#{File.dirname(__FILE__)}/helper.rb"
-
-def compare_scrub_methods
-  snip = "<div>foo</div><foo>fuxx <b>quux</b></foo><script>i have a chair</script>"
-  puts "starting with:\n#{snip}"
-  puts
-  puts RailsSanitize.new.sanitize(snip) # => Rails.sanitize / scrub!(:prune).to_s
-  puts Loofah::Helpers.sanitize(snip)
-  puts "--"
-  puts RailsSanitize.new.strip_tags(snip) # => Rails.strip_tags / parse().text
-  puts Loofah::Helpers.strip_tags(snip)
-  puts "--"
-  puts Sanitize.clean(snip, Sanitize::Config::RELAXED) # => scrub!(:strip).to_s
-  puts Loofah.scrub_fragment(snip, :strip).to_s
-  puts "--"
-  puts HTML5libSanitize.new.sanitize(snip) # => scrub!(:escape).to_s
-  puts Loofah.scrub_fragment(snip, :escape).to_s
-  puts "--"
-  puts HTMLFilter.new.filter(snip)
-  puts Loofah.scrub_fragment(snip, :strip).to_s
-  puts
-end
-
-module TestSet
-  def test_set options={}
-    scale = options[:rehearse] ? 10 : 1
-    puts self.class.name
-
-    n = 100 / scale
-    puts "  Large document, #{BIG_FILE.length} bytes (x#{n})"
-    bench BIG_FILE, n, false
-    puts
-
-    n = 1000 / scale
-    puts "  Small fragment, #{FRAGMENT.length} bytes (x#{n})"
-    bench FRAGMENT, n, true
-    puts
-
-    n = 10_000 / scale
-    puts "  Text snippet, #{SNIPPET.length} bytes (x#{n})"
-    bench SNIPPET, n, true
-    puts
-  end
-end
-
-class HeadToHead < Measure
-end
-
-class HeadToHeadRailsSanitize < Measure
-  include TestSet
-  def bench(content, ntimes, fragment_p)
-    clear_measure
-
-    measure "Loofah::Helpers.sanitize", ntimes do
-      Loofah::Helpers.sanitize content
-    end
-
-    sanitizer = RailsSanitize.new
-    measure "ActionView sanitize", ntimes do
-      sanitizer.sanitize(content)
-    end
-  end
-end
-
-class HeadToHeadRailsStripTags < Measure
-  include TestSet
-  def bench(content, ntimes, fragment_p)
-    clear_measure
-
-    measure "Loofah::Helpers.strip_tags", ntimes do
-      Loofah::Helpers.strip_tags content
-    end
-
-    sanitizer = RailsSanitize.new
-    measure "ActionView strip_tags", ntimes do
-      sanitizer.strip_tags(content)
-    end
-  end
-end
-
-class HeadToHeadSanitizerSanitize < Measure
-  include TestSet
-  def bench(content, ntimes, fragment_p)
-    clear_measure
-
-    measure "Loofah :strip", ntimes do
-      if fragment_p
-        Loofah.scrub_fragment(content, :strip).to_s
-      else
-        Loofah.scrub_document(content, :strip).to_s
-      end
-    end
-
-    measure "Sanitize.clean", ntimes do
-      Sanitize.clean(content, Sanitize::Config::RELAXED)
-    end
-  end
-end
-
-class HeadToHeadHtml5LibSanitize < Measure
-  include TestSet
-  def bench(content, ntimes, fragment_p)
-    clear_measure
-
-    measure "Loofah :escape", ntimes do
-      if fragment_p
-        Loofah.scrub_fragment(content, :escape).to_s
-      else
-        Loofah.scrub_document(content, :escape).to_s
-      end
-    end
-
-    html5_sanitizer = HTML5libSanitize.new
-    measure "HTML5lib.sanitize", ntimes do
-      html5_sanitizer.sanitize(content)
-    end
-  end
-end
-
-class HeadToHeadHTMLFilter < Measure
-  include TestSet
-  def bench(content, ntimes, fragment_p)
-    clear_measure
-
-    measure "Loofah::Helpers.sanitize", ntimes do
-      Loofah::Helpers.sanitize content
-    end
-
-    sanitizer = HTMLFilter.new
-    measure "HTMLFilter.filter", ntimes do
-      sanitizer.filter(content)
-    end
-  end
-end
-
-puts "Nokogiri version: #{Nokogiri::VERSION_INFO.inspect}"
-puts "Loofah version: #{Loofah::VERSION.inspect}"
-
-benches = []
-benches << HeadToHeadRailsSanitize.new
-benches << HeadToHeadRailsStripTags.new
-benches << HeadToHeadSanitizerSanitize.new
-benches << HeadToHeadHtml5LibSanitize.new
-benches << HeadToHeadHTMLFilter.new
-puts "---------- rehearsal ----------"
-benches.each { |bench| bench.test_set :rehearse => true }
-puts "---------- realsies ----------"
-benches.each { |bench| bench.test_set }

data/benchmark/fragment.html

@@ -1,96 +0,0 @@
-<div id="top_parent"></div>
-
-<div id="jump">
-        <a href="#main-articles">Stories</a>
-        <br>
-        <a href="#blocks">Slash Boxes</a>
-        <br>
-        <a href="#comments">Comments</a>
-</div>
-<a name="topothepage"></a>
-<div id="doc3" class="yui-t6 index2 mainpage ac   ">
-	<div id="hd" >
-		<div id="logo" >
-
-			
-			
-			<h1><a href="//slashdot.org"><span>Slashdot</span></a></h1>
-			<div id="slogan"><h2>News for nerds, stuff that matters</h2></div>
-		</div>
-		<a href="#articles" class="hidden">Jump to articles</a>
-		<div class="nav">
-			<ul>
-			
-				
-                                
-				<li><a href="//slashdot.org/submit.pl" title="Submit a story to Slashdot">Submit Story</a></li>
-                                <li><a href="//slashdot.org/help" title="Frequently asked questions on Slashdot">Help</a></li>
-				<li><a href="//slashdot.org/login.pl" onclick="show_login_box(); return false;">Log In</a></li>
-			
-			</ul>
-		</div>
-	
-		
-
-		
-
-		<div id="fh_picker_search" style="display: block;">
-		<form method="get" action="//slashdot.org/index2.pl">
-			<fieldset class="mode-filter mode-anon">
-				<legend>Search</legend>
-				
-				
-				<input class="query" type="text" name="fhfilter" value="" id="searchquery">						<input type="button" class="setfhfilter" value="Filter" id="viewsearch" style="display:none">					<input type="submit" class="setsearchfilter" value="Search" id="fhsearch" style="display:none">
-					<noscript><input type="submit" class="setsearchfilter" value="Search"></noscript>
-					
-					<script type="text/javascript">
-					var slash_search;
-						$(function(){
-							if (has_hose()) {
-							var	$search_text	= $any('searchquery'),
-								$panel			= $search_text.closest('fieldset');
-								$search_buttons	= $('#viewsearch,#fhsearch'),
-								ws				= /\s+/;
-
-							
-
-							// The search buttons set the firehose option named by their class.
-							$search_buttons.
-								click(function(){
-									var which=this.className;
-									$search_text.each(function(){
-										firehose_set_options(which, this.value);
-									});
-									return false;
-								});
-
-							// Provide a globally available function that does whatever clicking the search button would do.
-							slash_search = function( query ){
-								query!==undefined && $search_text.val(query);
-								$search_buttons.filter(':visible:first').click();
-							};
-
-							$search_text.
-								keydown(function( e ){	// ESCAPE restores the filter in-effect.
-									if ( e.which == $.ui.keyCode.ESCAPE ) {
-										$search_text.val(firehose_settings.fhfilter||'');
-										return true;
-									}
-									if ( e.which == $.ui.keyCode.ENTER ) {
-										slash_search();
-										return false;
-									}
-								});
-	
-							$(document).
-								bind('firehose-setting-setfhfilter firehose-setting-setsearchfilter', function( e, new_query ){
-									$('fieldset input[type=text]').each(function(){
-										$(this).blur().val(new_query);
-									});
-								}).
-								bind('set-options.firehose', function( e, data ){
-									data.select_section && $panel.toggleClass('mode-filter', data.id!=='unsaved');
-								});
-						}
-					});
-					</script>

data/benchmark/helper.rb

@@ -1,73 +0,0 @@
-require 'rubygems'
-require 'open-uri'
-require 'hpricot'
-require File.expand_path(File.dirname(__FILE__) + "/../lib/loofah")
-require 'benchmark'
-require "action_view"
-require "action_controller/vendor/html-scanner"
-require "sanitize"
-require 'hitimes'
-require 'htmlfilter'
-
-unless defined?(HTMLFilter)
-  HTMLFilter = HtmlFilter
-end
-
-class RailsSanitize
-  include ActionView::Helpers::SanitizeHelper
-  extend ActionView::Helpers::SanitizeHelper::ClassMethods
-end
-
-class HTML5libSanitize
-  require 'html5/html5parser'
-  require 'html5/liberalxmlparser'
-  require 'html5/treewalkers'
-  require 'html5/treebuilders'
-  require 'html5/serializer'
-  require 'html5/sanitizer'
-
-  include HTML5
-
-  def sanitize(html)
-    HTMLParser.parse_fragment(html, {
-      :tokenizer  => HTMLSanitizer,
-      :encoding   => 'utf-8',
-      :tree       => TreeBuilders::REXML::TreeBuilder
-    }).to_s
-  end
-end
-
-BIG_FILE = File.read(File.join(File.dirname(__FILE__), "www.slashdot.com.html"))
-FRAGMENT = File.read(File.join(File.dirname(__FILE__), "fragment.html"))
-SNIPPET = "This is typical form field input in <b>length and content."
-
-class Measure
-  def initialize
-    clear_measure
-  end
-
-  def clear_measure
-    @first_time = true
-    @baseline = nil
-  end
-
-  def measure(name, ntimes)
-    if @first_time
-      printf "  %-30s %7s  %8s  %5s\n", "", "total", "single", "rel"
-      @first_time = false
-    end
-    timer = Hitimes::TimedMetric.new(name)
-    timer.start
-    ntimes.times do |j|
-      yield
-    end
-    timer.stop
-    if @baseline
-      printf "  %30s %7.3f (%8.6f) %5.2fx\n", timer.name, timer.sum, timer.sum / ntimes, timer.sum / @baseline
-    else
-      @baseline = timer.sum
-      printf "  %30s %7.3f (%8.6f) %5s\n", timer.name, timer.sum, timer.sum / ntimes, "-"
-    end
-    timer.sum
-  end
-end