sanitize 1.1.1.dev.20091102 → 1.2.0.dev.20091104

data/HISTORY

@@ -1,7 +1,10 @@
 Sanitize History
 ================================================================================
 
-Version 1.1.1.dev (git)
+Version 1.2.0.dev (git)
+  * Added support for transformers, which allow you to filter and alter nodes
+    using your own custom logic, on top of (or instead of) Sanitize's core
+    filter. See the README for details.
   * Requires Nokogiri >= 1.4.0.
   * Added elements h1 through h6 to the Relaxed whitelist. [Suggested by David
     Reese]

data/README.rdoc

@@ -15,7 +15,7 @@ or maliciously-formed HTML. When in doubt, Sanitize always errs on the side of
 caution.
 
 *Author*::    Ryan Grove (mailto:ryan@wonko.com)
-*Version*::   1.1.1.dev (git)
+*Version*::   1.2.0.dev (git)
 *Copyright*:: Copyright (c) 2009 Ryan Grove. All rights reserved.
 *License*::   MIT License (http://opensource.org/licenses/mit-license.php)
 *Website*::   http://github.com/rgrove/sanitize
@@ -38,7 +38,8 @@ Latest development version:
 == Usage
 
 If you don't specify any configuration options, Sanitize will use its strictest
-settings by default, which means it will strip all HTML.
+settings by default, which means it will strip all HTML and leave only text
+behind.
 
   require 'rubygems'
   require 'sanitize'
@@ -145,6 +146,72 @@ include the symbol <code>:relative</code> in the protocol array:
     'a' => {'href' => ['http', 'https', :relative]}
   }
 
+=== Transformers
+
+Transformers allow you to filter and alter nodes using your own custom logic, on
+top of (or instead of) Sanitize's core filter. A transformer is any object that
+responds to <code>call()</code> (such as a lambda or proc) and returns either
+<code>nil</code> or a Hash containing certain optional response values.
+
+To use one or more transformers, pass them to the <code>:transformers</code>
+config setting:
+
+  Sanitize.clean(html, :transformers => [transformer_one, transformer_two])
+
+==== Input
+
+Each registered transformer's <code>call()</code> method will be called once for
+each element node in the HTML, and will receive as an argument an environment
+Hash that contains the following items:
+
+[<code>:config</code>]
+  The current Sanitize configuration Hash.
+
+[<code>:node</code>]
+  A Nokogiri::XML::Node object representing an HTML element.
+
+==== Processing
+
+Each transformer has full access to the Nokogiri::XML::Node that's passed into
+it and to the rest of the document via the node's <code>document()</code>
+method. Any changes will be reflected instantly in the document and passed on to
+subsequently-called transformers and to Sanitize itself. A transformer may even
+call Sanitize internally to perform custom sanitization if needed.
+
+Nodes are passed into transformers in the order in which they're traversed. It's
+important to note that Nokogiri traverses markup from the deepest node upward,
+not from the first node to the last node:
+
+  html        = '<div><span>foo</span></div>'
+  transformer = lambda{|env| puts env[:node].name }
+
+  # Prints "span", then "div".
+  Sanitize.clean(html, :transformers => transformer)
+
+Transformers have a tremendous amount of power, including the power to
+completely bypass Sanitize's built-in filtering. Be careful!
+
+==== Output
+
+A transformer may return either +nil+ or a Hash. A return value of +nil+
+indicates that the transformer does not wish to act on the current node in any
+way. A returned Hash may contain the following items, all of which are optional:
+
+[<code>:attr_whitelist</code>]
+  Array of attribute names to add to the whitelist for the current node, in
+  addition to any whitelisted attributes already defined in the current config.
+
+[<code>:node</code>]
+  A Nokogiri::XML::Node object that should replace the current node. All
+  subsequent transformers and Sanitize itself will receive this new node.
+
+[<code>:whitelist</code>]
+  If _true_, the current node (and only the current node) will be whitelisted,
+  regardless of the current Sanitize config.
+
+[<code>:whitelist_nodes</code>]
+  Array of specific Nokogiri::XML::Node objects to whitelist, anywhere in the
+  document, regardless of the current Sanitize config.
 
 == Contributors
 

data/lib/sanitize/config/relaxed.rb

@@ -20,6 +20,8 @@
 # SOFTWARE.
 #++
 
+
+
 class Sanitize
   module Config
     RELAXED = {

data/lib/sanitize/config.rb

@@ -47,7 +47,9 @@ class Sanitize
       # URL handling protocols to allow in specific attributes. By default, no
       # protocols are allowed. Use :relative in place of a protocol if you want
       # to allow relative URLs sans protocol.
-      :protocols => {}
+      :protocols => {},
+
+      :transformers => []
     }
   end
 end

data/lib/sanitize/version.rb

@@ -1,3 +1,3 @@
 class Sanitize
-  VERSION = '1.1.1.dev.20091102'
+  VERSION = '1.2.0.dev.20091104'
 end

data/lib/sanitize.rb

@@ -29,6 +29,7 @@ require 'sanitize/config/basic'
 require 'sanitize/config/relaxed'
 
 class Sanitize
+  attr_reader :config
 
   # Matches an attribute value that could be treated by a browser as a URL
   # with a protocol prefix, such as "http:" or "javascript:". Any string of zero
@@ -37,13 +38,44 @@ class Sanitize
   # IE6 and Opera will still parse).
   REGEX_PROTOCOL = /^([A-Za-z0-9\+\-\.\&\;\#\s]*?)(?:\:|&#0*58|&#x0*3a)/i
 
+  #--
+  # Class Methods
+  #++
+
+  # Returns a sanitized copy of _html_, using the settings in _config_ if
+  # specified.
+  def self.clean(html, config = {})
+    sanitize = Sanitize.new(config)
+    sanitize.clean(html)
+  end
+
+  # Performs Sanitize#clean in place, returning _html_, or +nil+ if no changes
+  # were made.
+  def self.clean!(html, config = {})
+    sanitize = Sanitize.new(config)
+    sanitize.clean!(html)
+  end
+
+  # Sanitizes the specified Nokogiri::XML::Node and all its children.
+  def self.clean_node!(node, config = {})
+    sanitize = Sanitize.new(config)
+    sanitize.clean_node!(node)
+  end
+
   #--
   # Instance Methods
   #++
 
   # Returns a new Sanitize object initialized with the settings in _config_.
   def initialize(config = {})
+    # Sanitize configuration.
     @config = Config::DEFAULT.merge(config)
+    @config[:transformers] = Array(@config[:transformers])
+
+    # Specific nodes to whitelist (along with all their attributes). This array
+    # is generated at runtime by transformers, and is cleared before and after
+    # a fragment is cleaned (so it applies only to a specific fragment).
+    @whitelist_nodes = []
   end
 
   # Returns a sanitized copy of _html_.
@@ -55,71 +87,16 @@ class Sanitize
   # Performs clean in place, returning _html_, or +nil+ if no changes were
   # made.
   def clean!(html)
+    @whitelist_nodes = []
     fragment = Nokogiri::HTML::DocumentFragment.parse(html)
+    clean_node!(fragment)
+    @whitelist_nodes = []
 
-    fragment.traverse do |node|
-      if node.comment?
-        node.unlink unless @config[:allow_comments]
-      elsif node.element?
-        name = node.name.to_s.downcase
-
-        # Delete any element that isn't in the whitelist.
-        unless @config[:elements].include?(name)
-          node.children.each { |n| node.add_previous_sibling(n) }
-          node.unlink
-          next
-        end
-
-        attr_whitelist = ((@config[:attributes][name] || []) +
-            (@config[:attributes][:all] || [])).uniq
-
-        if attr_whitelist.empty?
-          # Delete all attributes from elements with no whitelisted
-          # attributes.
-          node.attribute_nodes.each { |attr| attr.remove }
-        else
-          # Delete any attribute that isn't in the whitelist for this element.
-          node.attribute_nodes.each do |attr|
-            attr.unlink unless attr_whitelist.include?(attr.name.downcase)
-          end
-
-          # Delete remaining attributes that use unacceptable protocols.
-          if @config[:protocols].has_key?(name)
-            protocol = @config[:protocols][name]
-
-            node.attribute_nodes.each do |attr|
-              attr_name = attr.name.downcase
-              next false unless protocol.has_key?(attr_name)
-
-              del = if attr.value.to_s.downcase =~ REGEX_PROTOCOL
-                !protocol[attr_name].include?($1.downcase)
-              else
-                !protocol[attr_name].include?(:relative)
-              end
-
-              attr.unlink if del
-            end
-          end
-        end
-
-        # Add required attributes.
-        if @config[:add_attributes].has_key?(name)
-          @config[:add_attributes][name].each do |key, val|
-            node[key] = val
-          end
-        end
-      elsif node.cdata?
-        node.replace(Nokogiri::XML::Text.new(node.text, node.document))
-      end
-    end
-
-    # Nokogiri 1.3.3 (and possibly earlier versions) always returns a US-ASCII
-    # string no matter what we ask for. This will be fixed in 1.4.0, but for
-    # now we have to hack around it to prevent errors.
     output_method_params = {:encoding => 'utf-8', :indent => 0}
+
     if @config[:output] == :xhtml
       output_method = fragment.method(:to_xhtml)
-      output_method_params.merge!(:save_with => Nokogiri::XML::Node::SaveOptions::AS_XHTML)
+      output_method_params[:save_with] = Nokogiri::XML::Node::SaveOptions::AS_XHTML
     elsif @config[:output] == :html
       output_method = fragment.method(:to_html)
     else
@@ -127,29 +104,125 @@ class Sanitize
     end
 
     result = output_method.call(output_method_params)
+
+    # Nokogiri 1.3.3 (and possibly earlier versions) always returns a US-ASCII
+    # string no matter what we ask for. This will be fixed in 1.4.0, but for
+    # now we have to hack around it to prevent errors.
     result.force_encoding('utf-8') if RUBY_VERSION >= '1.9'
 
     return result == html ? nil : html[0, html.length] = result
   end
 
-  #--
-  # Class Methods
-  #++
+  # Sanitizes the specified Nokogiri::XML::Node and all its children.
+  def clean_node!(node)
+    raise ArgumentError unless node.is_a?(Nokogiri::XML::Node)
+
+    node.traverse do |traversed_node|
+      if traversed_node.element?
+        clean_element!(traversed_node)
+      elsif traversed_node.comment?
+        traversed_node.unlink unless @config[:allow_comments]
+      elsif traversed_node.cdata?
+        traversed_node.replace(Nokogiri::XML::Text.new(traversed_node.text,
+            traversed_node.document))
+      end
+    end
+
+    node
+  end
+
+  private
 
-  class << self
-    # Returns a sanitized copy of _html_, using the settings in _config_ if
-    # specified.
-    def clean(html, config = {})
-      sanitize = Sanitize.new(config)
-      sanitize.clean(html)
+  def clean_element!(node)
+    # Run this node through all configured transformers.
+    transform = transform_element!(node)
+
+    # If this node is in the dynamic whitelist array (built at runtime by
+    # transformers), let it live with all of its attributes intact.
+    return if @whitelist_nodes.include?(node)
+
+    name = node.name.to_s.downcase
+
+    # Delete any element that isn't in the whitelist.
+    unless transform[:whitelist] || @config[:elements].include?(name)
+      node.children.each { |n| node.add_previous_sibling(n) }
+      node.unlink
+      return
     end
 
-    # Performs Sanitize#clean in place, returning _html_, or +nil+ if no changes
-    # were made.
-    def clean!(html, config = {})
-      sanitize = Sanitize.new(config)
-      sanitize.clean!(html)
+    attr_whitelist = (transform[:attr_whitelist] +
+        (@config[:attributes][name] || []) +
+        (@config[:attributes][:all] || [])).uniq
+
+    if attr_whitelist.empty?
+      # Delete all attributes from elements with no whitelisted attributes.
+      node.attribute_nodes.each {|attr| attr.remove }
+    else
+      # Delete any attribute that isn't in the whitelist for this element.
+      node.attribute_nodes.each do |attr|
+        attr.unlink unless attr_whitelist.include?(attr.name.downcase)
+      end
+
+      # Delete remaining attributes that use unacceptable protocols.
+      if @config[:protocols].has_key?(name)
+        protocol = @config[:protocols][name]
+
+        node.attribute_nodes.each do |attr|
+          attr_name = attr.name.downcase
+          next false unless protocol.has_key?(attr_name)
+
+          del = if attr.value.to_s.downcase =~ REGEX_PROTOCOL
+            !protocol[attr_name].include?($1.downcase)
+          else
+            !protocol[attr_name].include?(:relative)
+          end
+
+          attr.unlink if del
+        end
+      end
+    end
+
+    # Add required attributes.
+    if @config[:add_attributes].has_key?(name)
+      @config[:add_attributes][name].each do |key, val|
+        node[key] = val
+      end
     end
+
+    transform
   end
 
+  def transform_element!(node)
+    output = {
+      :attr_whitelist => [],
+      :node           => node,
+      :whitelist      => false
+    }
+
+    @config[:transformers].inject(node) do |transformer_node, transformer|
+      transform = transformer.call({
+        :config => @config,
+        :node   => transformer_node
+      })
+
+      if transform.nil?
+        transformer_node
+      elsif transform.is_a?(Hash)
+        if transform[:whitelist_nodes].is_a?(Array)
+          @whitelist_nodes += transform[:whitelist_nodes] 
+          @whitelist_nodes.uniq!
+        end
+
+        output[:attr_whitelist]  += transform[:attr_whitelist] if transform[:attr_whitelist].is_a?(Array)
+        output[:whitelist]      ||= true if transform[:whitelist]
+        output[:node]             = transform[:node].is_a?(Nokogiri::XML::Node) ? transform[:node] : output[:node]
+      else
+        raise Error, "transformer output must be a Hash or nil"
+      end
+    end
+
+    node.replace(output[:node]) if node != output[:node]
+
+    return output
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: sanitize
 version: !ruby/object:Gem::Version 
-  version: 1.1.1.dev.20091102
+  version: 1.2.0.dev.20091104
 platform: ruby
 authors: 
 - Ryan Grove
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-11-02 00:00:00 -08:00
+date: 2009-11-04 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency