text_rank 1.1.0

data/.ruby-version

@@ -0,0 +1 @@
+ruby-2.1.2

data/.travis.yml

@@ -0,0 +1,7 @@
+language: ruby
+rvm:
+  - 2.1.2
+before_install: gem install bundler -v 1.11.2
+addons:
+    code_climate:
+        repo_token: 6ab030bf370ffc2abbf0ba4d70a1c8d9649f6fd1426f48f6d43d5c9eb15f187f

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,49 @@
+# Contributor Code of Conduct
+
+As contributors and maintainers of this project, and in the interest of
+fostering an open and welcoming community, we pledge to respect all people who
+contribute through reporting issues, posting feature requests, updating
+documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free
+experience for everyone, regardless of level of experience, gender, gender
+identity and expression, sexual orientation, disability, personal appearance,
+body size, race, ethnicity, age, religion, or nationality.
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery
+* Personal attacks
+* Trolling or insulting/derogatory comments
+* Public or private harassment
+* Publishing other's private information, such as physical or electronic
+  addresses, without explicit permission
+* Other unethical or unprofessional conduct
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+By adopting this Code of Conduct, project maintainers commit themselves to
+fairly and consistently applying these principles to every aspect of managing
+this project. Project maintainers who do not follow or enforce the Code of
+Conduct may be permanently removed from the project team.
+
+This code of conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting a project maintainer at david@bloomfire.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. Maintainers are
+obligated to maintain confidentiality with regard to the reporter of an
+incident.
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 1.3.0, available at
+[http://contributor-covenant.org/version/1/3/0/][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/3/0/

data/Gemfile

@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 David McCullars
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,137 @@
+# TextRank
+
+* README:         https://github.com/david-mccullars/text_rank
+* Documentation:  http://www.rubydoc.info/github/david-mccullars/text_rank
+* Bug Reports:    https://github.com/david-mccullars/text_rank/issues
+
+
+## Status
+
+[![Travis Build Status](https://travis-ci.org/david-mccullars/text_rank.svg?branch=master)](https://travis-ci.org/david-mccullars/text_rank)
+[![Code Climate](https://codeclimate.com/github/david-mccullars/text_rank/badges/gpa.svg)](https://codeclimate.com/github/david-mccullars/text_rank)
+[![Test Coverage](https://codeclimate.com/github/david-mccullars/text_rank/badges/coverage.svg)](https://codeclimate.com/github/david-mccullars/text_rank/coverage)
+
+
+## Description
+
+[TextRank](https://web.eecs.umich.edu/~mihalcea/papers/mihalcea.emnlp04.pdf) is
+an unsupervised keyword extraction algorithm based on
+[PageRank](http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf).  Other
+strategies for keyword extraction generally rely on either statistics (like
+inverse document frequency and term frequency) which ignore context, or they
+rely on machine learning, requiring a corpus of training data which likely will
+not be suitable for all applications.  TextRank is found to produce superior
+results in many situations with minimal computational cost.
+
+
+## Features
+
+* Multiple PageRank implementations to choose one best suited for the performance
+needs of your application
+* Framework for adding additional PageRank implementations (e.g. a native
+implemenation)
+* Extensible architecture to customize how text is filtered
+* Extensible architecture to customize how text is tokenized
+* Extensible architecture to customize how tokens are filtered
+* Extensible architecture to customize how keywords ranks are filtered/processed
+
+
+## Installation
+
+```
+gem install text_rank
+```
+
+## Requirements
+
+* Ruby 2.1.2 or higher
+* [engtagger](https://github.com/yohasebe/engtagger) gem is optional but
+required for `TextRank::TokenFilter::PartOfSpeech`
+* [nokogiri](https://github.com/sparklemotion/nokogiri) gem is optional but
+required for `TextRank::CharFilter::StripHtml`
+
+## Usage
+
+**TextRank**
+
+```ruby
+require 'text_rank'
+
+text = <<-END
+  In a castle of Westphalia, belonging to the Baron of Thunder-ten-Tronckh, lived
+  a youth, whom nature had endowed with the most gentle manners. His countenance
+  was a true picture of his soul. He combined a true judgment with simplicity of
+  spirit, which was the reason, I apprehend, of his being called Candide. The old
+  servants of the family suspected him to have been the son of the Baron's
+  sister, by a good, honest gentleman of the neighborhood, whom that young lady
+  would never marry because he had been able to prove only seventy-one
+  quarterings, the rest of his genealogical tree having been lost through the
+  injuries of time.
+END
+
+# Default, basic keyword extraction.  Try this first:
+keywords = TextRank.extract_keywords(text)
+
+# Keyword extraction with all of the bells and whistles:
+keywords = TextRank.extract_keywords_advanced(text)
+
+# Fully customized extraction:
+extractor = TextRank::KeywordExtractor.new(
+  strategy:   :sparse,  # Specify PageRank strategy (dense or sparse)
+  damping:    0.85,     # The probability of following the graph vs. randomly choosing a new node
+  tolerance:  0.0001,   # The desired accuracy of the results
+  char_filters: [...],  # A list of filters to be applied prior to tokenization
+  tokenizer: ...,       # A class or tokenizer instance to perform tokenization
+  token_filters: [...], # A list of filters to be applied to each token after tokenization
+  graph_strategy: ...,  # A class or strategy instance for producing a graph from tokens
+  rank_filters: [...],  # A list of filters to be applied to the keyword ranks after keyword extraction
+)
+
+# Add another filter to the end of the char_filter chain
+extractor.add_char_filter(:AsciiFolding)
+
+# Add a part of speech filter to the token_filter chain BEFORE the Stopwords filter
+pos_filter = TextRank::TokenFilter::PartOfSpeech.new(parts_to_keep: %w[nn])
+extractor.add_token_filter(pos_filter, before: :Stopwords)
+
+# Perform the extraction with at most 100 iterations
+extractor.extract(text, max_iterations: 100)
+```
+
+**PageRank**
+
+It is also possible to use this gem for PageRank only.
+
+```ruby
+require 'page_rank'
+
+PageRank.calculate(strategy: :sparse, damping: 0.8, tolerance: 0.00001) do
+  add('node_a', 'node_b', weight: 3.2)
+  add('node_b', 'node_d', weight: 2.1)
+  add('node_b', 'node_e', weight: 4.7)
+  add('node_e', 'node_a', weight: 1.3)
+end
+```
+
+There are currently two pure Ruby implementations of PageRank:
+
+1. **sparse**: A sparsely-stored strategy which performs multiplication proportional
+to the number of edges in the graph.  For graphs with a very low node-to-edge
+ratio, this will perform better in a pure Ruby setting.  It is recommended to
+use this strategy until such a time as there are native implementations.
+2. **dense**: A densely-stored matrix strategy which performs up to `max_iterations`
+matrix multiplications or until the tolerance is reached.  This is more of a
+canonical implementation and is fine for small or dense graphs, but it is not
+advised for large, sparse graphs as Ruby is not fast when it comes to matrix
+multiplication.  Each iteration is O(N^3) where N is the number of graph nodes.
+
+## License
+
+MIT. See the `LICENSE.txt` file.
+
+
+## References
+
+> R. Mihalcea and P. Tarau, “TextRank: Bringing Order into Texts,” in Proceedings of EMNLP 2004. Association for Computational Linguistics, 2004, pp. 404–411.
+
+> Brin, S.; Page, L. (1998). "The anatomy of a large-scale hypertextual Web search engine". Computer Networks and ISDN Systems 30: 107–117.

data/Rakefile

@@ -0,0 +1,12 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec
+
+require 'rdoc/task'
+RDoc::Task.new do |rdoc|
+  rdoc.main = "README.md"
+  rdoc.rdoc_files.include("README.md", "lib/**/*.rb")
+end

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "text_rank"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/page_rank/base.rb

@@ -0,0 +1,89 @@
+module PageRank
+  ##
+  # A base class for PageRank implementations.  This class provides the basic
+  # framework for adding (optionall weighted) nodes to the graph and then
+  # performing iterations of PageRank to within the desired tolerance (or maximum
+  # allowed number of iterations).
+  ##
+  class Base
+
+    # @param (see #damping=)
+    # @param (see #tolerance=)
+    def initialize(damping: nil, tolerance: nil, **_)
+      self.damping = damping
+      self.tolerance = tolerance
+    end
+
+    # Set the damping probability
+    # @param damping [Float] The probability of following the graph vs. randomly choosing a new node
+    # @return [Float]
+    def damping=(damping)
+      @damping = damping || 0.85
+      raise ArgumentError.new('Invalid damping factor') if @damping <= 0 || @damping > 1
+      @damping
+    end
+
+    # Set the tolerance value
+    # @param tolerance [Float] The desired accuracy of the results
+    # @return [Float]
+    def tolerance=(tolerance)
+      @tolerance = tolerance || 0.0001
+      raise ArgumentError.new('Invalid tolerance factor') if @tolerance < 0 || @tolerance > 1
+      @tolerance
+    end
+
+    # Adds a directed (and optionally weighted) edge to the graph
+    # @param source [Object] The source node
+    # @param dest [Object] The destination node
+    # @return [nil]
+    def add(_source, _dest, **_options)
+      raise NotImplementedError
+    end
+
+    # Perform the PageRank calculation
+    # @param max_iterations [Fixnum] Maximum number of PageRank iterations to perform (or -1 for no max)
+    # @return [Hash<Object, Float>] of nodes with rank
+    def calculate(max_iterations: -1, **_)
+      ranks = initial_ranks
+      loop do
+        break if max_iterations == 0
+        ranks, prev_ranks = calculate_step(ranks), ranks
+        break if distance(ranks, prev_ranks) < @tolerance
+        max_iterations -= 1
+      end
+      sort_ranks(ranks)
+    end
+
+    protected
+
+    # Should return the number of nodes in the graph
+    def node_count
+      raise NotImplementedError
+    end
+
+    # Should produce the initial ranks from which to start the first PageRank iteration
+    def initial_ranks
+      raise NotImplementedError
+    end
+
+    # Should apply any sort of sorting logic to the result rankings after PageRank has finished
+    def sort_ranks(_ranks)
+      raise NotImplementedError
+    end
+
+    # Performs a single step of the PageRank iteration
+    def calculate_step(_ranks)
+      raise NotImplementedError
+    end
+
+    # Calculate the Euclidean distance from one ranking to the next iteration
+    def distance(v1, v2)
+      sum_squares = node_count.times.reduce(0.0) do |sum, i|
+        d = v1[i] - v2[i]
+        sum + d * d
+      end
+      Math.sqrt(sum_squares)
+    end
+
+  end
+end

data/lib/page_rank/dense.rb

@@ -0,0 +1,89 @@
+require 'matrix'
+
+module PageRank
+  ##
+  # Implementation of PageRank using matrix multiplication
+  #
+  # Ruby is not known for its speed, especial for math computations.  As such this
+  # implementation is not well suited for large graphs, and it is especially not
+  # well suited for graphs that have a small edge-to-vertex ratio.  The primary
+  # purpose of this implementation is to provide a checkpoint against other
+  # implementations to verify their validity.
+  #
+  # If speed is desired, it would be best to implement a NativeDense class (and
+  # optionally NativeSparse) which would perform the algorithm in C.
+  ##
+  class Dense < Base
+
+    # Initialize with default damping and tolerance.
+    # A maximum number of iterations can also be supplied
+    # (default is no maximum, i.e. iterate until tolerance).
+    # @param (see Base#initialize)
+    def initialize(**options)
+      super(**options)
+
+      @out_links = []
+      @key_to_idx = {}
+      @idx_to_key = {}
+    end
+
+    # @param (see Base#add)
+    # @param weight [Float] Optional weight for the graph edge
+    # @return (see Base#add)
+    def add(source, dest, weight: 1.0)
+      return if source == dest
+      source_idx = index(source)
+      dest_idx = index(dest)
+      @out_links[source_idx] ||= []
+      @out_links[source_idx][dest_idx] ||= 0.0
+      @out_links[source_idx][dest_idx] += weight
+      nil
+    end
+
+    protected
+
+    def node_count
+      @key_to_idx.size
+    end
+
+    def initial_ranks
+      @matrix = to_matrix
+      Vector[*[1 / node_count.to_f] * node_count]
+    end
+
+    def calculate_step(ranks)
+      @matrix * ranks
+    end
+
+    def sort_ranks(ranks)
+      ranks.each_with_index.sort_by { |r, _| -r }.each_with_object({}) do |(r, i), all|
+        all[@idx_to_key[i]] = r
+      end
+    end
+
+    private
+
+    def index(key)
+      @key_to_idx[key] ||= begin
+        @idx_to_key[node_count] = key
+        node_count
+      end
+    end
+
+    def to_matrix
+      total_out_weights = @out_links.map do |links|
+        links.compact.reduce(:+) if links
+      end
+      Matrix.build(node_count, node_count) do |dest_idx, source_idx|
+        total = total_out_weights[source_idx]
+        if total
+          w = @out_links[source_idx][dest_idx] || 0.0
+          @damping * w / total + (1 - @damping) / node_count.to_f
+        else
+          1.0 / node_count.to_f
+        end
+      end
+    end
+
+  end
+end

data/lib/page_rank/sparse.rb

@@ -0,0 +1,87 @@
+require 'set'
+
+module PageRank
+  ##
+  # Implementation of PageRank using a sparse matrix representation of the graph
+  #
+  # Ruby is not known for its speed, especial for math computations.  However,
+  # if the number of edges is relatively small in relation to the number of nodes,
+  # this pure Ruby implementation should perform well enough for many applications.
+  # It uses a sparse matrix representation and thus avoids an order of mangitude
+  # of calculations that are not necessary.
+  #
+  # If speed is desired, it would be best to implement a NativeSparse class (and
+  # optionally NativeDense) which would perform the algorithm in C.
+  ##
+  class Sparse < Base
+
+    # Initialize with default damping and tolerance.
+    # A maximum number of iterations can also be supplied
+    # (default is no maximum, i.e. iterate until tolerance).
+    # @param (see Base#initialize)
+    def initialize(**options)
+      super(**options)
+
+      @graph = {}
+      @weight_totals = Hash.new(0.0)
+      @weights = {}
+      @nodes = Set.new
+    end
+
+    # @param (see Base#add)
+    # @param weight [Float] Optional weight for the graph edge
+    # @return (see Base#add)
+    def add(source, dest, weight: 1.0)
+      return false if source == dest
+      @graph[dest] ||= Set.new
+      @graph[dest] << source
+      @weights[source] ||= Hash.new(0.0)
+      @weights[source][dest] += weight
+      @weight_totals[source] ||= 0.0
+      @weight_totals[source] += weight
+      @nodes << source
+      @nodes << dest
+      nil
+    end
+
+    protected
+
+    def node_count
+      @nodes.size
+    end
+
+    def initial_ranks
+      @dangling_nodes = @nodes - @weight_totals.keys
+      @normalized_weights = @weights.each_with_object({}) do |(source, values), h|
+        h[source] = values.each_with_object({}) do |(dest, w), h2| 
+          h2[dest] = w / @weight_totals[source]
+        end
+      end
+      Hash[@nodes.map { |k| [k, 1.0 / node_count.to_f] }]
+    end
+
+    def calculate_step(ranks)
+      ranks.keys.each_with_object({}) do |dest, new_ranks|
+        sum = 0.0
+        Array(@graph[dest]).each do |source|
+          sum += ranks[source] * @normalized_weights[source][dest]
+        end
+        @dangling_nodes.each do |source|
+          sum += ranks[source] / node_count.to_f
+        end
+        new_ranks[dest] = @damping * sum + (1 - @damping)/node_count
+      end
+    end
+
+    def sort_ranks(ranks)
+      sum = 0.0
+      ranks.each { |_, v| sum += v }
+      Hash[ranks.map { |k, v| [k, v / sum] }.sort_by { |_, v| -v }]
+    end
+
+    def distance(v1, v2)
+      super(v1.values.to_a, v2.values.to_a)
+    end
+
+  end
+end

data/lib/page_rank.rb

@@ -0,0 +1,39 @@
+##
+# A module for supporting Ruby implementations of PageRank. Rather than rely on
+# one single implementation, this module allows for multiple implementations that
+# may be beneficial in different scenarios.
+#
+# = Example
+#
+#   PageRank.calculate(strategy: :dense, damping: 0.8, max_iterations: 100) do
+#     add('nodeA', 'nodeC', weight: 4.3)
+#     add('nodeA', 'nodeE', weight: 2.1)
+#     add('nodeB', 'nodeC', weight: 3.6)
+#     add('nodeE', 'nodeD', weight: 1.9)
+#     add('nodeA', 'nodeC', weight: 5.3)
+#   end
+##
+module PageRank
+
+  autoload :Base,   'page_rank/base'
+  autoload :Dense,  'page_rank/dense'
+  autoload :Sparse, 'page_rank/sparse'
+
+  # @option options [Symbol] :strategy PageRank strategy to use (either :sparse or :dense)
+  # @option options [Float]  :damping The probability of following the graph vs. randomly choosing a new node
+  # @option options [Float]  :tolerance The desired accuracy of the results
+  # @return [PageRank::Base]
+  def self.new(strategy: :sparse, **options)
+    const_get(strategy.to_s.capitalize).new(**options)
+  end
+
+  # Convenience method to quickly calculate PageRank. In the calling block, graph edges can be added.
+  # @option (see new)
+  # @return (see Base#calculate)
+  def self.calculate(**options, &block)
+    pr = new(**options)
+    pr.instance_exec(&block)
+    pr.calculate(**options)
+  end
+
+end

data/lib/text_rank/char_filter/ascii_folding.rb

@@ -0,0 +1,26 @@
+# coding: utf-8
+module TextRank
+  module CharFilter
+    ##
+    # Characater filter to transform non-ASCII (unicode) characters into ASCII-friendly versions.
+    #
+    # = Example
+    #
+    #  AsciiFolding.new.filter!("the Perigordian Abbé then made answer, because a poor beggar of the country of Atrébatie heard some foolish things said")
+    #  => "the Perigordian Abbe then made answer, because a poor beggar of the country of Atrebatie heard some foolish things said"
+    ##
+    class AsciiFolding
+
+      NON_ASCII_CHARS        = 'ÀÁÂÃÄÅàáâãäåĀāĂ㥹ÇçĆćĈĉĊċČčÐðĎďĐđÈÉÊËèéêëĒēĔĕĖėĘęĚěĜĝĞğĠġĢģĤĥĦħÌÍÎÏìíîïĨĩĪīĬĭĮįİıĴĵĶķĸĹĺĻļĽľĿŀŁłÑñŃńŅņŇňʼnŊŋÒÓÔÕÖØòóôõöøŌōŎŏŐőŔŕŖŗŘřŚśŜŝŞşŠšſŢţŤťŦŧÙÚÛÜùúûüŨũŪūŬŭŮůŰűŲųŴŵÝýÿŶŷŸŹźŻżŽž'
+      EQUIVALENT_ASCII_CHARS = 'AAAAAAaaaaaaAaAaAaCcCcCcCcCcDdDdDdEEEEeeeeEeEeEeEeEeGgGgGgGgHhHhIIIIiiiiIiIiIiIiIiJjKkkLlLlLlLlLlNnNnNnNnnNnOOOOOOooooooOoOoOoRrRrRrSsSsSsSssTtTtTtUUUUuuuuUuUuUuUuUuUuWwYyyYyYZzZzZz'
+
+      # Perform the filter
+      # @param text [String]
+      # @return [String]
+      def filter!(text)
+        text.tr!(NON_ASCII_CHARS, EQUIVALENT_ASCII_CHARS)
+      end
+
+    end
+  end
+end

data/lib/text_rank/char_filter/lowercase.rb

@@ -0,0 +1,22 @@
+module TextRank
+  module CharFilter
+    ##
+    # Character filter to force text to lowercase
+    #
+    # = Example
+    #
+    #  Lowercase.new.filter!("What a pessimist you are! - Candide")
+    #  => "what a pessimist you are! - candide"
+    ##
+    class Lowercase
+
+      # Perform the filter
+      # @param text [String]
+      # @return [String]
+      def filter!(text)
+        text.downcase!
+      end
+
+    end
+  end
+end

data/lib/text_rank/char_filter/strip_email.rb

@@ -0,0 +1,24 @@
+module TextRank
+  module CharFilter
+    ##
+    # Character filter to remove email addresses from text.
+    #
+    # = Example
+    #
+    #  StripEmail.new.filter!("That is a hard question said candide@gmail.com")
+    #  => "That is a hard question said "
+    ##
+    class StripEmail
+
+      EMAIL_REGEX = /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i
+
+      # Perform the filter
+      # @param text [String]
+      # @return [String]
+      def filter!(text)
+        text.gsub!(EMAIL_REGEX, '')
+      end
+
+    end
+  end
+end

data/lib/text_rank/char_filter/strip_html.rb

@@ -0,0 +1,41 @@
+require 'nokogiri'
+require 'stringio'
+
+module TextRank
+  module CharFilter
+    ##
+    # Character filter to remove HTML tags and convert HTML entities to text.
+    #
+    # = Example
+    #
+    #  StripHtml.new.filter!(""Optimism", said Cacambo, "What is that?"")
+    #  => "\"Optimism\", said Cacambo, \"What is that?\""
+    #
+    #  StringHtml.new.filter!("<b>Alas! It is the <u>obstinacy</u> of maintaining that everything is best when it is worst.</b>")
+    #  => "Alas! It is the obstinacy of maintaining that everything is best when it is worst."
+    ##
+    class StripHtml < Nokogiri::XML::SAX::Document
+
+      def initialize
+        @text = StringIO.new
+      end
+
+      # Perform the filter
+      # @param text [String]
+      # @return [String]
+      def filter!(text)
+        @text.rewind
+        Nokogiri::HTML::SAX::Parser.new(self).parse(text)
+        @text.string
+      end
+
+      protected
+
+      def characters(string)
+        @text << ' '
+        @text << string
+      end
+
+    end
+  end
+end