rambling-trie 0.9.3 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 54be3d6a000a38c0864250d0951a1986b2ceac2c
-  data.tar.gz: 5a8b13f581ce79c239f71e7e96c014572697f0c6
+  metadata.gz: 5e58cb5a9636b79abd8fb93b4fae30ec3486871a
+  data.tar.gz: 5cb485068db0e4a889692433eedb1aa1682394f1
 SHA512:
-  metadata.gz: 62bf472c3abfc664ca22e2df3ba55254d3e03e1d66ab7f3f64bba4755feaf6e4ce92de604ff5c7b479056840734b4e3ca0be4707af447f0521c327d1bbc274e6
-  data.tar.gz: c13d4ce29e89093292268426240cc543026f11a4e0747a33c65bcd7950b156629a39ac61a9a5a78a95320de82d734680a532a043ac78a4c56e0334e746f9bb72
+  metadata.gz: 7d035ffc3fa86ee5be924cac9f73701df016221158e5654d3c908ec2adf034f76bf2d9081d56027c5e22c812a99241f671de64826f6b094fba153c62014edaeb
+  data.tar.gz: ac2151ec179554ed23d990e92fdfbce683ca88086f04eaa4becf4ad392c77a947327766ea59c4de561e62d741ccd815d637241f415e16fcebbebb2d712a4cf0a

data/Gemfile

@@ -3,6 +3,8 @@ source 'https://rubygems.org'
 
 gemspec
 
+gem 'rubyzip'
+
 group :development do
   gem 'ruby-prof'
   gem 'memory_profiler'

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2012-2016 Edgar Gonzalez
+Copyright (c) 2012-2017 Edgar Gonzalez
 
 MIT License
 

data/README.md

@@ -2,7 +2,7 @@
 
 [![Gem Version][badge_fury_badge]][badge_fury_link] [![Dependency Status][gemnasium_badge]][gemnasium_link] [![Build Status][travis_ci_badge]][travis_ci_link] [![Code Climate][code_climate_badge]][code_climage_link] [![Coverage Status][coveralls_badge]][coveralls_link] [![Documentation Status][inch_ci_badge]][inch_ci_link]
 
-The Rambling Trie is a custom implementation of the Trie data structure with Ruby, which includes compression abilities and is designed to be very fast to traverse.
+The Rambling Trie is a Ruby implementation of the [trie data structure][trie_wiki], which includes compression abilities and is designed to be very fast to traverse.
 
 ## Installing the Rambling Trie
 
@@ -13,7 +13,7 @@ You will need:
 * Ruby 2.1.0 or up
 * RubyGems
 
-See [RVM][rvm] or [rbenv][rbenv] for more information on how to manage Ruby versions.
+See [RVM][rvm], [rbenv][rbenv] or [chruby][chruby] for more information on how to manage Ruby versions.
 
 ### Installation
 
@@ -29,15 +29,17 @@ Or, include it in your `Gemfile` and bundle it:
 gem 'rambling-trie'
 ```
 
-## How to use the Rambling Trie
+## Using the Rambling Trie
 
-To create the trie, initialize it like this:
+### Creation
+
+To create a new trie, initialize it like this:
 
 ``` ruby
 trie = Rambling::Trie.create
 ```
 
-You can also provide a block and the created instance will be yielded for you to perform any operation on it:
+You can also provide a block and the created trie instance will be yielded for you to perform any operation on it:
 
 ``` ruby
 Rambling::Trie.create do |trie|
@@ -62,52 +64,67 @@ the
 trie
 ```
 
-If you want to use a custom file format, you will need to provide a custom file reader that defines the `each_word` method that yields each word contained in the file. Look at the `Rambling::Trie::PlainTextReader` class for an example.
+If you want to use a custom file format, you will need to provide a custom file reader that defines an `#each_word` method that yields each word contained in the file. Look at the [`PlainText` reader][rambling_trie_plain_text_reader] class for an example, and at the [Configuration section][rambling_trie_configuration] to see how to add your own custom file readers.
+
+### Operations
 
-To add new words to the trie, use `add` or `<<`:
+To add new words to the trie, use `#add` or its alias `#<<`:
 
 ``` ruby
 trie.add 'word'
 trie << 'word'
 ```
 
-And to find out if a word already exists in the trie, use `word?` or `include?`:
+And to find out if a word already exists in the trie, use `#word?` or its alias `#include?`:
 
 ``` ruby
 trie.word? 'word'
 trie.include? 'word'
 ```
 
-If you wish to find if part of a word exists in the trie instance, you should call `partial_word?`:
+If you wish to find if part of a word exists in the trie instance, you should call `#partial_word?` or its alias `#match?`:
 
 ``` ruby
 trie.partial_word? 'partial_word'
 trie.match? 'partial_word'
 ```
 
-To get all the words that start with a particular string, you can use `scan`:
+To get all the words that start with a particular string, you can use `#scan` or its alias `#words`:
 
 ``` ruby
 trie.scan 'hi' # => ['hi', 'high', 'highlight', ...]
 trie.words 'hi' # => ['hi', 'high', 'highlight', ...]
 ```
 
+To get all the words within a given string, you can use `#words_within`:
+
+``` ruby
+trie.words_within 'ifdxawesome45someword3' # => ['if', 'aw', 'awe', ...]
+trie.words_within 'tktktktk' # => []
+```
+
+Or, if you're just interested in knowing whether a given string contains any valid words or not, you can use `#words_within?`:
+
+``` ruby
+trie.words_within? 'ifdxawesome45someword3' # => true
+trie.words_within? 'tktktktk' # => false
+```
+
 ### Compression
 
-By default, the Rambling Trie works as a Standard Trie.
-Starting from version 0.1.0, you can obtain a Compressed Trie from the Standard one, by using the compression feature.
-Just call the `compress!` method on the trie instance:
+By default, the Rambling Trie works as a standard trie. Starting from version 0.1.0, you can obtain a compressed trie from the standard one, by using the compression feature. Just call the `#compress!` method on the trie instance:
 
 ``` ruby
 trie.compress!
 ```
 
-This will reduce the amount of Trie nodes by eliminating the redundant ones, which are the only-child non-terminal nodes.
+This will reduce the size of the trie by using redundant node elimination (redundant nodes are the only-child non-terminal nodes).
 
-__Note that the `compress!` method acts over the trie instance it belongs to.__
-__Also, adding words after compression is not supported.__
+> _**Note**: The `#compress!` method acts over the trie instance it belongs to
+> and is destructive. Also, adding words after compression (with `#add` or
+> `#<<`) is not supported._
 
-You can find out if a trie instance is compressed by calling the `compressed?` method:
+You can find out if a trie instance is compressed by calling the `#compressed?` method:
 
 ``` ruby
 trie.compressed?
@@ -118,15 +135,93 @@ trie.compressed?
 Starting from version 0.4.2, you can use any `Enumerable` method over a trie instance, and it will iterate over each word contained in the trie. You can now do things like:
 
 ``` ruby
-trie.each do |word|
-  puts word
-end
-
+trie.each { |word| puts word }
 trie.any? { |word| word.include? 'x' }
+trie.all? { |word| word.include? 'x' }
 # etc.
 ```
 
-## Further Documentation
+### Serialization
+
+Starting from version 1.0.0, you can store a full trie instance on disk and retrieve/use it later on. Loading a trie from disk takes less time, less cpu and less memory than loading every word into the trie every time. This is particularly useful for production applications, when you have word lists that you know are going to be static, or that change with little frequency.
+
+To store a trie on disk, you can use `.dump` like this:
+
+``` ruby
+Rambling::Trie.dump trie, '/path/to/file'
+```
+
+Then, when you need to use a trie next time, you don't have to create a new one with all the necessary words. Rather, you can retrieve a previously stored one with `.load` like this:
+
+``` ruby
+trie = Rambling::Trie.load trie, '/path/to/file'
+```
+
+#### Supported formats
+
+Currently, these formats are supported to store tries on disk:
+
+- Ruby's [binary (Marshal)][marshal] format
+- [YAML][yaml]
+
+> When dumping into or loading from disk, the format is determined
+> automatically based on the file extension, so `.yml` or `.yaml` files will be
+> handled through `YAML` and `.marshal` files through `Marshal`.
+
+Optionally, you can use a `.zip` version of the supported formats. In order to do so, you'll have to install the [`rubyzip`][rubyzip] gem:
+
+``` bash
+gem install rubyzip
+```
+
+Or, include it in your `Gemfile` and bundle it:
+
+``` ruby
+gem 'rubyzip'
+```
+
+Then, you can load contents form a `.zip` file like this:
+
+``` ruby
+require 'zip'
+trie = Rambling::Trie.load trie, '/path/to/file.zip'
+```
+
+> For `.zip` files, the format is also determined automatically based on the
+> file extension, so `.yml.zip` or `.yaml.zip` files will be handled through
+> `YAML` after decompression and `.marshal.zip` files through `Marshal`.
+
+### Configuration
+
+Starting from version 1.0.0, you can change the configuration values used by Rambling Trie. You can now supply:
+
+* A `Compressor` object
+* A root `Node` builder
+* More `Readers` (implement `#each_word`)
+* Change the default `reader`
+* More `Serializers` (implement `#dump` and `#load`)
+* Change the default `serializer`
+
+You can configure those values by using `.config` like this:
+
+```ruby
+require 'rambling-trie'
+
+Rambling::Trie.config do |config|
+  config.compressor = MyCompressor.new
+  config.root_builder = lambda { MyNode.new }
+
+  config.readers.add :html, MyHtmlReader.new
+  config.readers.default = config.readers[:html]
+
+  config.serializers.add :json, MyJsonSerializer.new
+  config.serializers.default = config.serializers[:yml]
+end
+
+# Create a trie or load one from disk and do things with it...
+```
+
+### Further Documentation
 
 You can find further API documentation on the autogenerated [rambling-trie gem RubyDoc.info page][rubydoc] or if you want edge documentation, you can go the [GitHub project RubyDoc.info page][rubydoc_github].
 
@@ -139,16 +234,19 @@ The Rambling Trie has been tested with the following Ruby versions:
 * 2.2.x
 * 2.1.x
 
-Ruby 1.8.x, 1.9.x and 2.0.x are not supported.
+**No longer supported**:
+
+* 2.0.x (might still work, but is not officially supported)
+* 1.9.x
+* 1.8.x
 
 ## Contributing to Rambling Trie
 
-If you want to contribute to this project, you will need RSpec to run the tests.
-Also, be sure to add tests for any feature you may develop or bug you may fix.
+Take a look at the [contributing guide][rambling_trie_contributing_guide] to get started, or fire a question to [@gonzedge][github_user_gonzedge].
 
 ## License and copyright
 
-Copyright (c) 2012-2016 Edgar Gonzalez
+Copyright (c) 2012-2017 Edgar Gonzalez
 
 MIT License
 
@@ -160,17 +258,26 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI
 
 [badge_fury_badge]: https://badge.fury.io/rb/rambling-trie.svg
 [badge_fury_link]: https://badge.fury.io/rb/rambling-trie
+[chruby]: https://github.com/postmodern/chruby
 [code_climage_link]: https://codeclimate.com/github/gonzedge/rambling-trie
 [code_climate_badge]: https://codeclimate.com/github/gonzedge/rambling-trie/badges/gpa.svg
 [coveralls_badge]: https://img.shields.io/coveralls/gonzedge/rambling-trie.svg
 [coveralls_link]: https://coveralls.io/r/gonzedge/rambling-trie
 [gemnasium_badge]: https://gemnasium.com/gonzedge/rambling-trie.svg
 [gemnasium_link]: https://gemnasium.com/gonzedge/rambling-trie
+[github_user_gonzedge]: https://github.com/gonzedge
 [inch_ci_badge]: https://inch-ci.org/github/gonzedge/rambling-trie.svg?branch=master
 [inch_ci_link]: https://inch-ci.org/github/gonzedge/rambling-trie
+[marshal]: https://ruby-doc.org/core-2.4.0/Marshal.html
+[rambling_trie_configuration]: https://github.com/gonzedge/rambling-trie#configuration
+[rambling_trie_contributing_guide]: https://github.com/gonzedge/rambling-trie/blob/master/CONTRIBUTING.md
+[rambling_trie_plain_text_reader]: https://github.com/gonzedge/rambling-trie/blob/master/lib/rambling/trie/readers/plain_text.rb
 [rbenv]: https://github.com/sstephenson/rbenv
 [rubydoc]: http://rubydoc.info/gems/rambling-trie
 [rubydoc_github]: http://rubydoc.info/github/gonzedge/rambling-trie
+[rubyzip]: https://github.com/rubyzip/rubyzip
 [rvm]: https://rvm.io
 [travis_ci_badge]: https://travis-ci.org/gonzedge/rambling-trie.svg
 [travis_ci_link]: https://travis-ci.org/gonzedge/rambling-trie
+[trie_wiki]: https://en.wikipedia.org/wiki/Trie
+[yaml]: https://ruby-doc.org/stdlib-2.4.0/libdoc/yaml/rdoc/YAML.html

data/Rakefile

@@ -1,7 +1,6 @@
 require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
-require 'rambling-trie'
-require 'rambling/trie/tasks/performance'
+require_relative 'tasks/performance'
 
 RSpec::Core::RakeTask.new :spec
 

data/lib/rambling/trie.rb

@@ -1,8 +1,9 @@
 require 'forwardable'
+
 %w{
-  forwardable compression compressor inspector container enumerable
-  invalid_operation plain_text_reader node missing_node compressed_node
-  raw_node version
+  forwardable comparable compressable compressor configuration container
+  enumerable inspectable invalid_operation readers serializers stringifyable
+  node missing_node compressed_node raw_node version
 }.each do |file|
   require File.join('rambling', 'trie', file)
 end
@@ -12,15 +13,25 @@ module Rambling
   # Entry point for rambling-trie API.
   module Trie
     class << self
-      # Creates a new Trie. Entry point for the Rambling::Trie API.
+      extend Rambling::Trie::Forwardable
+
+      delegate [
+        :readers,
+        :serializers,
+        :compressor,
+        :root_builder
+      ] => :properties
+
+      # Creates a new Rambling::Trie. Entry point for the Rambling::Trie API.
       # @param [String, nil] filepath the file to load the words from.
+      # @param [Reader, nil] reader the file parser to get each word. See
+      #   {Rambling::Trie::Readers Readers}.
       # @return [Container] the trie just created.
       # @yield [Container] the trie just created.
       def create filepath = nil, reader = nil
-        reader ||= default_reader
-
-        Rambling::Trie::Container.new do |container|
+        Rambling::Trie::Container.new root_builder.call, compressor do |container|
           if filepath
+            reader ||= readers.resolve filepath
             reader.each_word filepath do |word|
               container << word
             end
@@ -30,10 +41,43 @@ module Rambling
         end
       end
 
+      # Loads an existing trie from disk into memory.
+      # @param [String] filepath the file to load the words from.
+      # @param [Serializer, nil] serializer the object responsible of loading the trie
+      #   from disk. See {Rambling::Trie::Serializers Serializers}.
+      # @return [Container] the trie just loaded.
+      # @yield [Container] the trie just loaded.
+      def load filepath, serializer = nil
+        serializer ||= serializers.resolve filepath
+        root = serializer.load filepath
+        Rambling::Trie::Container.new root, compressor do |container|
+          yield container if block_given?
+        end
+      end
+
+      # Dumps an existing trie from memory into disk.
+      # @param [Container] trie the trie to dump into disk.
+      # @param [String] filepath the file to dump to serialized trie into.
+      # @param [Serializer, nil] serializer the object responsible of
+      #   serializing and dumping the trie into disk. See
+      #   {Rambling::Trie::Serializers Serializers}.
+      def dump trie, filepath, serializer = nil
+        serializer ||= serializers.resolve filepath
+        serializer.dump trie.root, filepath
+      end
+
+      # Provides configuration properties for the Rambling::Trie gem.
+      # @return [Properties] the configured properties of the gem.
+      # @yield [Properties] the configured properties of the gem.
+      def config
+        yield properties if block_given?
+        properties
+      end
+
       private
 
-      def default_reader
-        Rambling::Trie::PlainTextReader.new
+      def properties
+        @properties ||= Rambling::Trie::Configuration::Properties.new
       end
     end
   end

data/lib/rambling/trie/comparable.rb

@@ -0,0 +1,16 @@
+module Rambling
+  module Trie
+    # Provides the comparable behavior for the trie data structure.
+    module Comparable
+      # Compares two nodes.
+      # @param [Node] other the node to compare against.
+      # @return [Boolean] `true` if the nodes' {Node#letter #letter} and
+      #   {Node#children_tree #children_tree} are equal, `false` otherwise.
+      def == other
+        letter == other.letter &&
+          terminal? == other.terminal? &&
+          children_tree == other.children_tree
+      end
+    end
+  end
+end

data/lib/rambling/trie/compressable.rb

@@ -0,0 +1,14 @@
+module Rambling
+  module Trie
+    # Provides the compressable behavior for the trie data structure.
+    module Compressable
+      # Indicates if the current {Rambling::Trie::Node Node} can be compressed
+      # or not.
+      # @return [Boolean] `true` for non-{Node#terminal? terminal} nodes with
+      #   one child, `false` otherwise.
+      def compressable?
+        !(root? || terminal?) && children_tree.size == 1
+      end
+    end
+  end
+end

data/lib/rambling/trie/compressed_node.rb

@@ -1,39 +1,41 @@
 module Rambling
   module Trie
-    # A representation of a node in an compressed Trie data structure.
+    # A representation of a node in an compressed trie data structure.
     class CompressedNode < Rambling::Trie::Node
-      # Always raises [Rambling::Trie::InvalidOperation] when trying to add a
-      # branch to the current trie node based on the word
-      # @param [String] word the word to add the branch from.
+      # Always raises {Rambling::Trie::InvalidOperation InvalidOperation} when
+      # trying to add a word to the current compressed trie node
+      # @param [String] word the word to add to the trie.
       # @raise [InvalidOperation] if the trie is already compressed.
+      # @return [nil] this never returns as it always raises an exception.
       def add word
-        raise Rambling::Trie::InvalidOperation, 'Cannot add branch to compressed trie'
+        raise Rambling::Trie::InvalidOperation, 'Cannot add word to compressed trie'
       end
 
-      # Checks if a path for set of characters exists in the trie.
-      # @param [Array] chars the characters to look for in the trie.
+      # Checks if a path for set a of characters exists in the trie.
+      # @param [Array<String>] chars the characters to look for in the trie.
       # @return [Boolean] `true` if the characters are found, `false` otherwise.
       def partial_word? chars
         chars.empty? || has_partial_word?(chars)
       end
 
       # Checks if a path for set of characters represents a word in the trie.
-      # @param [Array] chars the characters to look for in the trie.
+      # @param [Array<String>] chars the characters to look for in the trie.
       # @return [Boolean] `true` if the characters are found and form a word,
-      # `false` otherwise.
+      #   `false` otherwise.
       def word? chars
         chars.empty? ? terminal? : has_word?(chars)
       end
 
-      # Returns all words that start with the specified characters.
-      # @param [Array] chars the characters to look for in the trie.
-      # @return [Array] all the words contained in the trie that start with the specified characters.
+      # Returns the node that starts with the specified characters.
+      # @param [Array<String>] chars the characters to look for in the trie.
+      # @return [Node] the node that matches the specified characters.
+      #   {MissingNode MissingNode} when not found.
       def scan chars
         chars.empty? ? self : closest_node(chars)
       end
 
-      # Always return `true` for a raw (compressed) node.
-      # @return [Boolean] always true for a raw (compressed) node.
+      # Always return `true` for a compressed node.
+      # @return [Boolean] always `true` for a compressed node.
       def compressed?
         true
       end
@@ -65,6 +67,28 @@ module Rambling
         recursive_get(:scan, chars) || Rambling::Trie::MissingNode.new
       end
 
+      def children_match_prefix chars
+        return enum_for :children_match_prefix, chars unless block_given?
+
+        current_key = nil
+
+        while !chars.empty?
+          if current_key
+            current_key << chars.slice!(0)
+          else
+            current_key = chars.slice!(0)
+          end
+
+          child = children_tree[current_key.to_sym]
+
+          if child
+            child.match_prefix chars do |word|
+              yield word
+            end
+          end
+        end
+      end
+
       def recursive_get method, chars
         current_length = 0
         current_key = current_key chars.slice!(0)