ruby_ngrams 0.0.2 → 0.0.4

data/README.rdoc

@@ -1,18 +1,84 @@
 = ruby_ngrams
 
-== License
+Author:: Martin Velez
+Copyright:: Copyright (c) 2011 Martin Velez
+License:: Distributed under the same terms as Ruby
 
-Copyright 2011 Martin Velez
+= Description 
 
-== Features
+ruby_ngrams is an extension of Ruby's core String class.  It provides a String
+object with the capability to produce n-grams.
 
-* parses a string into a set of n-grams
+From Wikipedia, 
+ "In the fields of computational linguistics and probability, an n-gram is a 
+ contiguous sequence of n items from a given sequence of text or speech. The 
+ items in question can be phonemes, syllables, letters, words or base pairs 
+ according to the application. n-grams are collected from a text or speech corpus.
 
-== Installation
+ An n-gram of size 1 is referred to as a "unigram"; size 2 is a "bigram" 
+ (or, less commonly, a "digram"); size 3 is a "trigram"; size 4 is a "four-gram" 
+ and size 5 or more is simply called an "n-gram"." 
 
-* gem install ruby_ngrams
+= Design
 
-== Usage
+Instead of creating another namespace, this task seemed simple enough to merit 
+extending the String class.  A string is a sequence of characters.  
+It can be words, binary code, sentences, paragraphs, etc.  In short, 
+anything that you can store in a Ruby String object can be parsed into
+n-grams of length n. 
 
-usage goes here
+The main method being added to the String class is ngrams().  It produces an 
+array of n-grams from a Ruby String object.
 
+For example, let s be a Ruby String object.
+Then s.ngrams() returns array of n-grams of from s. 
+
+Tokenization of s is set to single characters by default.
+For example, if s = "Hello World!", 
+then the tokens of s are ["H","e","l","l","o"," ","W","o","r","l","d","!"].
+By specifying a regular expression, you can tokenize the string s in many
+different and useful ways.  
+
+If you set n = 4, then 
+s.ngrams = [["H", "e", "l", "l"], 
+["e", "l", "l", "o"], 
+["l", "l", "o", " "], 
+["l", "o", " ", "W"], 
+["o", " ", "W", "o"], 
+[" ", "W", "o", "r"], 
+["W", "o", "r", "l"], 
+["o", "r", "l", "d"], 
+["r", "l", "d", "!"]].
+Each item in the s.ngrams array can joined but doesn't need to be. 
+If you want to join them, normally you can do so easily if it is text.  
+Be careful if you are trying to join n-grams with non-printable characters.
+
+You can google "n-grams" to get more information about how n-grams are useful.
+
+= Installation
+
+gem install ruby_ngrams
+
+= Alternative Tools
+
+This is another tool I found but which did too much.  I only wanted 
+to produce n-grams from a string.
+1. raingrams[https://github.com/postmodern/raingrams]
+
+= Usage
+
+./ruby_ngrams --
+
+= Dependencies
+
+* Ruby 1.9.1 or greater
+* ruby_cli[https://github.com/martinvelez/ruby_cli] to run the gem executable
+
+= TODO
+
+* Test to determine limits of current approach which parses and stores n-grams
+in memory. 
+
+= Source Code
+
+https://github.com/martinvelez/ruby_ngrams

data/bin/ruby_ngrams

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby -w
+#!/usr/bin/env ruby
 
 require 'ruby_cli'
 require 'ruby_ngrams'
@@ -8,10 +8,11 @@ class App
 
 	def define_command_options() @options = {:regex => //, :n => 2}	end
 
-	# Define an OptionParser to parse the command line
-	def parse_options?
+	# Redefining the RubyCLI define_option_parser method
+	# Need to tell the OptionParser how to handle this command specific options.
+	def define_option_parser
 		#configure an OptionParser
-		@opt_parser = OptionParser.new do |opts|		
+		OptionParser.new do |opts|		
 			opts.banner = "Usage: #{__FILE__} [OPTIONS]... [FILE]..."
 			opts.separator ""
 			opts.separator "Specific options:"
@@ -25,12 +26,10 @@ class App
 			opts.on('-n', '--n NUM', Integer, 'set length n for n-grams') do |n|
 				@options[:n] = n
 			end
-			opts.on('-r', '--regex REGEX', Regexp, 'set regex to split string into tokens') do |r|
+			opts.on('-r', '--regex "REGEX"', Regexp, 'set regex to split string into tokens') do |r|
 				@options[:regex] = r
 			end
 		end
-		@opt_parser.parse!(@default_argv) rescue return false
-		true
 	end
 
 	def command
@@ -52,7 +51,7 @@ end
 
 
 if __FILE__ == $0
-	app = App.new(ARGV)
+	app = App.new(ARGV, __FILE__)
 	app.run
 end
 

metadata

@@ -1,13 +1,12 @@
 --- !ruby/object:Gem::Specification 
 name: ruby_ngrams
 version: !ruby/object:Gem::Version 
-  hash: 27
   prerelease: false
   segments: 
   - 0
   - 0
-  - 2
-  version: 0.0.2
+  - 4
+  version: 0.0.4
 platform: ruby
 authors: 
 - Martin Velez
@@ -15,10 +14,24 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-11-11 00:00:00 -08:00
+date: 2011-11-29 00:00:00 -08:00
 default_executable: 
-dependencies: []
-
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: ruby_cli
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        - 1
+        - 0
+        version: 0.1.0
+  type: :runtime
+  version_requirements: *id001
 description: A simple extension of the Ruby core string class to parse a string into n-grams
 email: mvelez999@gmail.com
 executables: 
@@ -46,7 +59,6 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
-      hash: 3
       segments: 
       - 0
       version: "0"
@@ -55,7 +67,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
-      hash: 3
       segments: 
       - 0
       version: "0"