Linguistics 1.0.3

data/README

@@ -0,0 +1,178 @@
+
+= Linguistics
+
+== Authors
+
+* Michael Granger <ged@FaerieMUD.org>
+* Martin Chase <stillflame@FaerieMUD.org>
+
+
+== Requirements
+
+* Ruby >= 1.8.0
+
+The following libraries are required, but are included in the redist/
+directory. The installer script should install these for you if you don't
+already have them.
+
+* HashSlice - Adds slicing to the builtin Hash class
+
+
+== Optional
+
+* CrossCase - Provide auto-aliasing of camelCase to under_barred methods and
+  vice-versa. (Included in redist/)
+
+* Ruby-WordNet (>= 0.02) - adds integration for the Ruby binding for the WordNet� lexical
+  reference system.
+
+  URL: http://www.deveiate.org/code/Ruby-WordNet.html
+  Download: http://www.deveiate.org/code/Ruby-WordNet-0.02.tar.gz
+
+* LinkParser (>= 0.0.4) - adds integration for the Ruby Link Grammar Parser by
+  Martin Chase.
+
+  URL: http://dev.faeriemud.org/~stillflame/linkparse.html
+  Download: http://www.faeriemud.org/code/Ruby-LinkParser-0.0.4.tgz
+
+
+== General Information
+
+This module is a framework for building linguistic utilities for Ruby objects in
+any language. It includes a generic language-independant front end, a module for
+mapping language codes into language names, and a module which contains various
+English-language utilities.
+
+
+=== Method Interface
+
+The Linguistics module comes with a language-independant mechanism for extending
+core Ruby classes with linguistic methods. 
+
+It consists of three parts: a core linguistics module which contains the
+class-extension framework for languages, a generic inflector class that serves
+as a delegator for linguistic methods on Ruby objects, and one or more
+language-specific modules which contain the actual linguistic functions.
+
+The module works by adding a single instance method for each language named
+after the language's two-letter code (or three-letter code, if no two-letter
+code is defined by ISO639) to various Ruby classes. This allows many
+language-specific methods to be added to objects without cluttering up the
+interface or risking collision between them, albeit at the cost of three or four
+more characters per method invocation.
+
+If you don't like extending core Ruby classes, the language modules should also
+allow you to use them as a function library as well.
+
+For example, the English-language module contains a #plural function which can
+be accessed via a method on a core class:
+
+  Linguistics::use( :en )
+  "goose".en.plural
+  # => "geese"
+  
+or via the Linguistics::EN::plural function directly:
+
+  include Linguistics::EN
+  plural( "goose" )
+  # => "geese"
+
+The class-extension mechanism actually uses the functional interface behind the
+scenes.
+
+A new feature with the 0.02 release: You can now omit the language-code method
+for unambiguous methods by calling Linguistics::use with the +:installProxy+
+configuration key, with the language code of the language module whose methods
+you wish to be available. For example, instead of having to call:
+
+  "goose".en.plural
+
+from the example above, you can now do this:
+
+  Lingusitics::use( :en, :installProxy => :en )
+  "goose".plural
+  # => "geese"
+
+More about how this works in the documentation for Linguistics::use.
+
+
+==== Adding Language Modules
+
+To add a new language to the framework, create a file named the same as the
+ISO639 2- or 3-letter language code for the language you're adding. It must be
+placed under lib/linguistics/ to be recognized by the linguistics module, but
+you can also just require it yourself prior to calling Linguistics::use(). This
+file should define a module under Linguistics that is an all-caps version of the
+code used in the filename. Any methods you wish to be exposed to users should be
+declared as module functions (ie., using Module#module_function).
+
+You may also wish to add your module to the list of default languages by adding
+the appropriate symbol to the Linguistics::DefaultLanguages array.
+
+For example, to create a Portuguese-language module, create a file called
+'lib/linguistics/pt.rb' which contains the following:
+
+  module Linguistics
+    module PT
+      Linguistics::DefaultLanguages << :pt
+
+      module_function
+	  <language methods here>
+    end
+  end
+
+See the English language module (lib/linguistics/en.rb) for an example.
+
+
+=== English Language Module
+
+See the README.english file for a synopsis.
+
+The English-language module currently contains linguistic functions ported from
+a few excellent Perl modules:
+
+  Lingua::EN::Inflect
+  Lingua::Conjunction
+  Lingua::EN::Infinitive
+
+See the lib/linguistics/en.rb file for specific attributions.
+
+New with version 0.02: integration with the Ruby WordNet� and LinkParser modules
+(which must be installed separately).
+
+
+== To Do
+  
+* I am planning on improving the results from the infinitive functions, which
+  currently return useful results only part of the time. Investigations into
+  additional stemming functions and some other strategies are ongoing.
+
+* Martin Chase <stillflame at FaerieMUD dot org> is working on an integration
+  module for his excellent work on a Ruby interface to the CMU Link Grammar (an
+  english-sentence parser). This will make writing fairly accurate natural
+  language parsers in Ruby much easier.
+
+* Suggestions (and patches) for any of these items or additional features are
+  welcomed.
+
+
+
+== Legal
+
+This module is Open Source Software which is Copyright (c) 2003 by The FaerieMUD
+Consortium. All rights reserved.
+
+You may use, modify, and/or redistribute this software under the terms of the
+Perl Artistic License, a copy of which should have been included in this
+distribution (See the file Artistic). If it was not, a copy of it may be
+obtained from http://language.perl.com/misc/Artistic.html or
+http://www.faeriemud.org/artistic.html).
+
+THIS PACKAGE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES,
+INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND
+FITNESS FOR A PARTICULAR PURPOSE.
+
+
+ $Id: README,v 1.6 2003/10/09 13:21:48 deveiant Exp $
+
+

data/README.english

@@ -0,0 +1,245 @@
+
+= English Ruby Linguistics Module - Synopsis
+
+This is an overview of the functionality currently in the English functions of
+the Ruby Linguistics module as of version 0.02:
+
+
+== Pluralization
+
+  require 'linguistics'
+  Linguistics::use( :en ) # extends Array, String, and Numeric
+
+  "box".en.plural
+  # => "boxes"
+
+  "mouse".en.plural
+  # => "mice"
+
+  "ruby".en.plural
+  # => "rubies"
+
+
+== Indefinite Articles
+
+  "book".en.a
+  # => "a book"
+
+  "article".en.a
+  # => "an article"
+
+
+== Present Participles
+
+  "runs".en.present_participle
+  # => "running"
+
+  "eats".en.present_participle
+  # => "eating"
+
+  "spies".en.present_participle
+  # => "spying"
+
+
+== Ordinal Numbers
+
+  5.en.ordinal
+  # => "5th"
+
+  2004.en.ordinal
+  # => "2004th"
+
+
+== Numbers to Words
+
+  5.en.numwords
+  # => "five"
+
+  2004.en.numwords
+  # => "two thousand and four"
+
+  2385762345876.en.numwords
+  # => "two trillion, three hundred and eighty-five billion,
+  seven hundred and sixty-two million, three hundred and
+  forty-five thousand, eight hundred and seventy-six"
+
+
+== Quantification
+
+  "cow".en.quantify( 5 )
+  # => "several cows"
+
+  "cow".en.quantify( 1005 )
+  # => "thousands of cows"
+
+  "cow".en.quantify( 20_432_123_000_000 )
+  # => "tens of trillions of cows"
+
+
+== Conjunctions
+
+  animals = %w{dog cow ox chicken goose goat cow dog rooster llama 
+  pig goat dog cat cat dog cow goat goose goose ox alpaca}
+  puts "The farm has: " + animals.en.conjunction
+
+  # => The farm has: four dogs, three cows, three geese, three goats,
+  two oxen, two cats, a chicken, a rooster, a llama, a pig, 
+  and an alpaca
+
+Note that 'goose' and 'ox' are both correctly pluralized, and the correct
+indefinite article 'an' has been used for 'alpaca'.
+
+You can also use the generalization function of the #quantify method to give
+general descriptions of object lists instead of literal counts:
+
+  allobjs = []
+  ObjectSpace::each_object {|obj| allobjs << obj.class.name}
+
+  puts "The current Ruby objectspace contains: " +
+    allobjs.en.conjunction( :generalize => true )
+
+which will print something like:
+
+  The current Ruby objectspace contains: thousands of Strings,
+  thousands of Arrays, hundreds of Hashes, hundreds of
+  Classes, many Regexps, a number of Ranges, a number of
+  Modules, several Floats, several Procs, several MatchDatas,
+  several Objects, several IOS, several Files, a Binding, a
+  NoMemoryError, a SystemStackError, a fatal, a ThreadGroup,
+  and a Thread
+
+
+== Infinitives
+
+New in version 0.02:
+
+  "leaving".en.infinitive
+  # => "leave"
+
+  "left".en.infinitive
+  # => "leave"
+
+  "leaving".en.infinitive.suffix
+  # => "ing"
+
+
+== WordNet� Integration
+
+Also new in version 0.02, if you have the Ruby-WordNet module installed, you can
+look up WordNet synsets using the Linguistics interface:
+
+  # Test to be sure the WordNet module loaded okay.
+  Linguistics::EN.has_wordnet?
+  # => true
+
+  # Fetch the default synset for the word "balance"
+  "balance".synset
+  # => #<WordNet::Synset:0x40376844 balance (noun): "a state of equilibrium"
+   (derivations: 3, antonyms: 1, hypernyms: 1, hyponyms: 3)>
+
+  # Fetch the synset for the first verb sense of "balance"
+  "balance".en.synset( :verb )
+  # => #<WordNet::Synset:0x4033f448 balance, equilibrate, equilibrize, equilibrise
+  (verb): "bring into balance or equilibrium; "She has to balance work and her
+  domestic duties"; "balance the two weights"" (derivations: 7, antonyms: 1,
+  verbGroups: 2, hypernyms: 1, hyponyms: 5)>
+
+  # Fetch the second noun sense
+  "balance".en.synset( 2, :noun )
+  # => #<WordNet::Synset:0x404ebb24 balance (noun): "a scale for weighing; depends
+  on pull of gravity" (hypernyms: 1, hyponyms: 5)>
+
+  # Fetch the second noun sense's hypernyms (more-general words, like a superclass)
+  "balance".en.synset( 2, :noun ).hypernyms
+  # => [#<WordNet::Synset:0x404e5620 scale, weighing machine (noun): "a measuring
+  instrument for weighing; shows amount of mass" (derivations: 2, hypernyms: 1,
+  hyponyms: 2)>]
+
+  # A simpler way of doing the same thing:
+  "balance".en.hypernyms( 2, :noun )
+  # => [#<WordNet::Synset:0x404e5620 scale, weighing machine (noun): "a measuring
+  instrument for weighing; shows amount of mass" (derivations: 2, hypernyms: 1,
+  hyponyms: 2)>]
+
+  # Fetch the first hypernym's hypernyms
+  "balance".en.synset( 2, :noun ).hypernyms.first.hypernyms
+  # => [#<WordNet::Synset:0x404c60b8 measuring instrument, measuring system,
+  measuring device (noun): "instrument that shows the extent or amount or quantity
+  or degree of something" (hypernyms: 1, hyponyms: 83)>]
+
+  # Find the synset to which both the second noun sense of "balance" and the
+  # default sense of "shovel" belong.
+  ("balance".en.synset( 2, :noun ) | "shovel".en.synset)
+  # => #<WordNet::Synset:0x40473da4 instrumentality, instrumentation (noun): "an
+  artifact (or system of artifacts) that is instrumental in accomplishing some
+  end" (derivations: 1, hypernyms: 1, hyponyms: 13)>
+
+  # Fetch just the words for the other kinds of "instruments"
+  "instrument".en.hyponyms.collect {|synset| synset.words}.flatten
+  # => ["analyzer", "analyser", "cautery", "cauterant", "drafting instrument",
+  "extractor", "instrument of execution", "instrument of punishment", "measuring
+  instrument", "measuring system", "measuring device", "medical instrument",
+  "navigational instrument", "optical instrument", "plotter", "scientific
+  instrument", "sonograph", "surveying instrument", "surveyor's instrument",
+  "tracer", "weapon", "arm", "weapon system", "whip"]
+
+There are many more WordNet methods supported � too many to list here. See the
+documentation for the complete list.
+
+
+== LinkParser Integration
+
+Another new feature in version 0.02 is integration with the Ruby version of the
+CMU Link Grammar Parser by Martin Chase. If you have the LinkParser module
+installed, you can create linkages from English sentences that let you query for
+parts of speech:
+
+  # Test to see whether or not the link parser is loaded.
+  Linguistics::EN.has_link_parser?
+  # => true
+
+  # Diagram the first linkage for a test sentence
+  puts "he is a big dog".sentence.linkages.first.to_s
+	  +---O*---+ 
+	  | +--Ds--+ 
+   +Ss+ |  +-A-+ 
+   |  | |  |   | 
+  he is a big dog
+
+  # Find the verb in the sentence
+  "he is a big dog".en.sentence.verb.to_s      
+  # => "is"
+
+  # Combined infinitive + LinkParser: Find the infinitive form of the verb of the
+  given sentence.
+  "he is a big dog".en.sentence.verb.infinitive
+  # => "be"
+
+  # Find the direct object of the sentence
+  "he is a big dog".en.sentence.object.to_s
+  # => "dog"
+
+  # Look at the raw LinkParser::Word for the direct object of the sentence.
+  "he is a big dog".en.sentence.object     
+  # => #<LinkParser::Word:0x403da0a0 @definition=[[{@A-}, Ds-, {@M+}, J-], [{@A-},
+  Ds-, {@M+}, Os-], [{@A-}, Ds-, {@M+}, Ss+, {@CO-}, {C-}], [{@A-}, Ds-, {@M+},
+  Ss+, R-], [{@A-}, Ds-, {@M+}, SIs-], [{@A-}, Ds-, {R+}, {Bs+}, J-], [{@A-}, Ds-,
+  {R+}, {Bs+}, Os-], [{@A-}, Ds-, {R+}, {Bs+}, Ss+, {@CO-}, {C-}], [{@A-}, Ds-,
+  {R+}, {Bs+}, Ss+, R-], [{@A-}, Ds-, {R+}, {Bs+}, SIs-]], @right=[], @suffix="",
+  @left=[#<LinkParser::Connection:0x403da028 @rword=#<LinkParser::Word:0x403da0a0
+  ...>, @lword=#<LinkParser::Word:0x403da0b4 @definition=[[Ss-, O+, {@MV+}], [Ss-,
+  B-, {@MV+}], [Ss-, P+], [Ss-, AF-], [RS-, Bs-, O+, {@MV+}], [RS-, Bs-, B-,
+  {@MV+}], [RS-, Bs-, P+], [RS-, Bs-, AF-], [{Q-}, SIs+, O+, {@MV+}], [{Q-}, SIs+,
+  B-, {@MV+}], [{Q-}, SIs+, P+], [{Q-}, SIs+, AF-]],
+  @right=[#<LinkParser::Connection:0x403da028 ...>], @suffix="", @left=[],
+  @name="is", @position=1>, @subName="*", @name="O", @length=3>], @name="dog",
+  @position=4>
+
+  # Combine WordNet + LinkParser to find the definition of the direct object of
+  # the sentence
+  "he is a big dog".en.sentence.object.gloss
+  # => "a member of the genus Canis (probably descended from the common wolf) that
+  has been domesticated by man since prehistoric times; occurs in many breeds;
+  \"the dog barked all night\""
+
+

data/TODO

@@ -0,0 +1,17 @@
+
+= Linguistics Module To-Do List
+
+* Add missing Lingua::EN::Inflect features:
+  * Number-insensitive Equality methods, e.g., +PL_eq()+, +PL_N_eq()+, etc.
+  * String iterpolation methods? This can already be done via things like +"The
+    #{'cat'.en.no(catCount)} in the hat."+, but it might be nice to have some
+    more convenient notation and an #interpolate method.
+  * User-defined inflections
+
+* Ask the stemmable.rb authors if they'd let me put it into the en.rb module.
+
+* Port more Perl modules:
+  * Lingua::Phonology?
+  * Others?
+
+

data/experiments/randobjlist.rb

@@ -0,0 +1,34 @@
+#!/usr/bin/ruby
+
+BEGIN {
+	$LOAD_PATH.unshift File::dirname(File::dirname( __FILE__ )) + "/lib"
+	require 'linguistics'
+}
+
+Linguistics::use( :en )
+
+# Just a fun little demo of the conjunction (junction, what's your) function.
+
+MinObjects = 5
+MaxObjects = 35
+Objects = %w[
+	butcher baker candlestick-maker
+	mouse clock
+	cat fiddle cow moon dog sport dish spoon
+	tisket tasket
+	jack jill hill pail crown
+]
+
+def randobjlist
+	objs = []
+	0.upto( rand(MaxObjects - MinObjects) + MinObjects ) do
+		objs << Objects[ rand(Objects.nitems) - 1 ]
+	end
+
+	return objs
+end
+
+
+puts "Random object list:\n\t" +
+	randobjlist().en.conjunction
+

data/install.rb

@@ -0,0 +1,154 @@
+#!/usr/bin/ruby
+#
+#	Linguistics Module Install Script
+#	$Id: install.rb,v 1.3 2003/10/09 13:23:09 deveiant Exp $
+#
+#	Thanks to Masatoshi SEKI for ideas found in his install.rb.
+#
+#	Copyright (c) 2001-2004 The FaerieMUD Consortium.
+#
+#	This is free software. You may use, modify, and/or redistribute this
+#	software under the terms of the Perl Artistic License. (See
+#	http://language.perl.com/misc/Artistic.html)
+#
+
+require './utils.rb'
+include UtilityFunctions
+
+require 'rbconfig'
+include Config
+
+require 'find'
+require 'ftools'
+
+
+$version	= %q$Revision: 1.3 $
+$rcsId		= %q$Id: install.rb,v 1.3 2003/10/09 13:23:09 deveiant Exp $
+
+# Define required libraries
+RequiredLibraries = [
+	# libraryname, nice name, RAA URL, Download URL
+# 	[ 'hashslice', "Ruby-Hashslice", 
+# 		'http://www.ruby-lang.org/en/raa-list.rhtml?name=Ruby-HashSlice',
+# 		'http://www.devEiate.org/code/Ruby-HashSlice-1.03.tar.gz' ],
+]
+
+class Installer
+
+	@@PrunePatterns = [
+		/CVS/,
+		/~$/,
+		%r:(^|/)\.:,
+		/\.tpl$/,
+	]
+
+	def initialize( testing=false )
+		@ftools = (testing) ? self : File
+	end
+
+	### Make the specified dirs (which can be a String or an Array of Strings)
+	### with the specified mode.
+	def makedirs( dirs, mode=0755, verbose=false )
+		dirs = [ dirs ] unless dirs.is_a? Array
+
+		oldumask = File::umask
+		File::umask( 0777 - mode )
+
+		for dir in dirs
+			if @ftools == File
+				File::mkpath( dir, $verbose )
+			else
+				$stderr.puts "Make path %s with mode %o" % [ dir, mode ]
+			end
+		end
+
+		File::umask( oldumask )
+	end
+
+	def install( srcfile, dstfile, mode=nil, verbose=false )
+		dstfile = File.catname(srcfile, dstfile)
+		unless FileTest.exist? dstfile and File.cmp srcfile, dstfile
+			$stderr.puts "   install #{srcfile} -> #{dstfile}"
+		else
+			$stderr.puts "   skipping #{dstfile}: unchanged"
+		end
+	end
+
+	public
+
+	def installFiles( src, dstDir, mode=0444, verbose=false )
+		directories = []
+		files = []
+		
+		if File.directory?( src )
+			Find.find( src ) {|f|
+				Find.prune if @@PrunePatterns.find {|pat| f =~ pat}
+				next if f == src
+
+				if FileTest.directory?( f )
+					directories << f.gsub( /^#{src}#{File::Separator}/, '' )
+					next 
+
+				elsif FileTest.file?( f )
+					files << f.gsub( /^#{src}#{File::Separator}/, '' )
+
+				else
+					Find.prune
+				end
+			}
+		else
+			files << File.basename( src )
+			src = File.dirname( src )
+		end
+		
+		dirs = [ dstDir ]
+		dirs |= directories.collect {|d| File.join(dstDir,d)}
+		makedirs( dirs, 0755, verbose )
+		files.each {|f|
+			srcfile = File.join(src,f)
+			dstfile = File.dirname(File.join( dstDir,f ))
+
+			if verbose
+				if mode
+					$stderr.puts "Install #{srcfile} -> #{dstfile} (mode %o)" % mode
+				else
+					$stderr.puts "Install #{srcfile} -> #{dstfile}"
+				end
+			end
+
+			@ftools.install( srcfile, dstfile, mode, verbose )
+		}
+	end
+
+end
+
+if $0 == __FILE__
+	header "Linguistics Installer #$version"
+
+	for lib in RequiredLibraries
+		testForRequiredLibrary( *lib )
+	end
+
+	viewOnly = ARGV.include? '-n'
+	verbose = ARGV.include? '-v'
+
+	# "Compatibility" with Aoki-san's install.rb
+	if ARGV.include?( 'config' ) || ARGV.include?( 'setup' )
+		print "Ok."
+		exit
+	end
+
+	debugMsg "Sitelibdir = '#{CONFIG['sitelibdir']}'"
+	sitelibdir = CONFIG['sitelibdir']
+	debugMsg "Sitearchdir = '#{CONFIG['sitearchdir']}'"
+	sitearchdir = CONFIG['sitearchdir']
+
+	message "Installing\n"
+	i = Installer.new( viewOnly )
+	i.installFiles( "redist", sitelibdir, 0444, verbose )
+	i.installFiles( "lib", sitelibdir, 0444, verbose )
+end
+	
+
+
+