fa 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 4b7eae1d9e0955a9ef22ede62cdf480030dbf27f
-  data.tar.gz: 2cb62222893579c9b18c61a550d90722417583f6
+  metadata.gz: 5f916d972953f8fa49a4b31c3523e0a360424333
+  data.tar.gz: b751b82025a64eccf35c143f01b43ddc35105daa
 SHA512:
-  metadata.gz: 6b4644001fd36dac35f6f0c033a510e8b14e448ae615ec0bb3d74094277c3ed025b95ae6226fca3687611f31631e644a1e95b925fe91d6e46ddde83e014ae9fb
-  data.tar.gz: 0aa3a9e2338f14cb695d39a0873075c49a84f4adb05c6d749159c1869c679c8ffd83d5d457beb3f083061833cf887debda4cf7b8fbc8971c213ebcbca06d91da
+  metadata.gz: eaac55dbfc978116be6cddff230df876dd0263b9a795d47b2032d669707081f17a49c98feaabac21c6fd1a3bf8be2c509431c494e279d199d71311d98d95a54a
+  data.tar.gz: 11e957f7da8b07f41337c80e5dc6aeb2583d660010ae705b57d8a59eb70ea711868beb4d5f7b7ea7268275e404f3309fd03355159607c47e9122380454db8819

data/README.md

@@ -1,8 +1,12 @@
 # Fa
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/fa`. To experiment with that code, run `bin/console` for an interactive prompt.
-
-TODO: Delete this and the text above, and describe your gem
+This gem provides bindings to [libfa](http://augeas.net/libfa/index.html),
+a library for doing algebra on [regular expressions](#syntax). If you've
+ever asked yourself questions like "Are these two regular expressions
+matching the same set of strings ?" or wanted to determine a regular
+expression that matches all strings that match one regular expression, but
+not a second one, this is the library that can answer these questions for
+you.
 
 ## Installation
 
@@ -20,17 +24,150 @@ Or install it yourself as:
 
     $ gem install fa
 
+For things to work out, you will also have to have `libfa` installed; the
+library is distributed as part of [augeas](http://augeas.net/). On Red
+Hat-derived distros like Fedora, CentOS, or RHEL, you will need to `yum
+install augeas-libs`, on Debian-derived distros, run `apt-get install
+libaugeas0`.
+
 ## Usage
 
-TODO: Write usage instructions here
+To perform computations on regular expressions, `libfa` needs to first
+convert your regular expression into a finite automaton. This is done by
+compiling your regular expression:
+
+```ruby
+    fa1 = Fa.compile("(a|b)")  # can also be written as Fa["(a|b)"]
+    fa2 = fa1.plus
+```
+
+Notice that the regular expression needs to be given as a
+string. Unfortunately, Ruby regular expressions allow constructs that go
+beyond the mathematical notion of a regular expression and can therefore
+not be used to do the kinds of computation that `libfa` performs. The
+regular expressions that `libfa` deals in must be written using a (subset
+of) the notation for
+[extended POSIX regular expressions](https://en.wikibooks.org/wiki/Regular_Expressions/POSIX-Extended_Regular_Expressions). The
+biggest difference between POSIX ERE and the syntax that `libfa`
+understands is that `libfa` does not allow backreferences, does not support
+anchors like `^` and `$`, and does not support named character classes like
+`[[:space:]]`.
+
+You can always turn a finite automaton back into a regular expression using
+`Fa#to_s`:
+
+```ruby
+
+    puts fa1
+    # "b|a"
+    puts fa1.minimize
+    # "[ab]"
+    puts fa1.union(fa2).minimize
+    # "[ab][ab]*"
+    puts fa1.concat(fa2).minimize
+    # "[ab][ab][ab]*"
+    puts fa2.intersect(fa1).minimize
+    # "b|a"
+    puts fa2.intersect(Fa["a*"]).minimize
+    # "aa*"
+    puts fa1.intersect(Fa["a*"]).minimize
+    # "a"
+    puts Fa["a+"].minus(Fa["a{2,}"])
+    # "a"
+```
+
+You can also compare finite automata, and therefore learn things on how
+they behave on _all_ strings, for example if they match the same exact set
+of strings, or if one matches strictly more strings than another:
+
+```ruby
+    fa = Fa["[a-z]"].intersect(Fa["a*"])
+    puts Fa["a"].equals(fa)
+    # true
+    fa = Fa["a"].union(Fa["b"]).star.concat(Fa["c"].plus)
+    puts Fa["(a|b)*c+"].equals(fa)
+    # true
+    puts Fa["[ab]*"].contains(Fa["a*"])
+    # true
+    puts Fa["a+"].minus(Fa["a*"]).empty?
+    # true
+```
+
+### Syntax
+
+The regular expressions that `libfa` understand are a subset of the POSIX
+extended regular expression syntax. If you are a regular expression
+aficionado, you should note that `libfa` does not support some popular
+syntax extensions. Most importantly, it does not support backreferences,
+anchors such as `^` and `$`, and named character classes such as
+`[[:upper:]]`. The first two are not supported since they take the notation
+out of the realm of finite automata and actual regular expressions. Named
+character classes are not implemented because there's a lot of work to
+support them, even though there is no objection from theory to them.
+
+The character set that `libfa` operates on is simple 8 bit ASCII. In other
+words, to `libfa`, a character is a byte, and it does not support larger
+character sets such as UTF-8.
+
+The following characters have special meaning for `libfa`. The symbols `R`,
+`S`, `T`, etc. in the list below can be regular expressions themselves. The
+list is ordered by increasing precendence of the operators.
+
+* `R|S`: matches anything that matches either `R` or `S`
+* `R*`: matches any number of `R`, including none
+* `R+`: matches any number of `R`, but at least one
+* `R?`: matches no or one occurence of `R`
+* `R{n,m}`: matches at least `n` but no more than `m` occurences of
+  `R`. `n` must be nonnegative. If `m` is missing or equals `-1`, matches
+  an unlimited number of `R`.
+* `(R)`: the parentheses are solely there for grouping, and this expression
+  matches the same strings as `R` alone
+* `[C]`: matches the characters in the character set `C`; see below for the
+  syntax of character sets
+* `.`: matches any single character except for newline
+* `\c`: the literal character `c`, even if it would otherwise have special
+  meaning; the expression `\(` matches an opening parenthesis.
+
+Character classes `[C]` use the following notation:
+
+* `[^C]`: matches all characters not in `[C]`. `[^a-zA-Z]` matches
+  everything that is not a letter.
+* `[a-z]`: matches all characters between `a` and `z`, inclusive. Multiple
+  ranges can be specified in the same character set. `[a-zA-Z0-9]` is a
+  perfectly valid character set.
+* if a character set should include `]`, it must be listed as the first
+  character. `[][]` matches the opening and closing bracket.
+* if a character set should include `-`, it must be listed as the last
+  character. `[-]` matches solely a dash.
+* no characters in character sets are special, and there is no backslash
+  escaping of characters in character classes. `[.]` matches a literal dot.
+
+The regular expression syntax has no notation for control characters: when
+`libfa` sees `\n` in a string you are compiling, it will match that against
+the character `n`, not a newline. That's not a problem as the strings you
+write in Ruby code go through Ruby's backslash interpretation first. When
+you write `Fa.compile("[\n]")`, `libfa` never sees the backslash as Ruby
+replaces `\n` with a newline character before that string ever makes it to
+`libfa`. That has the funny consequence that if you want to use a literal
+backslash in your regular expression, your input string must have _four_
+backslashes in it: when you write `Fa.compile("\\\\")`, Ruby first turns
+that into a string with two backslashes, which `libfa` then interprets as a
+single
+backslash. \[_If you are reading this in YARD documentation and only saw two backslashes in the `Fa.compile`, it's because YARD reduced them from the markdown source. Github does not, and so this example will always be wrong in one of them._\]
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies. Then,
+run `rake test` to run the tests. You can also run `bin/console` for an
+interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake
+install`. To release a new version, update the version number in
+`version.rb`, and then run `bundle exec rake release`, which will create a
+git tag for the version, push git commits and tags, and push the `.gem`
+file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/fa.
-
+Bug reports and pull requests are welcome on GitHub at
+https://github.com/lutter/ruby-fa.

data/Rakefile

@@ -1,5 +1,6 @@
 require "bundler/gem_tasks"
 require "rake/testtask"
+require 'yard'
 
 Rake::TestTask.new(:test) do |t|
   t.libs << "test"
@@ -7,4 +8,9 @@ Rake::TestTask.new(:test) do |t|
   t.test_files = FileList['test/**/*_test.rb']
 end
 
+YARD::Rake::YardocTask.new do |t|
+ #t.options = ['--any', '--extra', '--opts']
+ t.stats_options = ['--list-undoc']
+end
+
 task :default => :test

data/fa.gemspec

@@ -35,4 +35,5 @@ EOS
   spec.add_development_dependency "bundler", "~> 1.14"
   spec.add_development_dependency "rake", "~> 10.0"
   spec.add_development_dependency "minitest", "~> 5.0"
+  spec.add_development_dependency "yard"
 end

data/lib/fa.rb

@@ -1,11 +1,18 @@
 require "fa/version"
 require "fa/ffi"
 
+# Namespace for the libfa bindings
 module Fa
+  # A generic error encountered during an fa operation
   class Error < StandardError; end
 
+  # An operation in libfa failed because it could not allocate enough
+  # memory
   class OutOfMemoryError < Error; end
 
+  # The class representing a finite automaton. It contains a pointer to a
+  # +struct fa+ from +libfa+ and provides Ruby wrappers for the various
+  # +libfa+ operations.
   class Automaton < ::FFI::AutoPointer
     attr_reader :faptr
 
@@ -13,54 +20,130 @@ module Fa
       @faptr = faptr
     end
 
+    # Minimizes this automaton in place. Uses either Hopcroft's or
+    # Brzozowski's algorithm. Due to a stupid design mistake in +libfa+,
+    # the algorithm is selected through a global variable. It defaults to
+    # Hopcroft's algorithm though.
+    #
+    # @return [Fa::Automaton] this automaton
     def minimize
       r = FFI::minimize(faptr)
       raise Error if r < 0
       self
     end
 
+    # Concatenates +self+ with +other+, corresponding to +SO+. Neither
+    # +self+ nor +other+ will be modified.
+    #
+    # @param [Fa::Automaton] other
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the concatenation of +self+ and +other+
     def concat(other)
       from_ptr( FFI::concat(faptr, other.faptr) )
     end
 
+    # Produces the union of +self+ with +other+, corresponding to
+    # +S|O+. Neither +self+ nor +other+ will be modified.
+    #
+    # @param [Fa::Automaton] other
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the union of +self+ and +other+
     def union(other)
       from_ptr( FFI::union(faptr, other.faptr) )
     end
 
+    # Produces an iteration of +self+, corresponding to
+    # +S{min,max}+. +self+ will not be modified.
+    #
+    # @param [Int] min the minimum number of matches
+    # @param [Int] max the maximum number of matches, use -1 for an
+    #              unlimited number of matches
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def iter(min, max)
       from_ptr( FFI::iter(faptr, min, max) )
     end
 
+    # Produces an iteration of any number of +self+, corresponding to
+    # +S*+. +self+ will not be modified.
+    #
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def star
       iter(0, -1)
     end
 
+    # Produces an iteration of at least one +self+, corresponding to
+    # +S\++. +self+ will not be modified.
+    #
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def plus
       iter(1, -1)
     end
 
+    # Produces an iteration of zero or one +self+, corresponding to
+    # +S?+. +self+ will not be modified.
+    #
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def maybe
       iter(0, 1)
     end
 
+    # Produces the intersection of +self+ and +other+. Neither +self+ nor
+    # +other+ will be modified. The resulting automaton will match all
+    # strings that simultaneously match +self+ and +other+,
+    #
+    # @param [Fa::Automaton] other
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def intersect(other)
       from_ptr( FFI::intersect(faptr, other.faptr) )
     end
 
+    # Produces the difference of +self+ and +other+. Neither +self+ nor
+    # +other+ will be modified. The resulting automaton will match all
+    # strings that match +self+ but not +other+,
+    #
+    # @param [Fa::Automaton] other
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def minus(other)
       from_ptr( FFI::minus(faptr, other.faptr) )
     end
 
+    # Produces the complement of +self+. +self+ will not be modified. The
+    # resulting automaton will match all strings that do _not_ match
+    # +self+.
+    #
+    # @raise OutOfMemoryError if +libfa+ fails to allocate memory
+    # @return [Fa::Automaton] the iterated automaton
     def complement
       from_ptr( FFI::complement(faptr) )
     end
 
+    # Returns whether +self+ and +other+ match the same set of strings
+    #
+    # @param [Fa::Automaton] other
+    # @return [Boolean]
     def equals(other)
       r = FFI::equals(faptr, other.faptr)
       raise Error if r < 0
       return r == 1
     end
 
+    # Returns whether +self+ and +other+ match the same set of strings
+    #
+    # @param [Fa::Automaton] other
+    # @return [Boolean]
+    def ==(other); equals(other); end
+
+    # Returns whether +self+ matches all the strings that +other+
+    # matches. +self+ may match more strings than that.
+    #
+    # @param [Fa::Automaton] other
+    # @return [Boolean]
     def contains(other)
       # The C function works like fa1 <= fa2, and not how the
       # Ruby nomenclature would suggest it, so swap the arguments
@@ -69,20 +152,48 @@ module Fa
       return r == 1
     end
 
-    def is_basic(basic)
+    # Returns whether +other+ matches all the strings that +self+
+    # matches. +other+ may match more strings than that.
+    #
+    # @param [Fa::Automaton] other
+    # @return [Boolean]
+    def <=(other); other.contains(self); end
+
+    # Returns whether +self+ is the empty, epsilon or total automaton
+    #
+    # @param [:empty, :epsilon, :total] kind
+    # @return [Boolean]
+    def is_basic(kind)
       # FFI::is_basic checks if the automaton is structurally the same as
       # +basic+; we just want to check here if they accept the same
       # language. If is_basic fails, we therefore check for equality
-      r = FFI::is_basic(faptr, basic)
+      r = FFI::is_basic(faptr, kind)
       return true if r == 1
-      return equals(Fa::make_basic(basic))
+      return equals(Fa::make_basic(kind))
     end
 
+    # Returns whether +self+ is the empty automaton, i.e., matches no words
+    # at all
+    # @return [Boolean]
     def empty?; is_basic(:empty); end
+
+    # Returns whether +self+ is the epsilon automaton, i.e., matches only
+    # the empty string
+    # @return [Boolean]
     def epsilon?; is_basic(:epsilon); end
+
+    # Returns whether +self+ is the total automaton, i.e., matches all
+    # possible words.
+    # @return [Boolean]
     def total?; is_basic(:total); end
 
-    def as_regexp
+    # Return the representation of +self+ as a regular expression. Note
+    # that that regular expression can look pretty complicated, even for
+    # seemingly simple automata. Sometimes, minimizing the automaton before
+    # turning it into a string helps; sometimes it doesn't.
+    #
+    # @return [String] the regular expression for +self+
+    def to_s
       rx = ::FFI::MemoryPointer.new :string
       rx_len = ::FFI::MemoryPointer.new :size_t
       r = FFI::as_regexp(faptr, rx, rx_len)
@@ -97,21 +208,36 @@ module Fa
 
     :private
     def from_ptr(ptr)
-      raise OutOfMemoryError if ptr.nil?
+      raise OutOfMemoryError if ptr.null?
       Automaton.new(ptr)
     end
   end
 
+  # Compiles +rx+ into a finite automaton
+  # @param [String] rx a regular expression
+  # @return [Fa::Automaton] the finite automaton
   def self.compile(rx)
     faptr = ::FFI::MemoryPointer.new :pointer
     r = FFI::compile(rx, rx.size, faptr)
-    raise Error if r < 0
+    raise Error if r != 0 # REG_NOERROR is 0, at least for glibc
     Automaton.new(faptr.get_pointer(0))
   end
 
+  # Compiles +rx+ into a finite automaton
+  # @param [String] rx a regular expression
+  # @return [Fa::Automaton] the finite automaton
+  def self.[](rx)
+    compile(rx)
+  end
+
+  # Makes a basic finite automaton, either an empty, epsilon, or total
+  # finite automaton. Those match no words, only the empty word, or all
+  # words.
+  # @param [:empty, :epsilon, :total] kind
+  # @return [Fa::Automaton] the finite automaton
   def self.make_basic(kind)
     faptr = FFI::make_basic(kind)
-    raise OutOfMemoryError if faptr.nil?
+    raise OutOfMemoryError if faptr.null?
     Automaton.new(faptr)
   end
 end

data/lib/fa/version.rb

@@ -1,3 +1,3 @@
 module Fa
-  VERSION = "0.1.0"
+  VERSION = "0.1.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fa
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - 'David Lutterkort
@@ -68,6 +68,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: |
   Bindings for libfa, a library to manipulate finite automata. Automata are
   constructed from regular expressions, using extended POSIX syntax, and make