rweb 0.1.0

data/docs/rtangle.txt

@@ -0,0 +1,90 @@
+RTANGLE
+=======
+This is a simple utility that acts as a wrapper around RWEB's tangle and,
+instead of executing the resulting code, either prints it to stdout or saves it
+to a given file. Note that RTangle is itself written in RWEB's executable format
+with the boilerplate before the __END__ statement and the actual program
+document afterwards.
+
+The mainline code is quite simple:
+
+RWEB Mainline
+-------------
+> {<<global requires>>}
+> 
+> {<<command line processing>>}
+> 
+> {<<tangling>>}
+> 
+> {<<main>>}
+
+The only require we need to introduce is to require RWEB itself. This is not, of
+course, necessary if we execute the RWEB document directly. Since, however, it
+is highly likely that we may at some point wish to tangle the RTangle utility
+itself into a Ruby source file, we should, pro forma, require the library
+anyway.
+
+global requires
+---------------
+> require 'rweb'
+
+The tangling stage itself is a very simple function which takes any kind of IO
+object for input, tangles it, then places the resulting string into the output,
+itself an IO object of any kind. It also doesn't care in the slightest exactly
+what kind of IO object is being used, thus suited to application as a filter in
+a potentially longer chain of tools.
+
+tangling
+--------
+> def rtangle(io_in, io_out)
+>   io_out << RWEB.tangle(io_in)
+> end
+
+As can be seen, the true heavy lifting is done inside of the RWEB module. This
+utility is a hyper-simple shell around it by comparison.
+
+The mainline function is pretty simple as well. It just takes the arguments from
+the command line and passes them to the command line processor blindly,
+accepting in return a pair of IO objects -- one for input, the other for output.
+It then calls tangle with these objects, wrapping in a begin/ensure clause to
+make sure the IO objects are closed before exiting. The only additional step it
+takes is a call to the function find_block_begin which shadows the IO object
+used for input and silently sweeps away the stuff before __END__ in a document
+if it exists in the first 20 lines.
+
+main
+----
+> io_in, io_out = process_argv ARGV
+> begin
+>   rtangle(io_in, io_out)
+> ensure
+>   io_in.close
+>   io_out.close
+> end
+
+All that's left is the command line processor.
+
+command line processing
+-----------------------
+> def process_argv args
+>   io_in = STDIN
+>   io_out = STDOUT
+>   if args.length > 2
+>     {<<display usage>>}
+>   end
+>   arg = args.shift
+>   io_in = File.open(arg, "r") if arg
+>   arg = args.shift
+>   io_out = File.open(arg, "w") if arg
+>   return io_in, io_out
+> end
+
+The usage message is straightforward and broken out only to reduce code clutter.
+
+display usage
+-------------
+> puts "Incorrect command line."
+> puts "Usage:"
+> puts "  rtangle [input_file [output_file]]"
+> puts
+> puts "Files default to stdin and stdout respectively."

data/docs/rweave.txt

@@ -0,0 +1,90 @@
+RWEAVE
+======
+This is a simple utility that acts as a wrapper around RWEB's weave and, instead
+of executing the resulting code, either prints it to stdout or saves it to a
+given file. Note that RWeave is itself written in RWEB's executable format with
+the boilerplate before the __END__ statement and the actual program document
+afterwards.
+
+The mainline code is quite simple:
+
+RWEB Mainline
+-------------
+> {<<global requires>>}
+> 
+> {<<command line processing>>}
+> 
+> {<<weaving>>}
+> 
+> {<<main>>}
+
+The only require we need to introduce is to require RWEB itself. This is not, of
+course, necessary if we execute the RWEB document directly. Since, however, it
+is highly likely that we may at some point wish to weave the RWeave utility
+itself into a Ruby source file, we should, pro forma, require the library
+anyway.
+
+global requires
+---------------
+> require 'rweb'
+
+The weaving stage itself is a very simple function which takes any kind of IO
+object for input, weaves it, then places the resulting string into the output,
+itself an IO object of any kind. It also doesn't care in the slightest exactly
+what kind of IO object is being used, thus suited to application as a filter in
+a potentially longer chain of tools.
+
+weaving
+-------
+> def rweave(io_in, io_out)
+>   io_out << RWEB.weave(io_in)
+> end
+
+As can be seen, the true heavy lifting is done inside of the RWEB module. This
+utility is a hyper-simple shell around it by comparison.
+
+The mainline function is pretty simple as well. It just takes the arguments from
+the command line and passes them to the command line processor blindly,
+accepting in return a pair of IO objects -- one for input, the other for output.
+It then calls weave with these objects, wrapping in a begin/ensure clause to
+make sure the IO objects are closed before exiting. The only additional step it
+takes is a call to the function find_block_begin which shadows the IO object
+used for input and silently sweeps away the stuff before __END__ in a document
+if it exists in the first 20 lines.
+
+main
+----
+> io_in, io_out = process_argv ARGV
+> begin
+>   rweave(io_in, io_out)
+> ensure
+>   io_in.close
+>   io_out.close
+> end
+
+All that's left is the command line processor.
+
+command line processing
+-----------------------
+> def process_argv args
+>   io_in = STDIN
+>   io_out = STDOUT
+>   if args.length > 2
+>     {<<display usage>>}
+>   end
+>   arg = args.shift
+>   io_in = File.open(arg, "r") if arg
+>   arg = args.shift
+>   io_out = File.open(arg, "w") if arg
+>   return io_in, io_out
+> end
+
+The usage message is straightforward and broken out only to reduce code clutter.
+
+display usage
+-------------
+> puts "Incorrect command line."
+> puts "Usage:"
+> puts "  rweave [input_file [output_file]]"
+> puts
+> puts "Files default to stdin and stdout respectively."

data/lib/rweb.rb

@@ -0,0 +1,233 @@
+# RWEB.RB -- A library for literate programming in Ruby
+# Copyright (C) 2007 Michael T. Richter <ttmrichter@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; Version 2, June 1991 (and no other).
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program (in the file COPYING); if not, write to the Free
+# Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307,
+# USA
+
+# Ironically, this library cannot be written in the RWEB style because it
+# *implements* the RWEB style of programming for Ruby.
+
+=begin rdoc
+Our default directives are contained in a Hash object. We would like to be able
+to copy that set of defaults without having the original changed, so we make up
+for a slight design deficiency in the Ruby standard library by providing a
+(somewhat limited) deep copy method.
+=end
+class Object
+
+  # Deep copy an object by first marshalling, then unmarshalling it.
+  def deep_copy
+    Marshal.load(Marshal.dump(self))
+  end
+  
+end
+
+# The default values for directives.  The default style for weaving is plain
+# text and the default title is "Untitled".
+DEFAULT_DIRECTIVES = { 
+  :style => "Plain",
+  :title => "Untitled"
+}
+
+# Take a provided IO stream and disassemble it into a set of directives, an
+# intermediate document form and a collection of (unexpanded) chunks.  Any
+# boilerplate from a self-tangling script is passed over.
+def disassemble(lines)
+  directives, lines = find_directives(strip_boilerplate(lines), DEFAULT_DIRECTIVES.deep_copy)
+  docs, chunks = get_docs_and_chunks(lines)
+  return directives, docs, chunks
+end
+
+# Strip the boilerplate from self-tangling scripts before processing the rest of
+# the document.
+def strip_boilerplate(lines)
+  # if we find the __END__ directive in the lines, we return all the lines after
+  # that point
+  h = 0
+  if lines.find { |l| h += 1 ; l == "__END__\n" }
+    return lines[h..-1]
+  # otherwise this is not a self-tangling script so all the lines count
+  else
+    return lines
+  end
+end 
+
+# Take the IO stream and search for any directives.  The technique is to read
+# the IO object one line at a time, passing over any blank lines and searching
+# for any directives.  The first non-blank, non-directive line is assumed the
+# beginning of the documentation block and reinserted into the stream.
+def find_directives(lines, directives)
+  while line = lines.shift
+    # skip blank lines
+    if %r{^[:blank:]*$}.match line
+      next
+    # process directive lines
+    elsif %r{^\{([[:print:]]+)[[:blank:]]+=>[[:blank:]]+([[:print:]]+)\}\n$}.match line
+      set_directive $1, $2, directives if directives
+      next
+    # restore the first line of documentation, return the directives
+    else
+      return directives, lines.unshift(line)
+    end
+  end
+end
+
+# Take a given directive key/value pair and set if if valid.  Valid keys are, at
+# this point, "style" and "title".  "style" may have the values "Plain" or\
+# "XHTML" (not currently implemented).
+def set_directive(key, value, directives)
+  case key
+    when "style"
+      if ["Plain", "XHTML"].include?(value)
+        directives[:style] = value
+      else
+        raise ArgumentError, "unknown style #{value}"
+      end
+    when "title"
+      directives[:title] = value
+    else
+      raise ArgumentError, "unknown directive '#{m[1]}''"
+  end
+end
+
+# Break the main body of an RWEB document into the documentation intermediate
+# form and a collection of unexpanded chunks.
+def get_docs_and_chunks(lines)
+  docs = [] ; chunks = {}
+  # directives are already gone, so we start in documentation
+  while line = lines.shift
+    # if a chunk starts, push the name of the chunk into the docs and then
+    # extract the chunk
+    if %r{^\<\<([[:print:]]*)\{[[:blank:]]*$}.match line
+      name = $1.strip
+      docs.push "{<<#{name}>>}\n"
+      get_chunk lines, name, chunks, docs
+    # otherwise just push the line into the accumulated documentation
+    else
+      docs.push line
+    end
+  end
+  return docs, chunks
+end
+
+# Extract a detected chunk of code.
+def get_chunk(lines, name, chunks, docs)
+  chunks[name] = [] unless chunks.has_key?(name)
+  while line = lines.shift
+    # if we find the closing tag of a chunk, drop it and return
+    if %r{^\}\>\>[[:blank:]]*$}.match line
+      return
+    # otherwise add the current line to the named chunk and add the code,
+    # marked, to the documentation array for weaving
+    else
+      chunks[name].push line
+      docs.push "}}} #{line}" # flag the code in the document stream
+    end
+  end
+  raise RuntimeError, "chunk '#{name}' not closed by end of input"
+end
+
+# Take a raw chunk and expand all internal references.
+def expand_chunk(chunk, chunks, prefix = "", chain = [])
+  out = []
+  while line = chunk.shift
+    # if we spot a chunk reference, validate it and expand its contents into the
+    # current chunk
+    if %r{^([[:print:][:blank:]]*)\{\<\<([[:print:]]+)\>\>\}([[:print:][:blank:]]*)$}.match line
+      name = $2.strip
+      # Illegal conditions are:
+      # 1. A chunk referenced which has not been defined.
+      raise RuntimeError, "chunk '#{name}' referenced without definition" unless chunks.has_key?(name)
+      # 2. A chunk reference with anything but whitespace in front or behind.
+      raise RuntimeError, "invalid prefix '#{$1.strip}' when referencing chunk '#{name}'" unless $1.strip == ""
+      raise RuntimeError, "invalid suffix '#{$3.strip}' when referencing chunk '#{name}'" unless $3.strip == ""
+      # 3. A circular expansion of a chunk.
+      raise RuntimeError, "circular expansion of chunk '#{name}'\nexpansion chain is:\n=> #{chain.join("\n=> ")}\n" if chain.include?(name)
+      chain.push name
+      out.push "#{prefix + $1}# #{name}\n"
+      out.push "#{prefix + $1}# #{"-" * name.length}\n"
+      out += expand_chunk(chunks[name], chunks, prefix + $1, chain)
+      chain.pop
+    else
+      out.push prefix + line
+    end
+  end
+  out
+end
+
+# Generate plaintext documentation from the documentation intermediate form.
+# The following actions are taken:
+# 1. The title is placed at the top and double-underlined.
+# 2. Documentation lines are passed through unaltered.
+# 3. Chunk names are emitted without the tags and with single-underlining.
+# 4. Chunk contents are emitted with "> " prepended.
+def generate_plain_document(docs, directives)
+  raise RuntimeError, "style directive not set to Plain" unless directives[:style] == "Plain"
+  out = []
+  title = directives[:title]
+  
+  out << ["#{title}\n","#{"="*title.length}\n"]
+  docs.each do |line|
+    # if we find a chunk reference, extract the name and underline it, the
+    # mainline chunk being given the title "RWEB Mainline"
+    if %r{^\{\<\<([[:print:]]*)\>\>\}\n$}.match line
+      line = $1
+      line = "RWEB Mainline" if line == ""
+      out << ["#{line}\n","#{"-"*line.length}\n"]
+    # if we find a line of code, emit it with added "chicken feet" to the front.
+    elsif %r{^\}\}\} (.*)\n$}.match line
+      out << ["> #{$1}\n"]
+    # otherwise just emit the line
+    else
+      out << line
+    end
+  end
+  out.to_s
+end
+
+# Generate XHTML documentation from the documentation intermediate form.
+# NOT YET IMPLEMENTED!
+def generate_xhtml_document(docs, directives)
+  raise RuntimeError, "style directive not set to XHTML" unless directives[:style] == "XHTML"
+  raise NotImplementedError, "document style not implemented"
+end
+
+=begin rdoc
+The RWEB module is the public interface to the RWEB library and contains only
+three publicly-accessible symbols.  First, the version of the library is
+provided (for build purposes).  Second, the RWEB.tangle function is exposed
+which takes an IO object and returns a string containing the tangled code ready
+for execution.  Third, the RWEB.weave function is exposed which takes an IO
+object and returns a string containing the documentation in the appropriate
+format.
+=end
+module RWEB
+
+  VERSION = "0.1.0"
+  
+  # Tangle an RWEB document and return the code as a string.
+  def RWEB.tangle(io)
+    directives, docs, chunks = disassemble(io.readlines)
+    raise RuntimeError, "no mainline chunk found in file" unless chunks.has_key?("")
+    "# #{directives[:title]}\n# #{"=" * directives[:title].length}\n\n" + expand_chunk(chunks[""], chunks).to_s
+  end
+
+  # Weave the RWEB document into another format according to directives and
+  # return as a string.
+  def RWEB.weave(io)
+    directives, docs, chunks = disassemble(io.readlines)
+    eval "generate_#{directives[:style].downcase}_document(docs, directives)"
+  end
+  
+end

data/tests/tc_rweb.rb

@@ -0,0 +1,57 @@
+require 'rweb'
+require 'stringio'
+require 'test/unit'
+
+class TestRWEB < Test::Unit::TestCase
+
+  def setup
+  
+@pure_plaintext = <<-END
+This is an RWEB document that contains nothing but documentation. There are no
+directives nor are there any code chunks. What comes out when weaving should be
+identical to what goes in except for adding a title.
+END
+    
+@plaintext_with_directives1 =<<-END
+{style => Plain}
+{title => My Title}
+This is an RWEB document that contains documentation and valid directives. There
+are no code chunks. What comes out when weaving should be identical to what goes
+in except for adding a title and removing directives.
+END
+    
+@plaintext_with_directives2 =<<-END
+{style => XHTML}
+{title => My Title}
+This is an RWEB document that contains documentation and valid directives, but
+with the style being an unimplemented one. There are no code chunks. What comes
+out when weaving should be identical to what goes in except for adding a title
+and removing directives.
+END
+
+  end
+    
+  def test_pure_plaintext
+    out = RWEB.weave(StringIO.new(@pure_plaintext))
+    out = out.split("\n")
+    assert_equal("Untitled", out.shift)
+    assert_equal("========", out.shift)
+    assert_equal(@pure_plaintext.split("\n"), out)
+  end
+  
+  def test_plaintext_with_directives
+    out = []
+    assert_nothing_raised {
+      out = RWEB.weave(StringIO.new(@plaintext_with_directives1))
+      out = out.split("\n")
+    }
+    assert_equal("My Title", out.shift)
+    assert_equal("========", out.shift)
+    assert_equal(@plaintext_with_directives1.split("\n")[2..-1], out)
+    
+    assert_raise(NotImplementedError) {
+      RWEB.weave(StringIO.new(@plaintext_with_directives2))
+    }
+  end
+  
+end