mucgly 0.0.1

data/CHANGELOG.rdoc

@@ -0,0 +1,3 @@
+= Version history
+
+[0.0.1] Initial version.

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2014 tero.isannainen@gmail.com
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,292 @@
+= Mucgly
+
+== Introduction
+
+Mucgly is a macro expander for inline macros that exist in the middle of
+body text. The macros are mostly regular Ruby code, but a few special
+commands is also provided.
+
+A very simple example:
+  Adding 1 + 3 results: -<write (1+3).to_s>-
+
+After macro expansion the results is:
+  Adding 1 + 3 results: 4
+
+By default macro starts with "-<" and ends with ">-". These limiters
+are called hooks, hookbeg and hookend respectively. The code between
+hooks is regular Ruby code. The "write" call writes to selected IO
+stream.
+
+Ruby code is executed within a class instance reserved for this
+purpose. The instance is called the Execution Environment. It enables
+values from one macro to be visible in others.
+
+Previous example with multiple macros:
+  -<@a = 1>--<@b = 3>-\
+  Adding 1 + 3 results: -<write (@a+@b).to_s>-
+
+Result is exactly the same as in the previous execution. The first
+macro "-<@a = 1>-" produces no output, it just sets the variable "@a"
+to "1". Second macro is similar. The default "escape" character is
+"\\". When placed before newline character, it "eats" the newline and
+nothing is output. Thus the first line outputs nothing.
+
+The second line refers to settings from previous macros. Instance
+variables has to be used to maintain the data between macro calls (due
+to instance evaluation).
+
+
+== Features
+
+* User settable hooks to define macro boundaries. Can be set from
+  command line, configuration files, or from macro file.
+
+* Multiple sources for configuration: default config, environment
+  variable, command line.
+
+* Multiple passes for macro file.
+
+* Convention based output file naming.
+
+* Multiple convenience functions for macros to use.
+
+* Macro file introspection: line number, file name
+
+* Output stream de-muxing.
+
+* Many special commands: include, source, etc...
+
+
+
+== Applications
+
+* Replacement for M4 macro processor.
+
+* Code generation.
+
+* Document formatting.
+
+* Etc.
+
+
+
+== Special characters
+
+=== Hooks
+
+Hooks start and end the macro definition. By default hookbeg is "-<"
+and hookend is ">-". These values are not easily conflicting with the
+macro file body text.
+
+If literal hook string is required, the escape character should be
+place before the hook. For example "\-<" will produce literal "-<" to
+the output.
+
+User can set the hooks to whatever string value desired. The hooks can
+also have the same value. However nested macros are not possible
+then. Hooks can also be the same as the escape character, but this
+makes usage somewhat complicated.
+
+=== Escape
+
+Default escape character is "\\". It can be used to escape hooks,
+cancel space (" ") output, and cancel newline ("\\n") output.
+
+User can set the escape character to any character. Only single
+character value is accepted.
+
+=== Multipass
+
+If the macro starts with "#" character, the macro is not expanded. One
+"#" character is removed and the rest of the macro is output as it
+is. Each pass will remove one "#" character and when all are removed,
+the macro is evaluated. Usually two passes are enough and one "#"
+character is used to save a macro to the later pass.
+
+Multipass can be used for example to create a Table Of Contents. First
+pass collects information about document content and second pass will
+insert the Table Of Contents.
+
+Example, with functions (insert_toc, header...) defined elsewhere:
+  -<#insert_toc>-
+  -<header(1, "My header 1")>-
+  Paragraph1
+  -<header(2, "My header 1.1")>-
+  Paragraph2
+  -<header(1, "My header 2")>-
+  Paragraph3
+
+
+== Special commands
+
+In addition to regular Ruby code the macros are allowed to include so
+called Special Commands. These commands start with the ":" characters
+and with ".".
+
+Example:
+  ...
+  -<:hook [ ]->\
+  ...
+
+Would change the hookbeg and hookend to "[" and "]" respectively. One
+special command is allowed per macro and it can't be mixed normal Ruby
+code. Command name is separated by ":" in the beginning and by space
+(" ") in the end. The rest of the macro is taken as argument to the
+macro. Note that a command without an argument has to end with space
+as well.
+
+List of ":" style special commands:
+
+[include] Include reverts the input stream to the file given in the
+          argument. Command can be used to multiplex multiple files
+          into one, for example.
+
+[output] Select a new output file. Use with "close" command, when the
+         new output file is ready.
+
+[comment] Comment is used to add comments into the macro file. No
+          output is produced from comment macros.
+
+[source] Source reads a plain Ruby file into the Execution Environment.
+
+[hook] Sets both hookbeg and hookend to values separated by space. The
+       new hook values are valid immediately after the enclosing
+       macro.
+
+[hookbeg] Sets hookbeg.
+
+[hookend] Sets hookend.
+
+[escape] Sets escape character.
+
+[exit] Aborts the macro file.
+
+Variable value reference command starts with "." and is followed by
+the (instance) variable name.
+
+For example:
+  ... -<.my_name>- ...
+
+Would write out the value of the "@my_name" variable, thus it is equal
+to writing:
+  ... -<write @my_name>- ...
+
+
+
+== API methods
+
+API methods are instance methods of the Execution Environment,
+i.e. they are predefined and visible for the macro code. These methods
+are organized into two categories: Published and Hidden.
+
+=== Published methods
+
+Published methods are the most commonly used methods in macros. These are:
+
+[write] Writes string into selected output IO.
+[puts] Writes string (+newline) into selected output IO.
+[source] Sources a plain Ruby file.
+
+
+=== Hidden methods
+
+Hidded methods start with underscore ("_"). The naming is used to
+prevent poluting the Execution Environment's namespace.
+
+[_processFilePair] Process a pair of files. First half is input file
+                   and second is output file. File pair argument is an
+                   Array of two ([<fileI>, <fileO>]).
+
+[_processFilePairs] Uses "_processFilePair" to process a list of file pairs.
+
+[_openInput] Sets input stream to given file. When this file is
+             closed, the input stream is reverted to the original
+             file.
+
+[_openOutput] Sets output stream to given file. When this file is
+              closed, the input stream is reverted to the original
+              file.
+
+[_openString] Open a StringIO file for output.
+
+[_pushInput] Makes given file as top input stream. When this stream is
+             closed, the input stream is reverted to the original
+             file.
+
+[_pushOutput] Makes given file as top output stream. When this stream
+              is closed, the output stream is reverted to the original
+              file.
+
+[_closeInput] Close input stream.
+
+[_closeOutput] Close output stream.
+
+[_ofilename] Output file name as String.
+
+[_olinenumber] Output file line number.
+
+[_ifilename] Input file name as String.
+
+[_ilinenumber] Input file line number.
+
+[_eval] Perform instance_eval on the given argument.
+
+[_inIO] Handle to input stream (get/set).
+
+[_outIO] Handle to output stream (get/set).
+
+[_separators] Handle to Separators object. Separotors object includes
+              the "escapeChar", "hookBegChars", and "hookEndChars"
+              setter and getter methods.
+
+
+== Command line interface
+
+Please execute:
+  mucgly -h
+
+for an overview of the command line options.
+
+
+=== Input and Output files
+
+Input files can be listed to "-i" option. The default input stream for
+Mucgly is STDIN.
+
+Output files can be listed to "-o" option. The default output stream
+for Mucgly is STDOUT.
+
+If "-o" option is given without arguments and the input file list
+includes ".rx" in each name, the output file list is created by
+extracting ".rx" from each input file name.
+
+For example with:
+   mucgly -i foo.rx.txt bar.rx.txt -o
+
+Two files, foo.txt and bar.txt, would be created. This holds true
+unless the output streams are manipulated with macro commands.
+
+If the number of input and output files match, then they are used in pairs.
+
+
+== Configuration
+
+Mucgly checks for the existance of the "MUCGLY" environment
+variable. If the variable exists, the file in variable value is read
+in as plain Ruby.
+
+User configuration is searched from the users home directory:
+  $HOME/.mucgly
+It is read as plain Ruby file if it exists.
+
+Configuration files can also be given on command line using the "-c"
+option. These settings can be used to override the settings above.
+
+Additionally plain Ruby code can be given straight on the command line
+with "-l" option.
+
+
+== More information
+
+Check out the "test" directory within installation for detailed
+examples about usage.

data/Rakefile

@@ -0,0 +1,29 @@
+require 'rake/testtask'
+
+Rake::TestTask.new do |t|
+    t.libs << 'test'
+    t.libs << 'lib'
+end
+
+desc "Run tests"
+task :default => :test
+
+task :cleanup_test do
+    sh "rm -rf test/test"
+end
+
+task :build => :doc do
+    sh "gem build mucgly.gemspec"
+end
+
+task :doc do
+    sh "yardoc lib/* - README.rdoc CHANGELOG.rdoc"
+end
+
+task :publish do
+    if Dir.glob('mucgly-*gem').length == 1
+        sh "gem push mucgly*.gem"
+    else
+        raise "Multiple gems in the directory..."
+    end
+end

data/bin/mucgly

@@ -0,0 +1,220 @@
+#!/usr/bin/env ruby
+
+require 'como'
+include Como
+require_relative '../lib/easyfile'
+require_relative '../lib/mucgly'
+
+
+# Define command line arguments:
+Spec.command( "mucgly", "Tero Isannainen", "2009, 2013",
+   [
+    [ :opt_multi, "inputs", "-i", "Input file (default: <STDIN>)." ],
+    [ :opt_multi, "configs", "-c", "Ruby configure file(s)" ],
+    [ :opt_any, "output", "-o", "Output file (default: <STDOUT>)." ],
+    [ :opt_multi, "cl_cmds", "-l", "Command line configuration(s)." ],
+    [ :opt_any, "multipass", "-m", "Multipass rounds (default without arg: 2)." ],
+    [ :opt_single, "beg_sep", "-sb", "Set hookbeg separator (default: '-<')." ],
+    [ :opt_single, "end_sep", "-se", "Set hookend separator (default: '->')." ],
+    [ :opt_single, "seps", "-s", "Set both hookbeg and hookend separators." ],
+    [ :opt_single, "esc_char", "-e", "Set escape character (default: '\\')." ],
+    [ :opt_single, "all_sep", "-sa", "Set all separators and escape." ],
+    [ :switch, "no_inits", nil, "No initializations." ],
+    [ :switch, "no_user_init", nil, "No user initialization." ],
+    [ :switch, "no_env_init", nil, "No env initialization." ],
+    [ :switch, "warn_only", "-w", "Warnings only." ],
+    [ :switch, "debug", "-d", "Display debug messages." ],
+   ], { :tab => 15, } )
+
+
+filePairs = []
+
+
+# Set input streams.
+unless Opt['inputs'].given
+    filePairs.push [ '<STDIN>', nil ]
+else
+    Opt['inputs'].value.each do |i|
+        filePairs.push [ i, nil ]
+    end
+end
+
+
+# Set output streams.
+if Opt['output'].given
+
+    if Opt['output'].value.empty?
+
+        # Map all input files to output files where '.rx' is removed
+        # in the name.
+        
+        # Check that all inputs have '.rx' in the name.
+        noRx = false
+        name = nil
+        filePairs.each do |pair|
+            if /\.rx/.match( pair[0] )
+                pair[1] = pair[0].gsub( /\.rx/, '' )
+            else
+                noRx = true
+                name = pair[0]
+                break
+            end
+        end
+
+        if noRx
+            Mucgly::error( "Input file must have '.rx' in name\
+ when automapping: \"#{name}\"" )
+        end
+
+    elsif Opt['output'].value.length == filePairs.length
+
+        filePairs.each_index do |idx|
+            filePairs[ idx ][1] = Opt['output'].value[ idx ]
+        end
+        
+    else
+
+        Mucgly::error( "Input and output file counts does not match:\
+ #{Opt['output'].value.length}/#{filePairs.length}" )
+        
+    end
+
+else
+
+    # Default output, STDOUT.
+    filePairs[0][1] = '<STDOUT>'
+
+end
+
+
+execEnv = Mucgly::Env.new
+execEnv._separators = Mucgly::Separators.new
+
+
+# Change separators:
+if Opt['beg_sep'].given
+    execEnv._separators.hookBegChars = Opt['beg_sep'].value
+end
+
+if Opt['end_sep'].given
+    execEnv._separators.hookEndChars = Opt['end_sep'].value
+end
+
+if Opt['seps'].given
+    rest = Opt['seps'].value
+
+    if rest.split.length == 2
+        execEnv._separators.hookBegChars = rest.split[0]
+        execEnv._separators.hookEndChars = rest.split[1]
+    else
+        execEnv._separators.hookBegChars = rest
+        execEnv._separators.hookEndChars = rest
+    end
+end
+
+
+# Change escape:
+if Opt['esc_char'].given
+    execEnv._separators.escapeChar = Opt['esc_char'].value
+end
+
+
+# Change all specials.
+if Opt['all_sep'].given
+    execEnv._separators.hookBegChars = Opt['all_sep'].value
+    execEnv._separators.hookEndChars = Opt['all_sep'].value
+    execEnv._separators.escapeChar = Opt['all_sep'].value
+end
+
+
+# Read Mucgly env config.
+unless ( Opt['no_env_init'].given || Opt['no_inits'].given )
+    p = ENV["MUCGLY"]
+    execEnv.source( p ) if p
+end
+
+
+# Read default user config.
+unless ( Opt['no_user_init'].given || Opt['no_inits'].given )
+    userInit = "#{ENV['HOME']}/.mucgly"
+    if FileTest.exist? userInit
+        # load userInit
+        execEnv.source( userInit )
+    end
+end
+
+
+# Process provided configure files.
+Opt['configs'].given do |value|
+    value.each do |i|
+        execEnv.source( i )
+    end
+end
+
+
+# Process command line.
+Opt['cl_cmds'].given do |value|
+    value.each do |i|
+        execEnv.eval( i )
+    end
+end
+
+
+# Multipass processing.
+if Opt['multipass'].given
+
+    if Opt['multipass'].value.empty?    
+        passes = 2
+    else
+        passes = Opt['multipass'].value[0].to_i
+    end
+
+
+    if passes <= 1
+
+        execEnv._processFilePairs( filePairs )
+
+    else
+
+        filePairs.each do |filePair|
+
+            pass = 0
+            
+            # Pair (ping-pong) of StringIO handles.
+            sio = [nil]*2
+
+            while pass < passes
+                
+                # Use StringIO to pass content between passes.
+                if pass == 0
+                    # First pass.
+                    execEnv._openInput( filePair[ 0 ] )
+                    sio[(pass+0)%2] = execEnv._openString( "_pass0" )
+                    sio[(pass+1)%2] = execEnv._openString( "_pass1" )
+                    execEnv._outIO = sio[(pass+0)%2]
+                elsif pass >= ( passes-1 )
+                    # Last pass.
+                    execEnv._inIO = sio[(pass+1)%2]
+                    execEnv._inIO.rewind
+                    execEnv._openOutput( filePair[ 1 ] )
+                else
+                    # Middle pass.
+                    execEnv._inIO = sio[(pass+1)%2]
+                    execEnv._inIO.rewind
+                    execEnv._outIO = sio[(pass+0)%2]
+                end
+                
+                Mucgly::MucglyFile.new( execEnv )
+                pass += 1
+                
+            end
+            
+        end
+
+    end
+
+else
+
+    execEnv._processFilePairs( filePairs )
+
+end