rawk 0.1.2 → 0.1.3

data/README.rdoc

@@ -10,9 +10,13 @@ Obviously, you can use ruby -e and {other magic}[http://code.joejag.com/2009/usi
 
 == Install
 
-Clone the git repo or download the file as a .tar or .zip.  The bin directory contains the rawk executable.  Does not require any gems (except rspec if you want to run the tests)
+From the command line
 
-I will package rawk as a gem when I find a spare moment.
+  gem install rawk
+
+Using bundler
+ 
+ gem "rawk", "~> 0.1.2"
 
 == Example
 
@@ -35,7 +39,7 @@ Creates the following output
 
 This can be written using rawk as
 
-  $ ls -ltr | bin/rawk '
+  $ ls -ltr | rawk '
     start {puts "Starting..."}
     every {|record| puts "#{record.cols[8]} #{record.cols[0]}"}
     finish {puts "done"} '
@@ -64,7 +68,7 @@ Runs before any lines are read from the input stream.  Equivalent to a BEGIN con
 
   every {|record| <code>}
 
-Runs once for each line of input data.  Yields an object of type Line (see below)
+Runs once for each line of input data.  Yields an object of type Record (see below)
 Equivalent to an anonymous block such as awk '{print $1}'
 
   finish {<code>}
@@ -74,7 +78,7 @@ Equivalent to an END condition in awk
 
 You can provide multiple blocks of code for each condition.
 
-  ls -ltr | head -2 | bin/rawk '
+  ls -ltr | head -2 | rawk '
     every {|record| puts 1}
     every {|record| puts 2} '
     
@@ -89,11 +93,11 @@ prints
 
 * Conditional blocks 
 
-== Lines
+== Records
 
-every yields an object of type Line which is subclass of String that adds a cols method to access columns.  The cols method returns an array of column values.
+every yields an object of type Record which is subclass of String that adds a cols method to access columns.  The cols method returns an array of column values.
 
-  echo "hello world" | bin/rawk 'every do |record| 
+  echo "hello world" | rawk 'every do |record| 
     puts "#{record.cols.length} columns: #{record.cols.join(",")}"
   end'
   
@@ -101,19 +105,25 @@ every yields an object of type Line which is subclass of String that adds a cols
 
 Note that cols is aliased to c for convenience
 
-    echo "hello world" | bin/rawk 'every do |record| 
+    echo "hello world" | rawk 'every do |record| 
       puts record.c[0]
     end'
     
     -> hello
+    
+In most cases you will be dealing with a few columns of data so Record provides functions that allow you to access columns the first 10 columns directly by position name.
 
+  echo hello world from me | rawk 'every {|r| puts "#{r.first} #{r.third}"}'
+  
+  -> hello from
+    
 == Functions, classes and other ruby stuff
 
-You can use ruby as normal before specifying any blocks.  For example...
+You can use ruby as normal.  For example...
 
 Functions
 
-  echo hello world | bin/rawk '
+  echo hello world | rawk '
     def print_first_column(record)
       puts record.cols.first
     end
@@ -121,7 +131,7 @@ Functions
   
 Classes
 
-  echo hello world | bin/rawk '
+  echo hello world | rawk '
     class Printer
       def self.print_first(record)
         puts record.cols.first
@@ -133,28 +143,63 @@ Requires and gems
 
 require works as you would expect although rubygems is not required by default.
 
-  echo "ruby" | bin/rawk '
+  echo "ruby" | rawk '
     require "rubygems"
     require "active_support/all"
     every {|record| puts record.cols.first.pluralize} '
   
   -> rubies
 
+== Variables and Scope
+
+Variables defined inside the condition blocks (start, every, finish) are local to that block.  Create a member variables to share state between blocks.
+
+  ls | tail -2 | rawk '
+    start do
+      local = "foo"
+      @shared = "bar"
+      puts "Starting with #{local}"
+    end
+    every {|record| puts "Running with #{@shared}"} '
+    
+  -> Starting with foo
+     Running with bar
+     Running with bar
+
 == Builtins
 
 rawk provides builtins as member variables.  You can change them as you see fit.
 
 @nr holds the current record number 
-  ls -ltr | head -2 | bin/rawk 'every {puts @nr}'
+  ls -ltr | head -2 | rawk 'every {puts @nr}'
     
 @fs specifies the field separator applied to each record
   
-  echo "foo.bar" | bin/rawk ' 
+  echo "foo.bar" | rawk ' 
     start {@fs="."}
     every {|record| puts "1: #{record.cols[0]} 2: #{record.cols[1]}"} '
   
   -> 1: foo 2: bar
   
+@rs specifies the record separator" character
+  * Defaults to newline
+  * Note that, unlike awk, @rs can only be set in the start block.  It cannot be changed "in flight"
+  
+  ksh print -n "foo.bar." | bin/rawk '
+    start {@rs = "."}
+    every {|r| puts r.cols.first} '
+    
+  -> foo
+     bar
+  
+NF: Keeps a count of the number of fields in an input record. The last field in the input record can be designated by $NF.
+* Each Record yielded by the every block has a '.nf' method
+* $NF can be coded as 'every {|record| record.cols.last}'  
+
+ echo "foo bar" | rawk 'every {|record| puts "#{record.nf} fields"}'
+ 
+ -> 2 fields
+  
 === Not supported (yet)
 
 I'm working on support for the following awk built-ins 
@@ -164,21 +209,11 @@ Contains the name of the current input-file.
 * Reading input data is not supported yet
 * When I add it, I'll add @filename as a member
 
-RS: 
-Stores the current "record separator" character. Since, by default, an input line is the input record, the default record separator character is a "newline".
-* Will be @rs
-* Currently, records are delimited by newline
-
 
 === Redundant
 
 The following awk built-ins are redundant in ruby
 
-NF: 
-Keeps a count of the number of fields in an input record. The last field in the input record can be designated by $NF.
-* NF can be coded as 'every {|record| record.cols.size}'
-* $NF can be coded as 'every {|record| record.cols.last}'
-
 OFS: 
 Stores the "output field separator", which separates the fields when Awk prints them. The default is a "space" character.
 * Ruby's string handling is far superior to awk's so there is no point in implementing a print routine
@@ -192,4 +227,15 @@ OFMT: Stores the format for numeric output. The default format is "%.6g".
 
 == Using rawk inside a ruby program
 
-Technically, you can require rawk and use it's classes directly.  However, it will be messy until I add gem packaging so I'll wait until you can install rawk as a gem before I go ahead an document it.
+Rawk code is evaluated within an instance of Rawk::Program.  You can use rawk within your programs as follows...
+
+  require 'rubygems'
+  require 'rawk'
+
+  data = "foo\nbar"
+  program = Rawk::Program.new(data)
+  
+  program.run do
+    every {|record| puts record.cols.first}
+  end
+  

data/bin/rawk

@@ -5,7 +5,8 @@ $: << File.join(PROJECT_DIR, 'lib')
 
 require 'rawk/rawk'
 
-DATA = STDIN
-DSL_CODE = ARGV[0]
-
-Rawk::Program.new(DATA).run(DSL_CODE)
+module Rawk
+  DATA = STDIN
+  DSL_CODE = ARGV[0]
+  Program.new(DATA).run(DSL_CODE) if DSL_CODE
+end

data/lib/rawk/rawk.rb

@@ -3,26 +3,27 @@ require 'observer'
 
 module Rawk  
   class Program    
-    attr_reader :fs
+    attr_reader :fs, :rs
     
     def initialize(io)
       @start, @every, @finish = Set.new, Set.new, Set.new
+      initialize_builtins!
       @input_stream = InputStream.new(io)
       @input_stream.add_observer(self)
-      initialize_builtins!
     end
     
     private
     def initialize_builtins!
       @fs = " "
       @nr = 0
+      @rs = "\n"
     end
     
     public
     def on_new_line
       @nr += 1
     end
-    alias :update :on_new_line
+    alias :update :on_new_line # required by Observer
     
     def run(code = "", &block)
       load!(code, &block)
@@ -53,8 +54,9 @@ module Rawk
     
     def execute_code!
       @start.each {|b| b.call}
-      @input_stream.each_line do |row|
-        @every.each {|b| b.call(Line.new(row, @fs))}
+      @input_stream.each_line(@rs) do |str|
+        record = Record.new(str, @fs, @rs)
+        @every.each {|b| b.call(record)}
       end
       @finish.each {|b| b.call}
     end
@@ -67,8 +69,8 @@ module Rawk
       @io = io
     end
     
-    def each_line
-      @io.each_line do |line| 
+    def each_line(separator)
+      @io.each_line(separator) do |line| 
         changed
         notify_observers
         yield line
@@ -76,9 +78,15 @@ module Rawk
     end
   end
   
-  class Line < String
-    def initialize(str, fs)
-      self.replace(str.chomp)
+  class Record < String    
+    def self.column_postion_accessor(name, position)
+      define_method(name.to_sym) do
+        cols[position]
+      end
+    end
+    
+    def initialize(str, fs, eor = "\n")
+      self.replace(str.chomp(eor))
       @fs = fs
     end
     
@@ -90,5 +98,11 @@ module Rawk
     def nf 
       cols.length
     end
+    
+    {:first => 0,:second => 1, :third => 2, :fourth => 3,:fifth => 4,
+     :sixth => 5,:seventh => 6,:eighth => 7,:ninth => 8, :tenth => 9
+     }.each do |name, column_position|
+      column_postion_accessor name, column_position
+    end
   end
 end

data/rawk.gemspec

@@ -2,7 +2,7 @@ require 'rubygems'
 
 SPEC = Gem::Specification.new do |s|
   s.name = "rawk"
-  s.version = '0.1.2'
+  s.version = '0.1.3'
   s.author = "Adrian Mowat"
   s.homepage = "https://github.com/mowat27/rawk"
   s.summary = "An awk-inspired ruby DSL"

data/spec/rawk/rawk_spec.rb

@@ -113,34 +113,56 @@ module Rawk
     end
     
     describe "support for standard awk built-in variables" do
-      it "calculates the current record num as nr" do
-        record_nums = []
-        @program.run do 
-          start {record_nums << @nr} 
-          every {record_nums << @nr}
-          finish {record_nums << @nr}
+      describe "@nr (current record)" do
+        it "holds the current record number" do
+          record_nums = []
+          @program.run do 
+            start {record_nums << @nr} 
+            every {record_nums << @nr}
+            finish {record_nums << @nr}
+          end
+          record_nums.should == [0,1,2,3,3]
         end
-        record_nums.should == [0,1,2,3,3]
       end
       
-      describe "fs" do
+      describe "@fs (field separator)" do
         it "holds the current field separator expression" do
           @program.fs.should == " "
         end
         it "is applied to each line of data" do
           data = "line"
-          Line.should_receive(:new).with(data, " ")
+          Record.should_receive(:new).with(data, " ", "\n")
           Program.new(data).run {every {|l| nil}}
         end
         it "can be changed by the user's program" do
           data = "line"
-          Line.should_receive(:new).with(data, ",").and_return("dummy")
+          Record.should_receive(:new).with(data, ",", "\n").and_return("dummy")
           Program.new(data).run do
             start {@fs = ','}
             every {|l| nil}
           end
         end
       end
+      
+      describe "@rs (record separator)" do
+        it "defaults to a newline" do
+          @program.rs.should == "\n"
+        end
+        it "can be changed by the user" do
+          data = "line1.line2."
+          Record.should_receive(:new).
+            with("line1.", " ", ".").once.ordered.
+            and_return("")
+          Record.should_receive(:new).
+            with("line2.", " ", ".").once.ordered.
+            and_return("")
+            
+          Program.new(data).run do
+            start {@rs = "."}
+            every {|rec| nil}
+          end
+        end
+      end
     end
   end
 end

data/spec/rawk/record_spec.rb

@@ -0,0 +1,58 @@
+require 'spec_helper'
+
+module Rawk
+  describe Record do
+    before do
+      @space = " "
+      @data = "a b c\n"
+      @record = Record.new(@data, @space)
+    end
+    it "is a string" do
+      @record.is_a?(String).should be_true
+    end
+    it "chomps itself on creation" do
+      @record.should == "a b c"
+    end
+    it "calculates and array of columns" do
+      @record.cols.should == ["a","b","c"]
+      @record.c.should == @record.cols
+    end
+    it "calculates the number of fields" do
+      @record.nf.should == 3
+    end
+    
+    context "accessing columns by name" do
+      before do
+        @record = Record.new("a b c d e f g h i j", @space)
+      end
+    
+      { :first => "a", :second => "b", :third => "c",
+        :fourth => "d", :fifth => "e", :sixth => "f",
+        :seventh => "g", :eighth => "h", :ninth => "i",
+        :tenth => "j" }.each do |method, expected|
+        it "finds the #{method} column value" do
+          @record.send(method).should == expected
+        end
+      end
+    end
+    
+    context "with a field separator ','" do
+      it "splits the line into columns using a comma" do
+        record = Record.new("a,b,c d", ",") 
+        record.c.should == ["a","b","c d"]
+      end
+    end
+    context "with a regular expression field separator" do
+      it "splits the line into columns using the regular expression" do
+        record = Record.new("a,b|c", /[,|]/)
+        record.c.should == ["a","b","c"]
+      end
+    end
+    context "with an explict end of line marker" do
+      it "chomps the end of line marker" do
+        record = Record.new("a b c.", @space, ".")
+        record.should == "a b c"
+      end
+    end
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rawk
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.1.3
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2011-08-14 00:00:00.000000000Z
+date: 2011-08-16 00:00:00.000000000Z
 dependencies: []
 description: ! 'An awk-inspired ruby DSL
 
@@ -27,8 +27,8 @@ files:
 - bin/rawk
 - lib/rawk/rawk.rb
 - rawk.gemspec
-- spec/rawk/line_spec.rb
 - spec/rawk/rawk_spec.rb
+- spec/rawk/record_spec.rb
 - spec/spec_helper.rb
 homepage: https://github.com/mowat27/rawk
 licenses: []
@@ -55,6 +55,6 @@ signing_key:
 specification_version: 3
 summary: An awk-inspired ruby DSL
 test_files:
-- spec/rawk/line_spec.rb
 - spec/rawk/rawk_spec.rb
+- spec/rawk/record_spec.rb
 - spec/spec_helper.rb

data/spec/rawk/line_spec.rb

@@ -1,37 +0,0 @@
-require 'spec_helper'
-
-module Rawk
-  describe Line do
-    before do
-      @fs = " "
-      @data = "a b c\n"
-      @line = Line.new(@data, @fs)
-    end
-    it "is a string" do
-      @line.is_a?(String).should be_true
-    end
-    it "chomps itself on creation" do
-      @line.should == "a b c"
-    end
-    it "finds space-delimited columns by" do
-      @line.cols.should == ["a","b","c"]
-      @line.c.should == @line.cols
-    end
-    it "calculates the number of fields" do
-      @line.nf.should == 3
-    end
-    
-    context "with a field separator ','" do
-      it "splits the line into columns using a comma" do
-        line = Line.new("a,b,c d", ",") 
-        line.c.should == ["a","b","c d"]
-      end
-    end
-    context "with a regular expression field separator" do
-      it "splits the line into columns using the regular expression" do
-        line = Line.new("a,b|c", /[,|]/)
-        line.c.should == ["a","b","c"]
-      end
-    end
-  end
-end