semantictext 0.1.0 → 0.2.0

data/CHANGELOG

@@ -0,0 +1,2 @@
+2009-12-02: renamed SemanticText::Parser to SemanticText::Document
+2009-12-02: added support for arbitrary tags

data/README.rdoc

@@ -4,12 +4,29 @@ It takes a file or sequence of lines and returns an object model of the document
 including document metadata (e.g. doc creation time and title) and a tree of 
 interconnected objects describing the document structure.
 
-== RDOC API
-The rdoc can be found at http://www.greenbarsoft.co.uk/software/semantictext/rdoc/
-
 = How to use it
-* Parse with SemanticText::Parser.parse_from(file)
-* Generate HTML with SemanticText::Parser.export_html
+* You need to have installed gemcutter.org into your gem locations.
+* Install with:
+   gem install semantictext
+* Parse like this:
+
+   require 'rubygems'
+   require 'semantictext'
+    
+   p = SemanticText::Document.new
+   p.parse 'title: my happy document'
+   p.parse ''
+   p.parse "!I'm a title"
+   p.parse "I'm a paragraph..."
+   p.parse "... yep, I'm still that paragraph."
+   p.parse ''
+   p.parse "* I'm a bullet point"
+   p.parse "* I'm another bullet point"
+    
+   puts p.export_html           # this is how to export as a HTML fragment
+    
+* *rdoc* http://www.greenbarsoft.co.uk/software/semantictext/rdoc/
+* *source* http://github.com/dafydd/semantictext 
 
 = Semantic Markup
 Semantic text supports:
@@ -24,10 +41,10 @@ We intend to support these features in future:
 * custom markup tags e.g. postal code, youtube video embed, ...
 
 
-== Compatibility
+= Compatibility
 This project is being developed on OS X. Automated testing for Linux will be included in future releases.
  
-== Licence
+= Licence
 This is open source software and comes with no warranty. See COPYING for details.
 
 http://www.greenbarsoft.co.uk

data/TODO.rdoc

@@ -1,19 +1,23 @@
 ==to do
-* support custom structure tags
-* improve testing by mocking out tag factory used in tests - consider how/whether to do this
 * support urls as a special structure tag
+* tighten up error test cases on ftptag, httptag and mailtotag
+* improve testing by mocking out tag factory used in tests - consider how/whether to do this
 * support wikinames as a special custom tag
-* publish rdoc
-* build gem
-* test gem
-* publish gem
+* fix path to testfiles so test can run on gem - tests only run on source at the moment
 
 ==maybe
+* consider including section objects that are delimited by headings and that contain all objects in a section
 * think about how to support twitter with special structure tags e.g. #keyword and @user
 * refactor parser into header parser and text parser
 * pull out parsers for different parts and use the state pattern
 
 ==done
+* support custom structure tags
+* escape HTML < > and & on headings
+* build gem from rakefile
+* publish gem
+* build gem
+* publish rdoc
 * handle proper HTML escaping
 * remove html and head elements from html output (it's here to be embedded in webpages)
 * removed surrounding square brackets from text in Tag objects

data/lib/semantictext.rb

@@ -1 +1 @@
-require 'semantictext/parser'
+require 'semantictext/document'

data/lib/semantictext/bullet.rb

@@ -1,4 +1,4 @@
-require 'semantictext/parser'
+require 'semantictext/document'
 
 module SemanticText
   

data/lib/semantictext/default_tag_factory.rb

@@ -1,9 +1,26 @@
+require 'semantictext/ftptag.rb'
+require 'semantictext/httptag.rb'
+require 'semantictext/mailtotag.rb'
+require 'semantictext/unknown_tag'
+
 module SemanticText
   # I create SemanticText::Tag objects in response to create_tag(name,value) calls 
   # from a SemanticText::Parser
   class DefaultTagFactory
+
+    def initialize()
+      @taglookup = {}
+      @taglookup['http']=HTTPTag
+      @taglookup['ftp']=FTPTag
+      @taglookup['mailto']=MailToTag
+    end
+    
     def create_tag(name, value)
-      Tag.new(name,value)
+      target = @taglookup[name]
+      if target.nil?
+        throw UnknownTag.new("#{name}:#{value}")
+      end
+      target.new(name,value)
     end
   end
 end

data/lib/semantictext/{parser.rb → document.rb}

@@ -13,8 +13,11 @@ require 'semantictext/bulleted_list_parser'
 require 'semantictext/rich_text_parser'
 
 module SemanticText
-  
-  class Parser
+ 
+  #* I am the root note of a graph of objects that form the document.
+  #* I hold metadata. 
+  #* I initiate and co-ordinate document-wide operations.
+  class Document
     # title of the document
   	attr_reader :title
   	
@@ -30,13 +33,13 @@ module SemanticText
   	# the object model of the parsed document
   	attr_reader :content
 
-  	def initialize
+  	def initialize(tag_factory=DefaultTagFactory.new)
   	  @pathname=nil
   		@headers_completed = false
   		@content = []
   		@current_paragraph = nil
       @bulleted_list_parser = nil
-      @rich_text_parser = RichTextParser.new(DefaultTagFactory.new)
+      @rich_text_parser = RichTextParser.new(tag_factory)
   	end
 
     # export as html

data/lib/semantictext/ftptag.rb

@@ -0,0 +1,21 @@
+require 'semantictext/tag'
+
+module SemanticText
+  class FTPTag < Tag
+    attr_reader :host, :path
+    
+    def initialize(key, value)
+  		@text = value
+  		@key = key
+  		@path = value
+  		parts = value.split '/',4
+      if !(parts.size==4 && parts[0]=='' && parts[1]=='')
+        raise TagParsingFailed.new("Malformed FTP tag \"#{value}\"")
+      end
+      
+  		@host = parts[2]
+  		@path = '/'+parts[3]  		
+  	end
+  	
+  end
+end

data/lib/semantictext/httptag.rb

@@ -0,0 +1,14 @@
+require 'semantictext/tag'
+
+module SemanticText
+  class HTTPTag < Tag
+    attr_reader :link
+    
+    def initialize(key, value)
+  		@text = value
+  		@key = key
+  		@link = key+':'+value
+  	end
+  	
+  end
+end

data/lib/semantictext/mailtotag.rb

@@ -0,0 +1,13 @@
+require 'semantictext/tag'
+
+module SemanticText
+  class MailToTag < Tag
+    attr_reader :address
+    
+    def initialize(key, value)
+  		@text = value
+  		@address = @value = value
+  	end
+  	
+  end
+end

data/lib/semantictext/tag.rb

@@ -1,3 +1,5 @@
+require 'semantictext/span'
+
 module SemanticText
   class Tag < Span
     attr_reader :key

data/lib/semantictext/tag_parsing_failed.rb

@@ -0,0 +1,4 @@
+module SemanticText
+  class TagParsingFailed < Exception
+  end
+end

data/lib/semantictext/unknown_tag.rb

@@ -0,0 +1,4 @@
+module SemanticText
+  class UnknownTag < Exception
+  end
+end

data/test/default_tag_factory_test.rb

@@ -0,0 +1,35 @@
+require 'test/unit'
+require 'semantictext/default_tag_factory' 
+
+class DefaultTagFactoryTest < Test::Unit::TestCase
+  
+  def setup
+    @factory = SemanticText::DefaultTagFactory.new
+  end
+  
+  def test_http
+    tag = @factory.create_tag('http','//www.example.com')
+    assert_equal(SemanticText::HTTPTag, tag.class)
+    assert_equal("http://www.example.com",tag.link)
+  end
+  
+  def test_mailto
+    tag = @factory.create_tag('mailto','somebody@example.com')
+    assert_equal(SemanticText::MailToTag, tag.class)
+    assert_equal("somebody@example.com",tag.address)
+  end
+  
+  def test_ftp
+    tag = @factory.create_tag('ftp','//ftp.funet.fi/pub/standards/RFC/rfc959.txt')
+    assert_equal(SemanticText::FTPTag, tag.class)
+    assert_equal("ftp.funet.fi",tag.host)
+    assert_equal("/pub/standards/RFC/rfc959.txt", tag.path)
+  end
+  
+  def test_unregistered_tag
+    assert_throws(:"c2:RecentChanges") {
+      @factory.create_tag('c2','RecentChanges')
+      }
+  end
+  
+end

data/test/export_test.rb

@@ -1,10 +1,10 @@
 require 'test/unit'
-require 'semantictext/parser' 
+require 'semantictext/document' 
 
 class TestExport < Test::Unit::TestCase
 
 	def test_end_to_end_loading
-		unit = SemanticText::Parser.new
+		unit = SemanticText::Document.new
 		unit.parse_from(ENV['SANDBOX']+'/semantictext/testfiles/complex.art')
     actual = unit.export_html
     actual = actual.split /\n/
@@ -16,7 +16,7 @@ class TestExport < Test::Unit::TestCase
   end
   
   def test_escaping_paragraphs
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse ''
     unit.parse 'escaping test < > &'
     
@@ -24,7 +24,7 @@ class TestExport < Test::Unit::TestCase
   end
   
   def test_escaping_headings
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse ''
     unit.parse '!heading < > &'
     
@@ -32,7 +32,7 @@ class TestExport < Test::Unit::TestCase
   end
   
   def test_escaping_bullet_points
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse ''
     unit.parse '* < > &'
 
@@ -40,7 +40,7 @@ class TestExport < Test::Unit::TestCase
   end
   
   def test_escaping_link
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse ''
     unit.parse 'http://www.example.com/app?name1=value1&name2=value2'
 

data/test/parser_test.rb

@@ -1,5 +1,5 @@
 require 'test/unit'
-require 'semantictext/parser' 
+require 'semantictext/document' 
 
 class TestParser < Test::Unit::TestCase
 
@@ -9,7 +9,7 @@ class TestParser < Test::Unit::TestCase
   end
 
 	def test_end_to_end_loading
-		unit = SemanticText::Parser.new
+		unit = SemanticText::Document.new
 		unit.parse_from(ENV['SANDBOX']+'/semantictext/testfiles/simple.art')
 		assert_equal 'test title', unit.title
 		assert_equal 5, unit.createdAt.day
@@ -37,7 +37,7 @@ class TestParser < Test::Unit::TestCase
 	
 
   	def test_headerless_document_parse
-  		unit = SemanticText::Parser.new
+  		unit = SemanticText::Document.new
   		test_lines = <<EOF
 !First Big Heading
 
@@ -54,7 +54,7 @@ EOF
   	end
 
     def test_parse_paragraph_beginning_with_url
-        		unit = SemanticText::Parser.new
+        		unit = SemanticText::Document.new
         		test_lines = <<EOF
 
 http://www.dafydd.net/foogoo?blah see? http://www.example.com
@@ -71,7 +71,7 @@ EOF
     end
     
   	def test_headerless_document_parse_with_url
-  		unit = SemanticText::Parser.new
+  		unit = SemanticText::Document.new
   		test_lines = <<EOF
 
 Embedded link http://www.dafydd.net/foogoo?blah see?
@@ -95,7 +95,7 @@ EOF
   	end
       	
 	def test_paragraph_parsing
-		unit = SemanticText::Parser.new
+		unit = SemanticText::Document.new
 		test_lines = <<EOF
 title:test title
 createdAt:5 November 2005
@@ -122,7 +122,7 @@ EOF
 	end
 
 	def test_parsing_of_parameters
-		unit = SemanticText::Parser.new
+		unit = SemanticText::Document.new
 		unit.parse('title:test title')
 		unit.parse('createdAt:5 November 2005')
 		unit.parse('keywords: buzz, fuzz, muzz')
@@ -140,44 +140,44 @@ EOF
 	end
 	
 	def test_presendence_of_url_lower_than_tag
-    		unit = SemanticText::Parser.new
+    		unit = SemanticText::Document.new
     		unit.parse('')
     		unit.parse('Embedded tag [http://www.dafydd.net/foogoo?blah name:here] see?')
 
         result = unit.content[0]
         assert_equal SemanticText::Paragraph, result.class
         assert_element SemanticText::Span, "Embedded tag ", result.content[0]
-        assert_element SemanticText::Tag, "//www.dafydd.net/foogoo?blah name:here", result.content[1]
+        assert_element SemanticText::HTTPTag, "//www.dafydd.net/foogoo?blah name:here", result.content[1]
         assert_element SemanticText::Span, " see?", result.content[2]
         assert_equal 3, result.content.size
   end
   
 	def test_headerless_document_parse_with_tags
-    		unit = SemanticText::Parser.new
+    		unit = SemanticText::Document.new
     		test_lines = <<EOF
 
-Embedded tag [rfc:822] see?
+Embedded tag [mailto:fred@example.com] see?
 I wonder if it worked!
-a [tags:red balloon] b [c2:RecentChanges] c
+a [ftp://ftp.example.com/stuff/download] b [http://www.example.com] c
 EOF
         test_lines.each {|line| unit.parse(line)}
 
         result = unit.content[0]
         assert_equal SemanticText::Paragraph, result.class
         assert_element SemanticText::Span, "Embedded tag ", result.content[0]
-        assert_element SemanticText::Tag, "822", result.content[1]
+        assert_element SemanticText::MailToTag, "fred@example.com", result.content[1]
         assert_element SemanticText::Span, " see?", result.content[2]
         assert_element SemanticText::Span, "I wonder if it worked!", result.content[3]
         assert_element SemanticText::Span, 'a ', result.content[4]
-        assert_element SemanticText::Tag, 'red balloon', result.content[5]
+        assert_element SemanticText::FTPTag, '//ftp.example.com/stuff/download', result.content[5]
         assert_element SemanticText::Span, ' b ', result.content[6]
-        assert_element SemanticText::Tag, 'RecentChanges', result.content[7]
+        assert_element SemanticText::HTTPTag, '//www.example.com', result.content[7]
         assert_element SemanticText::Span, ' c', result.content[8]
         assert_equal 9, result.content.size
     	end
 
 	def test_paragraphs_headings_and_bullet_points
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse('')
     unit.parse('!heading')
     unit.parse('This is a paragraph')
@@ -219,10 +219,10 @@ EOF
 	end
 	
 	def test_bullet_points_with_urls_and_tags
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse('')
     unit.parse('* with url http://www.example.com see?')
-    unit.parse('* with tag [c2:RecentChanges] see?')
+    unit.parse('* with tag [http://www.example.com] see?')
 
     actual_list = unit.content[0]
     first_bullet = actual_list.content[0]
@@ -235,12 +235,12 @@ EOF
     assert_element SemanticText::Span, ' see?', first_bullet.content[2]
 
     assert_element SemanticText::Span, 'with tag ', second_bullet.content[0]
-    assert_element SemanticText::Tag, 'RecentChanges', second_bullet.content[1]
+    assert_element SemanticText::HTTPTag, '//www.example.com', second_bullet.content[1]
     assert_element SemanticText::Span, ' see?', second_bullet.content[2]    
 	end
 	
 	def test_bulleted_list_nesting
-    unit = SemanticText::Parser.new
+    unit = SemanticText::Document.new
     unit.parse('')
     unit.parse('* separate list')
     unit.parse('** nested bullet point 1')
@@ -277,7 +277,7 @@ EOF
   end
   
 	def test_bulleted_lsit_parsing_into_two_separate_lists
-	  unit = SemanticText::Parser.new
+	  unit = SemanticText::Document.new
     unit.parse('')
     unit.parse('* first bullet in first list')
     unit.parse('')

data/testfiles/complex.art

@@ -23,6 +23,6 @@ http://www.example.com Cool innit?
 ** subpoint 2.1
 ** subpoint 2.2
 
-This is another paragraph. This is a [c2:RecentChanges] tag.
+This is another paragraph. This is a [http://www.example.com] tag.
 
 http://www.example.com/foo?a=b&c=d

data/testfiles/regression-exportsample.txt

@@ -11,5 +11,5 @@
 		<ul><li> subpoint 2.1</li><li> subpoint 2.2</li>
 		</ul>
 	</ul>
-<p> This is another paragraph. This is a [c2:RecentChanges]  tag.</p>
+<p> This is another paragraph. This is a [http://www.example.com]  tag.</p>
 <p><a href="http://www.example.com/foo?a=b&c=d">http://www.example.com/foo?a=b&amp;c=d</a></p>

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: semantictext
 version: !ruby/object:Gem::Version 
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors: 
 - Dafydd Rees
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-11-29 00:00:00 +00:00
+date: 2009-12-02 00:00:00 +00:00
 default_executable: 
 dependencies: []
 
@@ -30,21 +30,27 @@ files:
 - lib/semantictext/bulletedlist.rb
 - lib/semantictext/date_extractor.rb
 - lib/semantictext/default_tag_factory.rb
+- lib/semantictext/document.rb
 - lib/semantictext/extraction_failed.rb
+- lib/semantictext/ftptag.rb
 - lib/semantictext/heading.rb
+- lib/semantictext/httptag.rb
 - lib/semantictext/keyword_extractor.rb
 - lib/semantictext/link.rb
+- lib/semantictext/mailtotag.rb
 - lib/semantictext/not_header_line.rb
 - lib/semantictext/paragraph.rb
-- lib/semantictext/parser.rb
 - lib/semantictext/rich_text_parser.rb
 - lib/semantictext/span.rb
 - lib/semantictext/tag.rb
+- lib/semantictext/tag_parsing_failed.rb
+- lib/semantictext/unknown_tag.rb
 - lib/semantictext.rb
 - lib/string.rb
 - test/bullet_test.rb
 - test/bulleted_list_parser_test.rb
 - test/dateextractor_test.rb
+- test/default_tag_factory_test.rb
 - test/export_test.rb
 - test/keywordextractor_test.rb
 - test/parser_test.rb
@@ -87,6 +93,7 @@ test_files:
 - ./test/bullet_test.rb
 - ./test/bulleted_list_parser_test.rb
 - ./test/dateextractor_test.rb
+- ./test/default_tag_factory_test.rb
 - ./test/export_test.rb
 - ./test/keywordextractor_test.rb
 - ./test/parser_test.rb