bluefeather 0.20 → 0.21

data/doc/author-and-license.html

@@ -18,7 +18,7 @@
 <li><a href="http://ja.wikipedia.org/wiki/GNU_General_Public_License">Wikipedia - GNU General Public License</a></li>
 </ul>
 
-<p>BlueFeather への要望、ご意見、バグ報告などがありましたら、<a href="http://ruby.morphball.net/bluefeather/">BlueFeather 配布サイト</a>よりメッセージを送信していただくか、もしくは Dice（<a href="&#x6D;&#097;&#x69;&#x6C;&#116;o&#058;&#116;&#101;&#116;&#114;&#097;&#100;&#105;&#099;&#x65;&#064;&#103;m&#x61;il&#x2E;&#x63;&#111;&#109;">&#x6D;&#097;&#x69;&#x6C;&#116;o&#058;&#116;&#101;&#116;&#114;&#097;&#100;&#105;&#099;&#x65;&#064;&#103;m&#x61;il&#x2E;&#x63;&#111;&#109;</a>）までメールでご連絡していただけるよう、よろしくお願いします。</p>
+<p>BlueFeather への要望、ご意見、バグ報告などがありましたら、<a href="http://ruby.morphball.net/bluefeather/">BlueFeather 配布サイト</a>よりメッセージを送信していただくか、もしくは Dice（<a href="&#109;&#x61;&#105;&#108;&#116;&#x6F;:&#x74;&#x65;&#116;&#x72;&#097;&#x64;i&#x63;&#101;&#064;&#103;&#x6D;&#097;&#105;&#x6C;&#x2E;&#099;&#x6F;&#109;">&#x74;&#x65;&#116;&#x72;&#097;&#x64;i&#x63;&#101;&#064;&#103;&#x6D;&#097;&#105;&#x6C;&#x2E;&#099;&#x6F;&#109;</a>）までメールでご連絡していただけるよう、よろしくお願いします。</p>
 
 </body>
 </html>

data/doc/en/author-and-license.html

@@ -24,7 +24,7 @@
 <p>If you want to send requests, comments, or bug reports to BlueFeather, please
 feel free to post messages from
 <a href="http://ruby.morphball.net/bluefeather/index_en.html">http://ruby.morphball.net/bluefeather/index_en.html</a> or to send mail to Dice.
-(<a href="&#109;&#097;&#x69;&#108;&#116;&#x6F;&#x3A;&#x74;&#x65;&#x74;&#114;&#097;&#x64;&#x69;&#x63;&#x65;&#x40;&#x67;&#109;&#x61;i&#x6C;&#x2E;&#099;&#x6F;&#x6D;">&#109;&#097;&#x69;&#108;&#116;&#x6F;&#x3A;&#x74;&#x65;&#x74;&#114;&#097;&#x64;&#x69;&#x63;&#x65;&#x40;&#x67;&#109;&#x61;i&#x6C;&#x2E;&#099;&#x6F;&#x6D;</a>)</p>
+(<a href="&#109;&#097;&#105;&#108;&#x74;&#x6F;:&#x74;e&#116;&#x72;&#097;&#x64;&#x69;&#x63;e&#064;&#103;&#x6D;&#097;&#105;&#108;&#046;&#x63;&#x6F;&#109;">&#x74;e&#116;&#x72;&#097;&#x64;&#x69;&#x63;e&#064;&#103;&#x6D;&#097;&#105;&#108;&#046;&#x63;&#x6F;&#109;</a>)</p>
 
 </body>
 </html>

data/doc/en/basic-usage.html

@@ -185,6 +185,8 @@ Test paragraph.
 
 <h3 id="commandline-options">Command-line Options</h3>
 
+
+
 <pre><code>% bluefeather --help
 bluefeather - Extended Markdown Converter
 
@@ -220,6 +222,8 @@ More info:
 %
 </code></pre>
 
+
+
 <h2 id="bfheader-ea4b8233c187e9e87b21ba52d38720e8">Use in Ruby Script</h2>
 
 <h3 id="bfheader-972e73b7a882d0802a4e3a16946a2f94">Basic</h3>

data/doc/en/format-extension.html

@@ -166,9 +166,13 @@ line same as it is parsed as code-block.
 </code></pre>
 
 <blockquote>
+
+
 <pre><code>A range from a line which starts with three or more tildes (~) to another
 line same as it is parsed as code-block.
 </code></pre>
+    
+
 </blockquote>
 
 <h3 id="bfheader-51c45b795d5d18a3e4e0c37e8b20a141">Table</h3>

data/doc/en/index.bfdoc

@@ -10,7 +10,7 @@ BlueFeather Manual
 
 -> [Japanese version (original)](../index.html)
 
-(2009-05-04: this document based on version 0.20)
+(2009-05-15: this document based on version 0.21)
 
  BlueFeather is software for converting text written by extended Markdown like 
 [PHP Markdown Extra][] to html. It is pair of command-line tool and pure Ruby

data/doc/en/index.html

@@ -12,7 +12,7 @@
 
 <p>-> <a href="../index.html">Japanese version (original)</a></p>
 
-<p>(2009-05-04: this document based on version 0.20)</p>
+<p>(2009-05-15: this document based on version 0.21)</p>
 
 <p>BlueFeather is software for converting text written by extended Markdown like 
 <a href="http://michelf.com/projects/php-markdown/extra/">PHP Markdown Extra</a> to html. It is pair of command-line tool and pure Ruby

data/doc/format-extension.html

@@ -170,9 +170,13 @@
 </code></pre>
 
 <blockquote>
+
+
 <pre><code>3つ以上のチルダ記号（~）がある行から、同じ数のチルダ記号がある行までは
 丸ごとコードブロックとして扱われます。
 </code></pre>
+    
+
 </blockquote>
 
 <h3 id="bfheader-3883c72da6a61cb25dc54394bef5dadd">表組み</h3>

data/doc/index.bfdoc

@@ -9,7 +9,7 @@ BlueFeather マニュアル
 → [English version](en/index.html)
 
 
-（2009-05-04 バージョン 0.20 準拠）
+（2009-05-15 バージョン 0.21 準拠）
 
 BlueFeather は、拡張 Markdown 記法で書かれたテキストを html に変換するソフトウェアです。
 コマンドラインツールと、Ruby スクリプト内で変換を行うためのライブラリがセットになっています。

data/doc/index.html

@@ -13,7 +13,7 @@
 
 <p>→ <a href="en/index.html">English version</a></p>
 
-<p>（2009-05-04 バージョン 0.20 準拠）</p>
+<p>（2009-05-15 バージョン 0.21 準拠）</p>
 
 <p>BlueFeather は、拡張 Markdown 記法で書かれたテキストを html に変換するソフトウェアです。
 コマンドラインツールと、Ruby スクリプト内で変換を行うためのライブラリがセットになっています。</p>

data/lib/bluefeather.rb

@@ -42,8 +42,13 @@ require 'uri'
 
 
 module BlueFeather
-	VERSION = '0.20'
-	VERSION_NUMBER = 0.20
+	VERSION = '0.21'
+	VERSION_NUMBER = 0.21
+	RELEASE_DATE = '2009-05-16'
+	VERSION_LABEL = "#{VERSION} (#{RELEASE_DATE})"
+	
+	UTF8_BOM = "\xef\xbb\xbf"
+	UTF8_BOM_PATTERN = /^#{UTF8_BOM}/
 
 
 	# Fancy methods
@@ -207,11 +212,18 @@ module BlueFeather
 					
 					# get headers
 					pos_before_gets = nil
+					first_line = true
 
 					loop do
 						pos_before_gets = input.pos
-						
 						line = input.gets
+						
+						# cut UTF-8 BOM
+						if first_line and line =~ UTF8_BOM_PATTERN then
+							line.slice!(UTF8_BOM_PATTERN)
+						end	
+						first_line = false
+						
 						if line and line.chomp =~ HEADER_PATTERN then
 							key = $1.downcase; value = $2
 							
@@ -222,6 +234,7 @@ module BlueFeather
 									
 									# rewind (reason => [ruby-list:45988])
 									input.pos = first_pos
+									first_line = true
 								else
 									$KCODE = kc
 								end
@@ -420,12 +433,14 @@ module BlueFeather
 			# Simplify blank lines
 			text.gsub!( /^ +$/, '' )
 			@log.debug "Tabs -> spaces/blank lines stripped: %p" % text
+			
 	
 			# Replace HTML blocks with placeholders
 			text = hide_html_blocks( text, rs )
 			@log.debug "Hid HTML blocks: %p" % text
 			@log.debug "Render state: %p" % rs
 			
+			
 			# Strip footnote definitions, store in render state
 			text = strip_footnote_definitions( text, rs )
 			@log.debug "Stripped footnote definitions: %p" % text
@@ -437,8 +452,6 @@ module BlueFeather
 			@log.debug "Stripped link definitions: %p" % text
 			@log.debug "Render state: %p" % rs
 			
-
-	
 			# Escape meta-characters
 			text = escape_special_chars( text )
 			@log.debug "Escaped special characters: %p" % text
@@ -622,8 +635,8 @@ module BlueFeather
 	
 			@log.debug "Applying block transforms to:\n  %p" % str
 			text = str
-			text = pretransform_block_separators(text, rs)
 			text = pretransform_fenced_code_blocks( text, rs )
+			text = pretransform_block_separators(text, rs)
 			
 			text = transform_headers( text, rs )
 			text = transform_toc(text, rs)
@@ -1270,7 +1283,7 @@ module BlueFeather
 	
 		# Pattern for matching codeblocks
 		CodeBlockRegexp = %r{
-			(?:\n\n|\A)
+			(?:\n\n|\A|\A\n)
 			(									# $1 = the code block
 			  (?:
 				(?:[ ]{#{TabWidth}} | \t)		# a tab or tab-width of spaces
@@ -1311,7 +1324,7 @@ module BlueFeather
 			@log.debug " Transforming fenced code blocks => standard code blocks"
 	
 			str.gsub( FencedCodeBlockRegexp ) {|block|
-				"\n" + transform_code_blocks(indent($2), rs) + "\n"
+				"\n~\n\n" + indent($2) + "\n~\n\n"
 			}
 		end
 

data/lib/bluefeather/cui.rb

@@ -95,7 +95,7 @@ More info:
 			op.on('-q', '--quiet'){ message_out = StringIO.new }
 			op.on('--suffix .SUF', String){|x| suffix = x}
 			op.on('--version'){
-				@stdout.puts "bluefeather #{VERSION}"
+				@stdout.puts "bluefeather #{VERSION_LABEL}"
 				return false
 			}
 			op.on('-h', '--help'){
@@ -110,7 +110,7 @@ More info:
 				return false
 			end
 			
-			message_out.puts "default encoding: #{encoding}"
+			message_out.puts "default encoding: #{encoding}" if verbose
 			
 			unless defined?(Encoding) then
 				# ruby 1.8 or earlier
@@ -177,16 +177,22 @@ More info:
 							end
 							
 							# parse
+							parser = BlueFeather::Parser.new
+							this_encoding = nil
 							parsing_sec = Benchmark.realtime{
 								case current_format
 								when DOCUMENT
-									message_out.puts "[document] #{src}" if verbose
-									html = BlueFeather.parse_document_file(src, encoding)
+									doc = nil
+									open(src, 'r'){|f|
+										doc = BlueFeather::Document.parse_io(f, encoding)
+									}
+									html = parser.document_to_html(doc)
+									this_encoding = doc.encoding_type
 								when TEXT
-									message_out.puts "[text] #{src}" if verbose
 									open_mode = (defined?(Encoding) ? "r:#{encoding}" : 'r')
 									text = src.open(open_mode){|x| x.read}
-									html = BlueFeather.parse_text(text)
+									html = parser.parse_text(text)
+									this_encoding = encoding
 								end
 							}
 						
@@ -197,7 +203,10 @@ More info:
 								message_out.puts "#{src} => #{dest} (#{File.size(dest)} byte)"
 						
 								if verbose then
-									message_out.puts sprintf('  (parsing time: %g sec)', parsing_sec)
+									message_out.puts "    Format: #{current_format}"
+									message_out.puts "    Encoding: #{this_encoding}"
+									message_out.puts sprintf('    Parsing Time: %g sec', parsing_sec)
+									message_out.puts
 								end
 							end # if html
 						end # if ext == suffix

data/spec/code-block.rb

@@ -1,7 +1,6 @@
 require 'pathname'
 require(Pathname.new(__FILE__).parent + 'lib/common.rb')
 
-
 describe 'Normal Code Block:' do
 	before(:each) do
 		@bf = BlueFeather::Parser.new
@@ -57,6 +56,8 @@ describe 'Fenced Code Block:' do
 			end
 		end
 		
+
+		
 		describe 'Many Symbols:' do
 			include Examples
 			before(:all) do
@@ -85,7 +86,60 @@ describe 'Fenced Code Block:' do
 			end
 		end
 
+	end
+
+
+	describe '0.20 Bug:' do
+		describe 'Last-line Header:' do
+			before(:all) do
+				@src = "~~~\nFenced Code\nBlock\n# Last-line header\n~~~"
+			end
+			
+			specify 'overview' do
+				@doc.should have_elements(1, 'pre code')
+				@doc.should_not have_element('p')
+				@doc.should_not have_element('h1')
+			end
+				
+			specify 'body' do
+				@doc.at('pre code').inner_text.should == "Fenced Code\nBlock\n# Last-line header\n"
+			end
+		end
 		
+		describe 'Definition List:' do
+			before(:all) do
+				@src = "~~~\nFenced Code\nBlock\n: dd\n~~~"
+			end
+			
+			specify 'overview' do
+				@doc.should have_elements(1, 'pre code')
+				@doc.should_not have_element('p')
+				@doc.should_not have_element('dl')
+			end
+				
+			specify 'body' do
+				@doc.at('pre code').inner_text.should == "Fenced Code\nBlock\n: dd\n"
+			end
+		end
+
+		describe 'Code block from second line:' do
+			before(:all) do
+				@src = "\n    a\n    b"
+			end
+			
+			specify 'overview' do
+				@doc.should have_elements(1, 'pre code')
+				@doc.should_not have_element('p')
+			end
+				
+			specify 'body' do
+				@doc.at('pre code').inner_text.should == "a\nb\n"
+			end
+		end
+
+
+
 	end
+	
 
 end

data/spec/cui.rb

@@ -40,7 +40,7 @@ describe 'CUI:' do
 	
 	specify '--version' do
 		@cui.run(%w(--version unknown-file))
-		@cui.should wrote_to_stdout("bluefeather #{BlueFeather::VERSION}\n")
+		@cui.should wrote_to_stdout("bluefeather #{BlueFeather::VERSION_LABEL}\n")
 		@cui.should_not wrote_to_stderr
 	end
 	

data/spec/encoding.rb

@@ -0,0 +1,98 @@
+# encoding: utf-8
+
+require 'pathname'
+require 'kconv'
+require(Pathname.new(__FILE__).parent + 'lib/common.rb')
+
+
+describe 'Encoding:' do
+	before(:each) do
+		@html = BlueFeather.parse_document_file(Pathname.new(__FILE__).parent + 'text/' + @src_name)
+		@doc = Hpricot(@html)
+	end
+	
+	share_as :Base do
+		specify 'header parse - css' do
+			@doc.should have_element('link[@rel="stylesheet"][@href="style.css"]')
+		end
+
+		specify 'header parse - encoding' do
+			@doc.should have_element(%Q|meta[@http-equiv="Content-Type"][@content="text/html; charset=#{@charset}"]|)
+		end
+		
+		specify 'header parse - title' do
+			base = '日本語の題名'.kconv(@kconv_encoding, Kconv::UTF8)
+			@doc.at('title').inner_html.should == base
+		end
+		
+		specify 'content parse - p' do
+			base = 'このテキストファイルは、エンコーディングテスト用のテキストファイルです。'.kconv(@kconv_encoding, Kconv::UTF8)
+			@doc.at('p').inner_html.should == base
+		end
+		
+		specify 'content parse - ul' do
+			@doc.should have_elements(1, 'ul')
+			
+			items = @doc.search('ul li')
+			%w(項目1 項目2 項目3).each_with_index do |str, i|
+				items[i].inner_html.should == str.kconv(@kconv_encoding, Kconv::UTF8)
+			end
+
+		end
+
+	end
+	
+	
+	describe 'Default:' do
+		include Base
+		
+		before(:all) do
+			@src_name = 'encoding_sample_default.bfdoc'
+			@charset = 'utf-8'
+			@kconv_encoding = Kconv::UTF8
+		end
+	end
+
+
+	describe 'UTF-8 (BOM):' do
+		include Base
+		
+		before(:all) do
+			@src_name = 'encoding_sample_utf-8.bfdoc'
+			@charset = 'utf-8'
+			@kconv_encoding = Kconv::UTF8
+		end
+	end
+	
+	
+	describe 'UTF-8N:' do
+		include Base
+		
+		before(:all) do
+			@src_name = 'encoding_sample_utf-8n.bfdoc'
+			@charset = 'utf-8'
+			@kconv_encoding = Kconv::UTF8
+		end
+	end
+
+	
+	describe 'Shift-JIS:' do
+		include Base
+		
+		before(:all) do
+			@src_name = 'encoding_sample_shift-jis.bfdoc'
+			@charset = 'shift_jis'
+			@kconv_encoding = Kconv::SJIS
+		end
+	end
+	
+	describe 'EUC-JP:' do
+		include Base
+		
+		before(:all) do
+			@src_name = 'encoding_sample_euc-jp.bfdoc'
+			@charset = 'euc-jp'
+			@kconv_encoding = Kconv::EUC
+		end
+	end
+end

data/spec/text/encoding_sample_default.bfdoc

@@ -0,0 +1,8 @@
+CSS: style.css
+Title: 日本語の題名
+
+このテキストファイルは、エンコーディングテスト用のテキストファイルです。
+
+* 項目1
+* 項目2
+* 項目3

data/spec/text/encoding_sample_euc-jp.bfdoc

@@ -0,0 +1,9 @@
+Encoding: EUC-JP
+CSS: style.css
+Title: ���ܸ����̾
+
+���Υƥ����ȥե�����ϡ����󥳡��ǥ��󥰥ƥ����ѤΥƥ����ȥե�����Ǥ���
+
+* ����1
+* ����2
+* ����3

data/spec/text/encoding_sample_shift-jis.bfdoc

@@ -0,0 +1,9 @@
+Encoding: Shift-JIS
+CSS: style.css
+Title: ���{��̑薼
+
+���̃e�L�X�g�t�@�C���́A�G���R�[�f�B���O�e�X�g�p�̃e�L�X�g�t�@�C���ł��B
+
+* ����1
+* ����2
+* ����3

data/spec/text/encoding_sample_utf-8.bfdoc

@@ -0,0 +1,9 @@
+CSS: style.css
+Encoding: UTF-8
+Title: 日本語の題名
+
+このテキストファイルは、エンコーディングテスト用のテキストファイルです。
+
+* 項目1
+* 項目2
+* 項目3

data/spec/text/encoding_sample_utf-8n.bfdoc

@@ -0,0 +1,9 @@
+CSS: style.css
+Encoding: UTF-8
+Title: 日本語の題名
+
+このテキストファイルは、エンコーディングテスト用のテキストファイルです。
+
+* 項目1
+* 項目2
+* 項目3

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: bluefeather
 version: !ruby/object:Gem::Version 
-  version: "0.20"
+  version: "0.21"
 platform: ruby
 authors: 
 - Dice
@@ -9,11 +9,12 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-05-04 00:00:00 +09:00
+date: 2009-05-16 00:00:00 +09:00
 default_executable: 
 dependencies: []
 
-description: BlueFeather is software for converting text written by extended Markdown to html. It is pair of command-line tool and pure Ruby library.
+description: " BlueFeather is software for converting text written by extended Markdown to\n\
+  html. It is pair of command-line tool and pure Ruby library.\n"
 email: tetradice@gmail.com
 executables: 
 - bluefeather
@@ -34,15 +35,17 @@ files:
 - spec/code-block.rb
 - spec/cui.rb
 - spec/dl.rb
-- spec/encoding_sample_eucjp.bfdoc
-- spec/encoding_sample_sjis.bfdoc
-- spec/encoding_sample_utf8.bfdoc
+- spec/encoding.rb
 - spec/escape-char.rb
 - spec/footnote.rb
 - spec/header-id.rb
-- spec/lib
 - spec/lib/common.rb
 - spec/table.rb
+- spec/text/encoding_sample_default.bfdoc
+- spec/text/encoding_sample_euc-jp.bfdoc
+- spec/text/encoding_sample_shift-jis.bfdoc
+- spec/text/encoding_sample_utf-8.bfdoc
+- spec/text/encoding_sample_utf-8n.bfdoc
 - spec/toc.rb
 - spec/warning.rb
 - original-tests/00_Class.tests.rb
@@ -50,7 +53,6 @@ files:
 - original-tests/10_Bug.tests.rb
 - original-tests/15_Contrib.tests.rb
 - original-tests/bftestcase.rb
-- original-tests/data
 - original-tests/data/antsugar.txt
 - original-tests/data/ml-announce.txt
 - original-tests/data/re-overflow.txt
@@ -91,8 +93,10 @@ files:
 - doc/index.html
 - doc/metadata-reference.bfdoc
 - doc/metadata-reference.html
-has_rdoc: false
+has_rdoc: true
 homepage: http://ruby.morphball.net/bluefeather/
+licenses: []
+
 post_install_message: 
 rdoc_options: []
 
@@ -113,9 +117,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 
 rubyforge_project: bluefeather
-rubygems_version: 1.3.1
+rubygems_version: 1.3.3
 signing_key: 
-specification_version: 2
+specification_version: 3
 summary: Extend Markdown Converter
 test_files: []
 

data/spec/encoding_sample_eucjp.bfdoc

@@ -1,3 +0,0 @@
-Encoding: EUC-JP
-
-���Υƥ����ȥե�����ϡ����󥳡��ǥ��󥰥ƥ����ѤΥƥ����ȤǤ���

data/spec/encoding_sample_sjis.bfdoc

@@ -1,3 +0,0 @@
-Encoding: Shift-JIS
-
-���̃e�L�X�g�t�@�C���́A�G���R�[�f�B���O�e�X�g�p�̃e�L�X�g�ł��B

data/spec/encoding_sample_utf8.bfdoc

@@ -1,3 +0,0 @@
-Encoding: UTF-8
-
-このテキストファイルは、エンコーディングテスト用のテキストです。