pseudohikiparser 0.0.4 → 0.0.5.develop

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 38493340a4c48448be37a5e6d800bf6ad0fa8f8f
-  data.tar.gz: 43c9ef5c8dadb24c502d0bdb8f3dd47a511c3157
+  metadata.gz: 5be7e5cb75ef3ae2d601b8dbf03aa768c5d0e1c1
+  data.tar.gz: 6d9de4c42fa1339e0922903764c07cd8edc13c5e
 SHA512:
-  metadata.gz: 381ec4accb316ca35c4c82d49e9189657a330cfb318e12c0783f528b1c5b41a156352538c0297e232358cb43058c4a3ca90a6fefaf11bea378244a2b3da0c497
-  data.tar.gz: 552e791bf5fc72f022d499a069911b2705d121304c2a23381b3e6e7039287d6019a9dfea005f769c2fe05ea63ecfc99411ccf9bf1fdd8781a19645a23e761bc6
+  metadata.gz: 2d5e5d27c733773551dc372df645933aa14d3db2c5c212d73646010db2fc59c6756ba13c37c662b9f5e56f48ebba4e98f014c853d8ab9c7e9cb53c628f645dbc
+  data.tar.gz: 9bb79366e51c3ca3f0c6d51abdcdd7f399d905130113ae86d0311f16734258da159f3824727c436ca6551f3a233fb2cf50bd861f65233ec76d617e4e426a6445

data/README.ja.md

@@ -1,7 +1,7 @@
 PseudoHikiParser
 ================
 
-PseudoHikiParserは[Hiki](https://github.com/hiki/hikidoc)に似た記法で書かれたテキストをHTML・Markdownその他のファイル形式に変換するコンバータです。
+PseudoHikiParserは[Hiki](https://github.com/hiki/hikidoc)に似た記法で書かれたテキストをパースし、HTML・Markdownその他のファイル形式に変換します。
 
 このツールは次の目的を念頭において作成中です。
 
@@ -26,7 +26,7 @@ gem install pseudohikiparser
 もし実験的な機能も試してみたければ、
 
 ```
-gem install pseudohikiparser --pre
+gem install pseudohikiparser --version 0.0.5.develop
 ```
 
 ## 使い方
@@ -44,16 +44,16 @@ gem install pseudohikiparser --pre
 
 このサンプルは[develop branch](https://github.com/nico-hn/PseudoHikiParser/tree/develop/samples)に置いてあります。
 
-### pseudohiki2html.rb
+### pseudohiki2html
 
-_(pseudohiki2html.rbは現時点ではPseudoHikiParserの利用例として提供されており、現段階ではオプションも継続的に変更されることにご注意ください。)_
+_(pseudohiki2htmlは現時点ではPseudoHikiParserの利用例として提供されており、現段階ではオプションも継続的に変更されることにご注意ください。)_
 
-インストールが終わると**pseudohiki2html.rb**というコマンドが使えるようになります。
+インストールが終わると**pseudohiki2html**というコマンドが使えるようになります。
 
 コマンドラインで次のように入力すると
 
 ```
-pseudohiki2html.rb <<TEXT
+pseudohiki2html <<TEXT
 !! The first heading
 The first paragraph
 TEXT
@@ -88,7 +88,7 @@ The first paragraph
 `--output`オプションを使ってファイル名を指定した場合は
 
 ```
-pseudohiki2html.rb --output first_example.html <<TEXT
+pseudohiki2html --output first_example.html <<TEXT
 !! The first heading
 The first paragraph
 TEXT
@@ -96,10 +96,12 @@ TEXT
 
 で、結果がfirst\_example.htmlに保存されます。
 
-その他のオプションについては`pseudohiki2html.rb --help`でご参照ください。
+その他のオプションについては`pseudohiki2html --help`でご参照ください。
 
 #### 非互換な変更
 
+version 0.0.4まで、コマンド名は`pseudohiki2html.rb`でした。
+
 version 0.0.0.9.developから、コマンドラインのオプション名を以下の通り変更しています。
 
 |元の名前           |新しい名前            |注記                                                         |
@@ -406,7 +408,7 @@ paragraph 3
 
 以下のクラスのうちの一部は、部分的にしか実装されていないかテストが十分ではないことにご注意ください。
 
-### [HtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L8), [XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L312)
+### [HtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L8), [XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L320)
 
 これらのクラスのクラスメソッド(HtmlFormat|XhtmlFormat).formatは[HtmlElement](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/htmlelement.rb)オブジェクトを木にしたものを返すため、以下の例のように後から手を加えることができます。
 
@@ -450,11 +452,11 @@ paragraph 2 that contains <a class="pdf" href="http://www.example.org/example.pd
 </div>
 ```
 
-### [Xhtml5Format](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L318)
+### [Xhtml5Format](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L326)
 
 HTML5への変換用のVisitorクラスです。
 
-現時点では\<section\>要素の扱い以外に[XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L312)との違いは余りありません。
+現時点では\<section\>要素の扱い以外に[XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L320)との違いは余りありません。
 
 ### [PlainTextFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/plaintextformat.rb)  
 

data/README.md

@@ -1,7 +1,7 @@
 PseudoHikiParser
 ================
 
-PseudoHikiParser is a converter of texts written in a [Hiki](https://github.com/hiki/hikidoc) like notation, into HTML, Markdown or other formats.
+PseudoHikiParser parses texts written in a [Hiki](https://github.com/hiki/hikidoc) like notation, and converts them into HTML, Markdown or other formats.
 
 I am writing this tool with following objectives in mind,
 
@@ -26,7 +26,7 @@ gem install pseudohikiparser
 or if you also want to try out experimental features,
 
 ```
-gem install pseudohikiparser --pre
+gem install pseudohikiparser --version 0.0.5.develop
 ```
 
 ## Usage
@@ -44,16 +44,16 @@ And results of conversion
 
 You will find these samples in [develop branch](https://github.com/nico-hn/PseudoHikiParser/tree/develop/samples).
 
-### pseudohiki2html.rb
+### pseudohiki2html
 
-_(Please note that pseudohiki2html.rb is currently provided as a showcase of PseudoHikiParser, and the options will be continuously changed at this stage of development.)_
+_(Please note that pseudohiki2html is currently provided as a showcase of PseudoHikiParser, and the options will be continuously changed at this stage of development.)_
 
-After the installation of PseudoHikiParser, you can use a command: **pseudohiki2html.rb**.
+After the installation of PseudoHikiParser, you can use a command: **pseudohiki2html**.
 
 Type the following lines at the command prompt:
 
 ```
-pseudohiki2html.rb <<TEXT
+pseudohiki2html <<TEXT
 !! The first heading
 The first paragraph
 TEXT
@@ -88,7 +88,7 @@ The first paragraph
 And if you specify a file name with `--output` option:
 
 ```
-pseudohiki2html.rb --output first_example.html <<TEXT
+pseudohiki2html --output first_example.html <<TEXT
 !! The first heading
 The first paragraph
 TEXT
@@ -96,10 +96,12 @@ TEXT
 
 the result will be saved in first\_example.html.
 
-For more options, please try `pseudohiki2html.rb --help`
+For more options, please try `pseudohiki2html --help`
 
 #### Incompatible changes
 
+Until version 0.0.4, the name of the command was `pseudohiki2html.rb`.
+
 From version 0.0.0.9.develop, command line options are renamed as follows:
 
 |old name       |new name         |note                                                       |
@@ -406,7 +408,7 @@ paragraph 3
 
 Please note that some of the following classes are implemented partly or not tested well.
 
-### [HtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L8), [XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L312)
+### [HtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L8), [XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L320)
 
 Their class method (HtmlFormat|XhtmlFormat).format returns a tree of [HtmlElement](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/htmlelement.rb) objects, and you can traverse the tree as in the following example.
 
@@ -450,11 +452,11 @@ paragraph 2 that contains <a class="pdf" href="http://www.example.org/example.pd
 </div>
 ```
 
-### [Xhtml5Format](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L318)
+### [Xhtml5Format](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L326)
 
 This visitor is for HTML5.
 
-Currently there aren't many differences with [XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L312) except for the handling of \<section\> elements.
+Currently there aren't many differences with [XhtmlFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/htmlformat.rb#L320) except for the handling of \<section\> elements.
 
 ### [PlainTextFormat](https://github.com/nico-hn/PseudoHikiParser/blob/develop/lib/pseudohiki/plaintextformat.rb)  
 
@@ -548,7 +550,7 @@ The first paragraph
 
 #### Limitations
 
-You can not convert malformed lists with this visitor class. That means list items must be nested hierarchically and if you skip a level in the sequence of items, the result of coversions will be corrupted.
+You cannot convert malformed lists with this visitor class. That means list items must be nested hierarchically and if you skip a level in the sequence of items, the result of coversions will be corrupted.
 
 The following is an example of malformed list in which the first level is skipped:
 

data/bin/{pseudohiki2html.rb → pseudohiki2html}

@@ -3,7 +3,7 @@
 require 'pseudohiki/converter'
 
 options = PseudoHiki::OptionManager.new
-options.set_options_from_command_line
+options.parse_command_line_options
 
 if $KCODE
   PseudoHiki::OptionManager::ENCODING_REGEXP.each do |pat, encoding|

data/lib/htmlelement/utils.rb

@@ -0,0 +1,132 @@
+#!/usr/bin/env ruby
+
+require 'htmlelement'
+require 'uri'
+
+class HtmlElement
+  module Utils
+    def self.collect_elements(tree)
+      [].tap do |elms|
+        tree.traverse do |elm|
+          matched = yield elm
+          elms.push elm if matched
+        end
+      end
+    end
+
+    def self.collect_elements_by_name(tree, name)
+      collect_elements(tree) do |elm|
+        elm.kind_of? HtmlElement and elm.tagname == name
+      end
+    end
+
+    class LinkManager
+      SEP = "/".freeze
+      SCHEME_RE = /^(https?|ftp):\/\//
+      DEFAULT_SCHEME = 'http://'
+
+      def initialize(domain_name, from_host_names=[], scheme=DEFAULT_SCHEME)
+        domain_name += SEP unless domain_name.end_with?(SEP)
+        domain_name = scheme + domain_name unless SCHEME_RE =~ domain_name
+        @domain_name = URI.parse(domain_name)
+        @domain_name_re = Regexp.compile(Regexp.escape(domain_name))
+        unless from_host_names.empty?
+          @from_host_names_re = compile_from_names_re(from_host_names)
+        end
+      end
+
+      def unify_host_names(url)
+        return url unless @from_host_names_re
+        url.sub(@from_host_names_re, @domain_name.host)
+      end
+
+      def convert_to_relative_path(url)
+        return url unless SCHEME_RE =~ url
+        return "./".freeze if default_domain?(url)
+        (URI.parse(url) - @domain_name).to_s
+      end
+
+      def use_relative_path_for_in_domain_links(html)
+        links = Utils.collect_elements_by_name(html, "a".freeze)
+        links.each do |a|
+          href = a["href"]
+          href = unify_host_names(href)
+          href = convert_to_relative_path(href)
+          a["href"] = href
+        end
+        html
+      end
+
+      def external_link?(url)
+        if SCHEME_RE.match(url)
+          URI.parse(url).host != @domain_name.host
+        end
+      end
+
+      private
+
+      def compile_from_names_re(from_host_names)
+        escaped_names = from_host_names.map {|name| Regexp.escape(name) }
+        Regexp.compile(escaped_names.join("|"))
+      end
+
+      def default_domain?(url)
+        url += SEP unless url.end_with?(SEP)
+        (URI.parse(url) - @domain_name).to_s.empty?
+      end
+    end
+
+    class TableManager
+      TH, TD, CAPTION = %w(th td caption)
+      ROWSPAN, COLSPAN = %w(rowspan colspan)
+      SCOPE, COL, ROW = %w(scope col row)
+
+      def self.assign_scope(table)
+        @manager.assign_scope(table)
+      end
+
+      def guess_header_scope(table)
+        col_scope = COL
+        row_scope = ROW
+
+        cell_with_index(table) do |cell, i, j|
+          return if span_set?(cell, ROWSPAN) or span_set?(cell, COLSPAN)
+          col_scope = nil unless (i == 0) == (cell.tagname == TH)
+          row_scope = nil unless (j == 0) == (cell.tagname == TH)
+        end
+
+        col_scope or row_scope
+      end
+
+      def assign_scope(table)
+        scope = guess_header_scope(table)
+        return table unless scope
+        Utils.collect_elements_by_name(table, TH).each do |th|
+          th[SCOPE] = scope
+        end
+        table
+      end
+
+      private
+
+      def cell_with_index(table)
+        children_except_caption(table).each_with_index do |tr, i|
+          tr.children.each_with_index do |cell, j|
+            yield cell, i, j
+          end
+        end
+      end
+
+      def children_except_caption(table)
+        children = table.children
+        children[0].tagname == CAPTION ? children[1..-1] : children
+      end
+
+      def span_set?(cell, span)
+        cell[span] && cell[span] > 1
+      end
+
+      @manager = new
+    end
+  end
+end

data/lib/htmlelement.rb

@@ -41,7 +41,7 @@ class HtmlElement
 
   ELEMENT_TYPES = {
     :BLOCK => %w(html body div table colgroup thead tbody ul ol dl head p pre blockquote style),
-    :HEADING_TYPE_BLOCK => %w(dt dd tr title h1 h2 h3 h4 h5 h6),
+    :HEADING_TYPE_BLOCK => %w(dt dd tr title caption h1 h2 h3 h4 h5 h6),
     :LIST_ITEM_TYPE_BLOCK => %w(li col),
     :EMPTY_BLOCK => %w(img meta link base input hr)
   }

data/lib/pseudohiki/blockparser.rb

@@ -38,14 +38,21 @@ module PseudoHiki
     end
 
     class BlockStack < TreeStack
-      def pop
-        current_node.parse_leafs
-        super
+      attr_reader :stack
+
+      def pop_with_breaker(breaker=nil)
+        current_node.parse_leafs(breaker)
+        pop
+      end
+
+      def current_heading_level
+        i = @stack.rindex {|node| node.kind_of? BlockElement::HeadingNode }
+        @stack[i].level || 0
       end
     end
 
     class BlockLeaf < BlockStack::Leaf
-      attr_accessor :level, :node_id
+      attr_accessor :level, :node_id, :decorator
 
       class << self
         attr_accessor :head_re
@@ -81,7 +88,7 @@ module PseudoHiki
         super(stack)
       end
 
-      def parse_leafs
+      def parse_leafs(breaker)
         parsed = InlineParser.parse(join)
         clear
         concat(parsed)
@@ -131,6 +138,10 @@ module PseudoHiki
         first.level if first # @cached_level ||= (first.level if first)
       end
 
+      def decorator
+        first.decorator if first
+      end
+
       def push_self(stack)
         @stack = stack
         super(stack)
@@ -140,11 +151,11 @@ module PseudoHiki
         not (kind_of? breaker.block and level == breaker.level)
       end
 
-      def parse_leafs; end
+      def parse_leafs(breaker); end
 
       def add_leaf(line, blockparser)
         leaf = create_leaf(line, blockparser)
-        blockparser.stack.pop while blockparser.breakable?(leaf)
+        blockparser.stack.pop_with_breaker(leaf) while blockparser.breakable?(leaf)
         blockparser.stack.push leaf
       end
 
@@ -156,8 +167,8 @@ module PseudoHiki
     end
 
     class NonNestedBlockNode < BlockNode
-      def parse_leafs
-        each {|leaf| leaf.parse_leafs }
+      def parse_leafs(breaker)
+        each {|leaf| leaf.parse_leafs(breaker) }
       end
     end
 
@@ -175,13 +186,15 @@ module PseudoHiki
       end
     end
 
+    class UnmatchedSectioningTagError < StandardError; end
+
     module BlockElement
       {
-        BlockLeaf => %w(DescLeaf VerbatimLeaf TableLeaf CommentOutLeaf BlockNodeEnd HrLeaf),
+        BlockLeaf => %w(DescLeaf VerbatimLeaf TableLeaf CommentOutLeaf BlockNodeEnd HrLeaf DecoratorLeaf SectioningNodeEnd),
         NonNestedBlockLeaf => %w(QuoteLeaf ParagraphLeaf),
         NestedBlockLeaf => %w(HeadingLeaf),
         ListTypeLeaf => %w(ListLeaf EnumLeaf),
-        BlockNode => %w(DescNode VerbatimNode TableNode CommentOutNode HrNode),
+        BlockNode => %w(DescNode VerbatimNode TableNode CommentOutNode HrNode DecoratorNode SectioningNode),
         NonNestedBlockNode => %w(QuoteNode ParagraphNode),
         NestedBlockNode => %w(HeadingNode),
         ListTypeBlockNode => %w(ListNode EnumNode),
@@ -204,15 +217,100 @@ module PseudoHiki
         attr_accessor :in_block_tag
 
         def add_leaf(line, blockparser)
-          return @stack.pop if VERBATIM_END =~ line
+          return @stack.pop_with_breaker if VERBATIM_END =~ line
           return super(line, blockparser) unless @in_block_tag
           line = " #{line}" if BlockNodeEnd.head_re =~ line and not @in_block_tag
           @stack.push VerbatimLeaf.create(line, @in_block_tag)
         end
       end
 
+      class DecoratorNode
+        DECORATOR_PAT = /\A(?:([^\[\]:]+))?(?:\[([^\[\]]+)\])?(?::\s*(\S.*))?/o
+        LABEL_SEP = [":"]
+
+        class DecoratorItem < Struct.new(:string, :type, :id, :value)
+          def self.create(leaf)
+            m = DECORATOR_PAT.match(leaf.join)
+            return nil unless m
+            args = m.to_a
+            if i = leaf.index(LABEL_SEP)
+              value = leaf.dup
+              value.shift(i + 1)
+              args[-1] = lstrip_value!(value)
+            end
+            self.new(*args)
+          end
+
+          def self.lstrip_value!(value)
+            head_val = value[0][0]
+            if head_val.kind_of? String and head_val.start_with? " ".freeze
+              value[0][0] = head_val.lstrip
+            end
+            value
+          end
+        end
+
+        def parse_leafs(breaker)
+          decorator = {}
+          breaker.decorator = decorator
+          @stack.remove_current_node.each do |leaf|
+            if item = DecoratorItem.create(leaf)
+              decorator[item.type || :id] = item
+            end
+          end
+        end
+
+        def breakable?(breaker)
+          return super if breaker.kind_of? DecoratorLeaf
+          parse_leafs(breaker)
+          @stack.current_node.breakable?(breaker)
+        end
+      end
+
+      class DecoratorLeaf
+        def push_sectioning_node(stack, node_class)
+          node = node_class.new
+          m = DecoratorNode::DECORATOR_PAT.match(join)
+          node.node_id = m[2]
+          node.section_level = stack.current_heading_level if node.kind_of? SectioningNode
+          stack.push(node)
+        end
+
+        def push_self(stack)
+          decorator_type = self[0][0]
+          if decorator_type.start_with? "begin[".freeze
+            push_sectioning_node(stack, SectioningNode)
+          elsif decorator_type.start_with? "end[".freeze
+            push_sectioning_node(stack, SectioningNodeEnd)
+          else
+            super
+          end
+        end
+      end
+
+      class SectioningNode
+        attr_accessor :section_level
+
+        def breakable?(breaker)
+          breaker.kind_of? HeadingLeaf and @section_level >= breaker.level
+        end
+      end
+
+      class SectioningNodeEnd
+        def push_self(stack)
+          n = stack.stack.rindex do |node|
+            node.kind_of? SectioningNode and node.node_id == node_id
+          end
+          raise UnmatchedSectioningTagError unless n
+          stack.pop until stack.stack.length == n
+        rescue UnmatchedSectioningTagError => e
+          STDERR.puts "#{e}: The start tag for '#{node_id}' is not found."
+          # FIXME: The handling of this error should be changed appropriately.
+        end
+      end
+
       class QuoteNode
-        def parse_leafs
+        def parse_leafs(breaker)
           self[0] = BlockParser.parse(self[0], AutoLink::Off)
         end
       end
@@ -273,7 +371,8 @@ module PseudoHiki
      [HrLeaf, HrNode],
      [ListLeaf, ListNode],
      [EnumLeaf, EnumNode],
-     [BlockNodeEnd, BlockNodeEnd] # special case
+     [BlockNodeEnd, BlockNodeEnd], # special case
+     [DecoratorLeaf, DecoratorNode]
     ].each do |leaf, node|
       PARENT_NODE[leaf] = node
     end
@@ -286,6 +385,7 @@ module PseudoHiki
                           ['!', HeadingLeaf],
                           ['""', QuoteLeaf],
                           ['||', TableLeaf],
+                          ['//@', DecoratorLeaf],
                           ['//', CommentOutLeaf],
                           ['----\s*$', HrLeaf]]
 
@@ -331,7 +431,7 @@ module PseudoHiki
     def read_lines(lines)
       each_line = lines.respond_to?(:each_line) ? :each_line : :each
       lines.send(each_line) {|line| @stack.current_node.add_leaf(line, self) }
-      @stack.pop
+      @stack.pop_with_breaker
     end
   end