marker 0.3.3 → 0.4.0

data/README

@@ -1,4 +1,6 @@
-== Marker
+{{ style }}
+
+= Marker =
 
 Marker is a markup language parser designed for two needs:
 # Need to mimic MediaWiki syntax
@@ -10,22 +12,24 @@ be very difficult to exactly copy the MediaWiki parser and it probably wouldn't
 be worth the time because MediaWiki is intended for a wiki and needs to be
 adapted to be used as a markup lanaguage--especially for multiple output
 formats.  The purpose of mimicing MediaWiki syntax is so that users don't have
-to learn more than one markup language, so the implementation doesn't *need* to
-be exact anyway.
+to learn more than one markup language, so the implementation doesn't ''need''
+to be exact anyway.
 
 Marker differs from MediaWiki in several ways, because it is a grammar-based
-implementation.  The grammar is written as a
-Treetop[http://treetop.rubyforge.org/] parsing expression grammar (PEG).
+implementation.  The grammar is written as a [http://treetop.rubyforge.org/ Treetop]
+parsing expression grammar
+([http://en.wikipedia.org/wiki/Parsing_expression_grammar PEG]).
 
 Not implemented:
 # Table of contents
 # Tables
 
-== Use
+== Use ==
 
 Parsing is done with either Marker.parse or Marker.parse_file.  Both parse
-methods will return a parse tree that has to_html and to_s methods that
-"render" the markup.  Both render methods will accept an options hash.
+methods will return a parse tree that has {{tt|to_html}} and {{tt|to_s}}
+methods that "render" the markup.  Both render methods will accept an options
+hash.
 
 Example:
  >> require 'marker'
@@ -42,13 +46,14 @@ Example:
  <p>paragraph with <b>bold</b> text</p>
  => nil
 
-=== Templates
+=== Templates ===
 
 Templates are implemented as method calls to a templates module.  Each method
 in the templates module is considered a template and can be called using the
-"{{ name }}" syntax.  Each template method is expected to take three arguments:
-the render format (:html or :text), an array of positional parameters, and a
-hash of named parameters.  For example,
+"{{ template | template_name }}" syntax.  Each template method is
+expected to take three arguments: the render format ({{tt|:html}} or
+{{tt|:text}}), an array of positional parameters, and a hash of named
+parameters.  For example,
  module MyTemplates
    def logo( format, pos_params, name_params )
      case format
@@ -60,7 +65,7 @@ hash of named parameters.  For example,
    end
  end
 
-Template modules are passed to Marker by setting the +templates+ property:
+Template modules are passed to Marker by setting the {{tt|templates}} property:
  require 'my_templates'
  require 'marker'
  
@@ -76,14 +81,14 @@ insensitive for markup writers.  For example,
  "{{ My Template }}"  => :my_template
  "{{NaMe}}"           => :name
 
-=== Internal Links
+=== Internal Links ===
 
 Internal links are implemented as links with default prefixes.  The link prefix
-is specified by setting the +link_base+ property:
+is specified by setting the {{tt|link_base}} property:
  require 'marker'
  
  Marker.link_base = 'http://example.com/pages/'
-
+ 
  >> puts Marker.parse( '[[target|name]]' ).to_html
  <p><a href='http://example.com/pages/target'>name</a></p>
 
@@ -91,18 +96,19 @@ The link target is appended to the link prefix, along with a beginning '/'.  If
 no link base is given, links are just the link target with a beginning '/'.
 The link base can also be given as a render option.
 
-=== Unlabelled Links
+=== Unlabelled Links ===
 
-(write me)
+Unlabelled, or "bare" links are detected if they start with a recognized URL
+scheme such as {{tt|http}} or {{tt|https}}.  The URL is used as the link text.
 
-== Command Line Program
+== Command Line Program ==
 
 Marker comes with a command-line program that will render both HTML and text.
 If no input file is given, it reads from stdin.
 
  Usage: marker [--format (html|text)] [input-file]
 
-== License
+== License ==
 
 Marker is copyright 2009 Ryan Blue and distributed under the terms of the GNU
 General Public License (GPL).  See the LICENSE file for further information on

data/lib/marker/common.rb

@@ -36,5 +36,9 @@ module Marker #:nodoc:
     def r
       nil
     end
+
+    def single?
+      r.nil?
+    end
   end
 end

data/lib/marker/language.treetop

@@ -1,35 +1,24 @@
 #--
-# Copyright 2009 Ryan Blue.
+# Copyright 2009-2012 Ryan Blue.
 # Distributed under the terms of the GNU General Public License (GPL).
 # See the LICENSE file for further information on the GPL.
 #++
 
 module Marker
   grammar Language
-    rule markup
-      h:block ws rnl r:markup <Markup>
+    rule document
+      rnl* markup aws <Document>
       /
-      h:block ws <Markup>
+      aws "" <Document>
     end
 
-    rule block
-      line
-      /
-      text
+    rule markup
+      h:block (ws rnl)+ r:markup <Markup>
       /
-      ws { # allows blank lines
-        def to_html( options = {} )
-          ""
-        end
-
-        def to_s( options = {} )
-          ""
-        end
-      }
+      h:block aws <Markup>
     end
 
-    ##### special lines
-    rule line
+    rule block
       heading
       /
       list
@@ -37,8 +26,82 @@ module Marker
       verbatim_area
       /
       horiz_rule
+      /
+      known_text_paragraph
+    end
+
+    ##### templates
+
+    rule template
+      template_start aws t:plain_phrase aws arg_delimiter ws args:arg_list aws template_end <Template>
+      /
+      template_start aws t:plain_phrase aws template_end <Template>
+    end
+
+    rule arg_list
+      h:arg aws arg_delimiter ws r:arg_list <Arguments>
+      /
+      h:arg "" <Arguments>
+    end
+
+    # a plain (positional) argument or a named (name=text) argument
+    #
+    # there are two types of values matched here, inline and block.  inline
+    # values are plain text areas that can be included in the containing
+    # paragraph.  block values are things like lists and groups of paragraphs
+    # that would normally be rendered as paragraphs themselves.  this
+    # distinction is necessary to gracefully handle different types of uses for
+    # templates.  examples:
+    #
+    # inline: text with a fixed-width {{tt|word}} embedded.
+    # block:
+    # {{ ruby |
+    # def /( str )
+    #   return File.join( self, str )
+    # end
+    # }}
+    rule arg
+      name:plain_phrase ws heading_toggle ws val:arg_safe_known_markup <BlockArgument>
+      /
+      name:plain_phrase ws heading_toggle ws val:arg_safe_known_text <InlineArgument>
+      /
+      val:arg_safe_known_markup "" <BlockArgument>
+      /
+      val:arg_safe_known_text "" <InlineArgument>
+      /
+      "" "" <InlineArgument>
+    end
+
+    ##### template-safe markup definitions
+
+    # this is used to start lines with unsafe text and continue into a section
+    # of markup
+    rule arg_safe_known_markup
+      rnl h:arg_safe_markup <Markup>
+      /
+      h:arg_safe_known_text_paragraph (ws rnl)+ r:arg_safe_markup <Markup>
+    end
+
+    rule arg_safe_markup
+      h:arg_safe_block (ws rnl)+ r:arg_safe_markup <Markup>
+      /
+      h:arg_safe_block aws <Markup>
+    end
+
+    rule arg_safe_block
+      arg_safe_heading
+      /
+      arg_safe_list
+      /
+      arg_safe_verbatim_area
+      /
+      arg_safe_horiz_rule
+      /
+      arg_safe_known_text_paragraph
     end
 
+    ##### headings
+
     rule heading
       s:heading_toggle+ ws l:heading_enclosed_text ws e:heading_toggle+ <Heading>
     end
@@ -85,6 +148,52 @@ module Marker
       term_delimiter
     end
 
+    rule arg_safe_heading
+      s:heading_toggle+ ws l:arg_safe_heading_enclosed_text ws e:heading_toggle+ <Heading>
+    end
+
+    rule arg_safe_heading_enclosed_text
+      h:heading_toggle ws r:arg_safe_heading_enclosed_text <Phrase>
+      /
+      h:arg_safe_heading_enclosed_word ws r:arg_safe_heading_enclosed_text <Phrase>
+      /
+      h:arg_safe_heading_enclosed_word "" <Phrase>
+    end
+
+    rule arg_safe_heading_enclosed_word
+      # equivalent to:
+      # !(template_end / arg_delimiter) heading_enclosed_word
+      bold
+      /
+      italic
+      /
+      link
+      /
+      template
+      /
+      url
+      /
+      plain_word
+      /
+      bold_toggle
+      /
+      italic_toggle
+      /
+      internal_link_start
+      /
+      internal_link_end
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+      /
+      term_delimiter
+    end
+
+    ##### general lists
+
     # a series of list items, so they can be coalesced
     rule list
       h:list_item ws rnl r:list <List>
@@ -104,31 +213,69 @@ module Marker
     end
 
     rule bulleted
-      '*' ws list_item <Bulleted>
+      '*' ws item:list_item <Bulleted>
       /
-      '*' ws phrase <Bulleted>
+      '*' ws content:phrase <Bulleted>
     end
 
     rule numbered
-      '#' ws list_item <Numbered>
+      '#' ws item:list_item <Numbered>
       /
-      '#' ws phrase <Numbered>
+      '#' ws content:phrase <Numbered>
     end
 
     rule indented
-      ':' ws list_item <Indented>
+      ':' ws item:list_item <Indented>
+      /
+      ':' ws content:phrase <Indented>
+    end
+
+    # a series of list items, so they can be coalesced
+    rule arg_safe_list
+      h:arg_safe_list_item ws rnl r:arg_safe_list <List>
       /
-      ':' ws phrase <Indented>
+      h:arg_safe_list_item ws <List>
     end
 
+    # for matching any list item
+    rule arg_safe_list_item
+      arg_safe_bulleted
+      /
+      arg_safe_numbered
+      /
+      arg_safe_indented
+      /
+      arg_safe_definition
+    end
+
+    rule arg_safe_bulleted
+      '*' ws item:arg_safe_list_item <Bulleted>
+      /
+      '*' ws content:arg_safe_phrase <Bulleted>
+    end
+
+    rule arg_safe_numbered
+      '#' ws item:arg_safe_list_item <Numbered>
+      /
+      '#' ws content:arg_safe_phrase <Numbered>
+    end
+
+    rule arg_safe_indented
+      ':' ws item:arg_safe_list_item <Indented>
+      /
+      ':' ws content:arg_safe_phrase <Indented>
+    end
+
+    ##### definition lists
+
     rule definition
-      ';' ws term:definition_term_text ws term_delimiter ws definition:phrase <Definition>
+      ';' ws term:definition_term_phrase ws term_delimiter ws definition:phrase <Definition>
       /
       ';' ws term:phrase <Definition>
     end
 
-    rule definition_term_text
-      h:definition_term_word ws r:definition_term_text <Phrase>
+    rule definition_term_phrase
+      h:definition_term_word ws r:definition_term_phrase <Phrase>
       /
       h:definition_term_word "" <Phrase>
     end
@@ -167,6 +314,50 @@ module Marker
       arg_delimiter
     end
 
+    rule arg_safe_definition
+      ';' ws term:arg_safe_definition_term_phrase ws term_delimiter ws definition:arg_safe_phrase <Definition>
+      /
+      ';' ws term:arg_safe_phrase <Definition>
+    end
+
+    rule arg_safe_definition_term_phrase
+      h:arg_safe_definition_term_word ws r:arg_safe_definition_term_phrase <Phrase>
+      /
+      h:arg_safe_definition_term_word "" <Phrase>
+    end
+
+    rule arg_safe_definition_term_word
+      arg_safe_bold
+      /
+      arg_safe_italic
+      /
+      arg_safe_link
+      /
+      template
+      /
+      url
+      /
+      plain_word
+      /
+      bold_toggle
+      /
+      italic_toggle
+      /
+      heading_toggle
+      /
+      internal_link_start
+      /
+      internal_link_end
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+    end
+
+    ##### verbatim areas
+
     rule verbatim_area
       h:verbatim rnl r:verbatim_area <VerbatimArea>
       /
@@ -175,29 +366,80 @@ module Marker
 
     rule verbatim
       " " v:(!"\n" .)* <Verbatim>
+      /
+      "\t" v:(!"\n" .)* <Verbatim>
+    end
+
+    rule arg_safe_verbatim_area
+      h:arg_safe_verbatim rnl r:arg_safe_verbatim_area <VerbatimArea>
+      /
+      h:arg_safe_verbatim "" <VerbatimArea>
     end
 
+    rule arg_safe_verbatim
+      " " v:( !("\n" / arg_delimiter / template_end) .)* <Verbatim>
+      /
+      "\t" v:(!"\n" .)* <Verbatim>
+    end
+
+    ##### horizontal rules
+
+    # horizontal rules deviate from how MediaWiki does things.
+    # consider this:
+    #   ---- a para-
+    #   graph
+    # MediaWiki renders (newlines removed): <hr /><p>a para-</p><p>graph</p>
+    # this renders (newlines removed): <hr /><p>a para-graph</p>
+
     rule horiz_rule
-      # this deviates from how MediaWiki does things.  consider this:
-      #   ---- a para-
-      #   graph
-      # MediaWiki renders (newlines removed): <hr /><p>a para-</p><p>graph</p>
-      # this renders (newlines removed): <hr /><p>a para-graph</p>
-      "----" "-"* ws t:unsafe_text <HorizRule>
+      "----" "-"* ws t:known_text_paragraph <HorizRule>
+      /
+      "----" "-"* <HorizRule>
+    end
+
+    rule arg_safe_horiz_rule
+      "----" "-"* ws t:arg_safe_known_text_paragraph <HorizRule>
       /
       "----" "-"* <HorizRule>
     end
 
     ##### combination rules
 
+    rule paragraph
+      text "" <Paragraph>
+    end
+
+    rule arg_safe_paragraph
+      text:arg_safe_text "" <Paragraph>
+    end
+
     # a block of normal text, including single newlines but not a special line
     rule text
-      !line h:phrase ws rnl r:text <Paragraph>
+      !(heading / list / verbatim_area / horiz_rule) h:phrase ws rnl r:text <TextBlock>
+      /
+      !(heading / list / verbatim_area / horiz_rule) h:phrase ws <TextBlock>
+    end
+
+    # arg-safe variant
+    rule arg_safe_text
+      !(arg_safe_heading / arg_safe_list / arg_safe_verbatim_area / arg_safe_horiz_rule) h:arg_safe_phrase ws rnl r:arg_safe_text <TextBlock>
       /
-      !line h:phrase ws <Paragraph>
+      !(arg_safe_heading / arg_safe_list / arg_safe_verbatim_area / arg_safe_horiz_rule) h:arg_safe_phrase ws <TextBlock>
+    end
+
+    # paragraph wrapper for known text
+    rule known_text_paragraph
+      text:known_text "" <Paragraph>
+    end
+
+    # paragraph wrapper for arg-safe known text
+    rule arg_safe_known_text_paragraph
+      text:arg_safe_known_text "" <Paragraph>
     end
 
-    # a block of text that does not require a !line check
+    # "known" text a block of text that does not require a !line check
+    #
+    # in other words, it is known that this block is text and not line-anchored
     #
     # for situations where text may follow structures that are normally on a
     # single line---e.g., horizontal rules--- the text should be handled like a
@@ -205,12 +447,20 @@ module Marker
     # the text starts like a special line: cases like this: "---- ----".
     # Because it follows a structure without a new line, it is safe to assume
     # the text is a phrase.
-    rule unsafe_text
-      h:phrase ws rnl r:text <Paragraph>
+    rule known_text
+      h:phrase ws rnl r:text <TextBlock>
       /
-      h:phrase ws <Paragraph>
+      h:phrase ws <TextBlock>
     end
 
+    rule arg_safe_known_text
+      h:arg_safe_phrase ws rnl r:arg_safe_text <TextBlock>
+      /
+      h:arg_safe_phrase ws <TextBlock>
+    end
+
+    ##### general words and phrases
+
     # a phrase of words without newlines
     rule phrase
       h:word ws r:phrase <Phrase>
@@ -218,6 +468,7 @@ module Marker
       h:word "" <Phrase>
     end
 
+    # words not inside any container
     rule word
       bold
       /
@@ -231,10 +482,70 @@ module Marker
       /
       plain_word
       /
-      delimiter # catch all remaining unmatched delimiters
+      bold_toggle
+      /
+      italic_toggle
+      /
+      heading_toggle
+      /
+      internal_link_start
+      /
+      internal_link_end
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+      /
+      template_end
+      /
+      arg_delimiter
+      /
+      term_delimiter
     end
 
-    ##### text markup
+    # a phrase of arg-safe words without newlines
+    rule arg_safe_phrase
+      h:arg_safe_word ws r:arg_safe_phrase <Phrase>
+      /
+      h:arg_safe_word "" <Phrase>
+    end
+
+    # words inside templates but not other markup
+    rule arg_safe_word
+      arg_safe_bold
+      /
+      arg_safe_italic
+      /
+      arg_safe_link
+      /
+      template
+      /
+      url
+      /
+      plain_word
+      /
+      bold_toggle
+      /
+      italic_toggle
+      /
+      heading_toggle
+      /
+      internal_link_start
+      /
+      internal_link_end
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+      /
+      term_delimiter
+    end
+
+    ##### bold
 
     rule bold
       bold_toggle ws bold_enclosed_text ws bold_toggle <Bold>
@@ -277,6 +588,45 @@ module Marker
       term_delimiter
     end
 
+    rule arg_safe_bold
+      bold_toggle ws bold_enclosed_text:arg_safe_bold_enclosed_text ws bold_toggle <Bold>
+    end
+
+    rule arg_safe_bold_enclosed_text
+      h:arg_safe_bold_enclosed_word ws r:arg_safe_bold_enclosed_text <Phrase>
+      /
+      h:arg_safe_bold_enclosed_word "" <Phrase>
+    end
+
+    # things that can be inside arg-safe bold areas
+    rule arg_safe_bold_enclosed_word
+      arg_safe_italic
+      /
+      arg_safe_link
+      /
+      template
+      /
+      url
+      /
+      plain_word
+      /
+      heading_toggle
+      /
+      internal_link_start
+      /
+      internal_link_end
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+      /
+      term_delimiter
+    end
+
+    ##### italics
+
     rule italic
       italic_toggle ws italic_enclosed_text ws italic_toggle_no_lookahead <Italic>
     end
@@ -318,23 +668,71 @@ module Marker
       term_delimiter
     end
 
+    rule arg_safe_italic
+      italic_toggle ws italic_enclosed_text:arg_safe_italic_enclosed_text ws italic_toggle_no_lookahead <Italic>
+    end
+
+    rule arg_safe_italic_enclosed_text
+      h:arg_safe_italic_enclosed_word ws r:arg_safe_italic_enclosed_text <Phrase>
+      /
+      h:arg_safe_italic_enclosed_word "" <Phrase>
+    end
+
+    # things that can be inside arg-safe italic areas
+    rule arg_safe_italic_enclosed_word
+      # this is equivalent to:
+      # !(template_end / arg_delimiter) italic_enclosed_word
+      arg_safe_bold
+      /
+      arg_safe_link
+      /
+      template
+      /
+      url
+      /
+      plain_word
+      /
+      heading_toggle
+      /
+      internal_link_start
+      /
+      internal_link_end
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+      /
+      term_delimiter
+    end
+
+    ##### links
+
     rule link
       internal_link
       /
       external_link
     end
 
-    # internal links:
+    rule arg_safe_link
+      arg_safe_internal_link
+      /
+      arg_safe_external_link
+    end
+
+    ##### internal links
+
     # [[ link target | link label ]]
     # * can contain white space
     # * cannot contain new lines
-    #-- TODO: handle [[ url | label ]] and variants ++
+
     rule internal_link
-      internal_link_start ws t:plain_text ws a:arg_delimiter ws l:internal_link_enclosed_text ws internal_link_end <InternalLink>
+      internal_link_start ws t:plain_phrase ws a:arg_delimiter ws l:internal_link_enclosed_text ws internal_link_end <InternalLink>
       /
-      internal_link_start ws t:plain_text ws a:arg_delimiter ws internal_link_end <InternalLink>
+      internal_link_start ws t:plain_phrase ws a:arg_delimiter ws internal_link_end <InternalLink>
       /
-      internal_link_start ws t:plain_text ws internal_link_end <InternalLink>
+      internal_link_start ws t:plain_phrase ws internal_link_end <InternalLink>
     end
 
     rule internal_link_enclosed_text
@@ -373,7 +771,50 @@ module Marker
       term_delimiter
     end
 
-    # external links:
+    rule arg_safe_internal_link
+      internal_link_start ws t:plain_phrase ws a:arg_delimiter ws l:arg_safe_internal_link_enclosed_text ws internal_link_end <InternalLink>
+      /
+      internal_link_start ws t:plain_phrase ws a:arg_delimiter ws internal_link_end <InternalLink>
+      /
+      internal_link_start ws t:plain_phrase ws internal_link_end <InternalLink>
+    end
+
+    rule arg_safe_internal_link_enclosed_text
+      h:arg_safe_internal_link_enclosed_word ws r:arg_safe_internal_link_enclosed_text <Phrase>
+      /
+      h:arg_safe_internal_link_enclosed_word "" <Phrase>
+    end
+
+    rule arg_safe_internal_link_enclosed_word
+      # this is equivalent to:
+      # !(template_end / arg_delimiter) internal_link_enclosed_word
+      arg_safe_bold
+      /
+      arg_safe_italic
+      /
+      template
+      /
+      plain_word
+      /
+      bold_toggle
+      /
+      italic_toggle
+      /
+      heading_toggle
+      /
+      internal_link_start
+      /
+      external_link_start
+      /
+      external_link_end
+      /
+      template_start
+      /
+      term_delimiter
+    end
+
+    ##### external links
+
     # [ url label ]
     # * can contain white space
     # * cannot contain new lines
@@ -385,6 +826,7 @@ module Marker
     #
     # also use this rule to match bad internal link syntax [[ url | label ]]
     # because the rendering rules are the same
+
     rule external_link
 #      external_link_start ws t:url rws l:external_link_enclosed_text ws external_link_end <ExternalLink>
 #      /
@@ -439,39 +881,34 @@ module Marker
       term_delimiter
     end
 
-    rule template
-      template_start aws t:plain_text aws arg_delimiter aws args:arg_list aws template_end <Template>
+    rule arg_safe_external_link
+#      external_link_start ws t:url rws l:external_link_enclosed_text ws external_link_end <ExternalLink>
+#      /
+#      external_link_start ws t:url ws external_link_end <ExternalLink>
+#      /
+      external_link_start ws t:implicit_url rws l:arg_safe_external_link_enclosed_text ws external_link_end <ExternalLink>
       /
-      template_start aws t:plain_text aws template_end <Template>
-    end
-
-    rule arg_list
-      h:arg aws arg_delimiter aws r:arg_list <Arguments>
+      external_link_start ws t:implicit_url ws external_link_end <ExternalLink>
       /
-      h:arg "" <Arguments>
-    end
-
-    # a plain (positional) argument or a named (name=text) argument
-    rule arg
-      name:plain_text aws heading_toggle aws val:arg_list_enclosed_text <Argument>
+      internal_link_start ws t:implicit_url ws arg_delimiter ws l:arg_safe_internal_link_enclosed_text ws internal_link_end <ExternalLink>
       /
-      val:arg_list_enclosed_text "" <Argument>
+      internal_link_start ws t:implicit_url rws l:arg_safe_internal_link_enclosed_text ws internal_link_end <ExternalLink>
       /
-      "" "" <Argument>
+      internal_link_start ws t:implicit_url ws internal_link_end <ExternalLink>
     end
 
-    rule arg_list_enclosed_text
-      h:arg_list_enclosed_word aws r:arg_list_enclosed_text <Phrase>
+    rule arg_safe_external_link_enclosed_text
+      h:arg_safe_external_link_enclosed_word ws r:arg_safe_external_link_enclosed_text <Phrase>
       /
-      h:arg_list_enclosed_word "" <Phrase>
+      h:arg_safe_external_link_enclosed_word "" <Phrase>
     end
 
-    rule arg_list_enclosed_word
-      bold
+    rule arg_safe_external_link_enclosed_word
+      # this is equivalent to:
+      # !(template_end / arg_delimiter) external_link_enclosed_word
+      arg_safe_bold
       /
-      italic
-      /
-      link
+      arg_safe_italic
       /
       template
       /
@@ -479,7 +916,7 @@ module Marker
       /
       plain_word
       /
-      bold_toggle # allow unmatched delimiters after we have ruled out structures
+      bold_toggle
       /
       italic_toggle
       /
@@ -491,25 +928,25 @@ module Marker
       /
       external_link_start
       /
-      external_link_end
-      /
       template_start
       /
       term_delimiter
     end
 
+    ##### plain words and phrases
+
     # a phrase of plain words
-    rule plain_text
-      h:plain_word ws r:plain_text <Phrase>
+    rule plain_phrase
+      h:plain_word ws r:plain_phrase <Phrase>
       /
       h:plain_word "" <Phrase>
     end
 
     # avoids using a white-list so that utf-8 characters are accepted
     #
-    # matches anything up to a delimiter or whitespace
+    # matches anything other than a delimiter or whitespace
     rule plain_word
-      (!delimiter ![ \t\r\n] .)+ <Word>
+      (!delimiter [^ \t\r\n])+ <Word>
     end
 
     ##### delimiters