marker 0.3.3 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README +26 -20
- data/lib/marker/common.rb +4 -0
- data/lib/marker/language.treetop +515 -78
- data/lib/marker/lists.rb +142 -112
- data/lib/marker/markup.rb +34 -16
- data/lib/marker/templates.rb +51 -4
- data/lib/marker/text.rb +37 -5
- data/test/encodings_test.rb +3 -3
- data/test/formatting_test.rb +1 -1
- data/test/helper.rb +9 -0
- data/test/lists_test.rb +12 -5
- data/test/templates_test.rb +138 -23
- data/test/test_helper.rb +12 -1
- metadata +58 -59
data/README
CHANGED
@@ -1,4 +1,6 @@
|
|
1
|
-
|
1
|
+
{{ style }}
|
2
|
+
|
3
|
+
= Marker =
|
2
4
|
|
3
5
|
Marker is a markup language parser designed for two needs:
|
4
6
|
# Need to mimic MediaWiki syntax
|
@@ -10,22 +12,24 @@ be very difficult to exactly copy the MediaWiki parser and it probably wouldn't
|
|
10
12
|
be worth the time because MediaWiki is intended for a wiki and needs to be
|
11
13
|
adapted to be used as a markup lanaguage--especially for multiple output
|
12
14
|
formats. The purpose of mimicing MediaWiki syntax is so that users don't have
|
13
|
-
to learn more than one markup language, so the implementation doesn't
|
14
|
-
be exact anyway.
|
15
|
+
to learn more than one markup language, so the implementation doesn't ''need''
|
16
|
+
to be exact anyway.
|
15
17
|
|
16
18
|
Marker differs from MediaWiki in several ways, because it is a grammar-based
|
17
|
-
implementation. The grammar is written as a
|
18
|
-
|
19
|
+
implementation. The grammar is written as a [http://treetop.rubyforge.org/ Treetop]
|
20
|
+
parsing expression grammar
|
21
|
+
([http://en.wikipedia.org/wiki/Parsing_expression_grammar PEG]).
|
19
22
|
|
20
23
|
Not implemented:
|
21
24
|
# Table of contents
|
22
25
|
# Tables
|
23
26
|
|
24
|
-
== Use
|
27
|
+
== Use ==
|
25
28
|
|
26
29
|
Parsing is done with either Marker.parse or Marker.parse_file. Both parse
|
27
|
-
methods will return a parse tree that has to_html and to_s
|
28
|
-
"render" the markup. Both render methods will accept an options
|
30
|
+
methods will return a parse tree that has {{tt|to_html}} and {{tt|to_s}}
|
31
|
+
methods that "render" the markup. Both render methods will accept an options
|
32
|
+
hash.
|
29
33
|
|
30
34
|
Example:
|
31
35
|
>> require 'marker'
|
@@ -42,13 +46,14 @@ Example:
|
|
42
46
|
<p>paragraph with <b>bold</b> text</p>
|
43
47
|
=> nil
|
44
48
|
|
45
|
-
=== Templates
|
49
|
+
=== Templates ===
|
46
50
|
|
47
51
|
Templates are implemented as method calls to a templates module. Each method
|
48
52
|
in the templates module is considered a template and can be called using the
|
49
|
-
"{{
|
50
|
-
the render format (
|
51
|
-
|
53
|
+
"{{ template | template_name }}" syntax. Each template method is
|
54
|
+
expected to take three arguments: the render format ({{tt|:html}} or
|
55
|
+
{{tt|:text}}), an array of positional parameters, and a hash of named
|
56
|
+
parameters. For example,
|
52
57
|
module MyTemplates
|
53
58
|
def logo( format, pos_params, name_params )
|
54
59
|
case format
|
@@ -60,7 +65,7 @@ hash of named parameters. For example,
|
|
60
65
|
end
|
61
66
|
end
|
62
67
|
|
63
|
-
Template modules are passed to Marker by setting the
|
68
|
+
Template modules are passed to Marker by setting the {{tt|templates}} property:
|
64
69
|
require 'my_templates'
|
65
70
|
require 'marker'
|
66
71
|
|
@@ -76,14 +81,14 @@ insensitive for markup writers. For example,
|
|
76
81
|
"{{ My Template }}" => :my_template
|
77
82
|
"{{NaMe}}" => :name
|
78
83
|
|
79
|
-
=== Internal Links
|
84
|
+
=== Internal Links ===
|
80
85
|
|
81
86
|
Internal links are implemented as links with default prefixes. The link prefix
|
82
|
-
is specified by setting the
|
87
|
+
is specified by setting the {{tt|link_base}} property:
|
83
88
|
require 'marker'
|
84
89
|
|
85
90
|
Marker.link_base = 'http://example.com/pages/'
|
86
|
-
|
91
|
+
|
87
92
|
>> puts Marker.parse( '[[target|name]]' ).to_html
|
88
93
|
<p><a href='http://example.com/pages/target'>name</a></p>
|
89
94
|
|
@@ -91,18 +96,19 @@ The link target is appended to the link prefix, along with a beginning '/'. If
|
|
91
96
|
no link base is given, links are just the link target with a beginning '/'.
|
92
97
|
The link base can also be given as a render option.
|
93
98
|
|
94
|
-
=== Unlabelled Links
|
99
|
+
=== Unlabelled Links ===
|
95
100
|
|
96
|
-
|
101
|
+
Unlabelled, or "bare" links are detected if they start with a recognized URL
|
102
|
+
scheme such as {{tt|http}} or {{tt|https}}. The URL is used as the link text.
|
97
103
|
|
98
|
-
== Command Line Program
|
104
|
+
== Command Line Program ==
|
99
105
|
|
100
106
|
Marker comes with a command-line program that will render both HTML and text.
|
101
107
|
If no input file is given, it reads from stdin.
|
102
108
|
|
103
109
|
Usage: marker [--format (html|text)] [input-file]
|
104
110
|
|
105
|
-
== License
|
111
|
+
== License ==
|
106
112
|
|
107
113
|
Marker is copyright 2009 Ryan Blue and distributed under the terms of the GNU
|
108
114
|
General Public License (GPL). See the LICENSE file for further information on
|
data/lib/marker/common.rb
CHANGED
data/lib/marker/language.treetop
CHANGED
@@ -1,35 +1,24 @@
|
|
1
1
|
#--
|
2
|
-
# Copyright 2009 Ryan Blue.
|
2
|
+
# Copyright 2009-2012 Ryan Blue.
|
3
3
|
# Distributed under the terms of the GNU General Public License (GPL).
|
4
4
|
# See the LICENSE file for further information on the GPL.
|
5
5
|
#++
|
6
6
|
|
7
7
|
module Marker
|
8
8
|
grammar Language
|
9
|
-
rule
|
10
|
-
|
9
|
+
rule document
|
10
|
+
rnl* markup aws <Document>
|
11
11
|
/
|
12
|
-
|
12
|
+
aws "" <Document>
|
13
13
|
end
|
14
14
|
|
15
|
-
rule
|
16
|
-
|
17
|
-
/
|
18
|
-
text
|
15
|
+
rule markup
|
16
|
+
h:block (ws rnl)+ r:markup <Markup>
|
19
17
|
/
|
20
|
-
|
21
|
-
def to_html( options = {} )
|
22
|
-
""
|
23
|
-
end
|
24
|
-
|
25
|
-
def to_s( options = {} )
|
26
|
-
""
|
27
|
-
end
|
28
|
-
}
|
18
|
+
h:block aws <Markup>
|
29
19
|
end
|
30
20
|
|
31
|
-
|
32
|
-
rule line
|
21
|
+
rule block
|
33
22
|
heading
|
34
23
|
/
|
35
24
|
list
|
@@ -37,8 +26,82 @@ module Marker
|
|
37
26
|
verbatim_area
|
38
27
|
/
|
39
28
|
horiz_rule
|
29
|
+
/
|
30
|
+
known_text_paragraph
|
31
|
+
end
|
32
|
+
|
33
|
+
##### templates
|
34
|
+
|
35
|
+
rule template
|
36
|
+
template_start aws t:plain_phrase aws arg_delimiter ws args:arg_list aws template_end <Template>
|
37
|
+
/
|
38
|
+
template_start aws t:plain_phrase aws template_end <Template>
|
39
|
+
end
|
40
|
+
|
41
|
+
rule arg_list
|
42
|
+
h:arg aws arg_delimiter ws r:arg_list <Arguments>
|
43
|
+
/
|
44
|
+
h:arg "" <Arguments>
|
45
|
+
end
|
46
|
+
|
47
|
+
# a plain (positional) argument or a named (name=text) argument
|
48
|
+
#
|
49
|
+
# there are two types of values matched here, inline and block. inline
|
50
|
+
# values are plain text areas that can be included in the containing
|
51
|
+
# paragraph. block values are things like lists and groups of paragraphs
|
52
|
+
# that would normally be rendered as paragraphs themselves. this
|
53
|
+
# distinction is necessary to gracefully handle different types of uses for
|
54
|
+
# templates. examples:
|
55
|
+
#
|
56
|
+
# inline: text with a fixed-width {{tt|word}} embedded.
|
57
|
+
# block:
|
58
|
+
# {{ ruby |
|
59
|
+
# def /( str )
|
60
|
+
# return File.join( self, str )
|
61
|
+
# end
|
62
|
+
# }}
|
63
|
+
rule arg
|
64
|
+
name:plain_phrase ws heading_toggle ws val:arg_safe_known_markup <BlockArgument>
|
65
|
+
/
|
66
|
+
name:plain_phrase ws heading_toggle ws val:arg_safe_known_text <InlineArgument>
|
67
|
+
/
|
68
|
+
val:arg_safe_known_markup "" <BlockArgument>
|
69
|
+
/
|
70
|
+
val:arg_safe_known_text "" <InlineArgument>
|
71
|
+
/
|
72
|
+
"" "" <InlineArgument>
|
73
|
+
end
|
74
|
+
|
75
|
+
##### template-safe markup definitions
|
76
|
+
|
77
|
+
# this is used to start lines with unsafe text and continue into a section
|
78
|
+
# of markup
|
79
|
+
rule arg_safe_known_markup
|
80
|
+
rnl h:arg_safe_markup <Markup>
|
81
|
+
/
|
82
|
+
h:arg_safe_known_text_paragraph (ws rnl)+ r:arg_safe_markup <Markup>
|
83
|
+
end
|
84
|
+
|
85
|
+
rule arg_safe_markup
|
86
|
+
h:arg_safe_block (ws rnl)+ r:arg_safe_markup <Markup>
|
87
|
+
/
|
88
|
+
h:arg_safe_block aws <Markup>
|
89
|
+
end
|
90
|
+
|
91
|
+
rule arg_safe_block
|
92
|
+
arg_safe_heading
|
93
|
+
/
|
94
|
+
arg_safe_list
|
95
|
+
/
|
96
|
+
arg_safe_verbatim_area
|
97
|
+
/
|
98
|
+
arg_safe_horiz_rule
|
99
|
+
/
|
100
|
+
arg_safe_known_text_paragraph
|
40
101
|
end
|
41
102
|
|
103
|
+
##### headings
|
104
|
+
|
42
105
|
rule heading
|
43
106
|
s:heading_toggle+ ws l:heading_enclosed_text ws e:heading_toggle+ <Heading>
|
44
107
|
end
|
@@ -85,6 +148,52 @@ module Marker
|
|
85
148
|
term_delimiter
|
86
149
|
end
|
87
150
|
|
151
|
+
rule arg_safe_heading
|
152
|
+
s:heading_toggle+ ws l:arg_safe_heading_enclosed_text ws e:heading_toggle+ <Heading>
|
153
|
+
end
|
154
|
+
|
155
|
+
rule arg_safe_heading_enclosed_text
|
156
|
+
h:heading_toggle ws r:arg_safe_heading_enclosed_text <Phrase>
|
157
|
+
/
|
158
|
+
h:arg_safe_heading_enclosed_word ws r:arg_safe_heading_enclosed_text <Phrase>
|
159
|
+
/
|
160
|
+
h:arg_safe_heading_enclosed_word "" <Phrase>
|
161
|
+
end
|
162
|
+
|
163
|
+
rule arg_safe_heading_enclosed_word
|
164
|
+
# equivalent to:
|
165
|
+
# !(template_end / arg_delimiter) heading_enclosed_word
|
166
|
+
bold
|
167
|
+
/
|
168
|
+
italic
|
169
|
+
/
|
170
|
+
link
|
171
|
+
/
|
172
|
+
template
|
173
|
+
/
|
174
|
+
url
|
175
|
+
/
|
176
|
+
plain_word
|
177
|
+
/
|
178
|
+
bold_toggle
|
179
|
+
/
|
180
|
+
italic_toggle
|
181
|
+
/
|
182
|
+
internal_link_start
|
183
|
+
/
|
184
|
+
internal_link_end
|
185
|
+
/
|
186
|
+
external_link_start
|
187
|
+
/
|
188
|
+
external_link_end
|
189
|
+
/
|
190
|
+
template_start
|
191
|
+
/
|
192
|
+
term_delimiter
|
193
|
+
end
|
194
|
+
|
195
|
+
##### general lists
|
196
|
+
|
88
197
|
# a series of list items, so they can be coalesced
|
89
198
|
rule list
|
90
199
|
h:list_item ws rnl r:list <List>
|
@@ -104,31 +213,69 @@ module Marker
|
|
104
213
|
end
|
105
214
|
|
106
215
|
rule bulleted
|
107
|
-
'*' ws list_item <Bulleted>
|
216
|
+
'*' ws item:list_item <Bulleted>
|
108
217
|
/
|
109
|
-
'*' ws phrase <Bulleted>
|
218
|
+
'*' ws content:phrase <Bulleted>
|
110
219
|
end
|
111
220
|
|
112
221
|
rule numbered
|
113
|
-
'#' ws list_item <Numbered>
|
222
|
+
'#' ws item:list_item <Numbered>
|
114
223
|
/
|
115
|
-
'#' ws phrase <Numbered>
|
224
|
+
'#' ws content:phrase <Numbered>
|
116
225
|
end
|
117
226
|
|
118
227
|
rule indented
|
119
|
-
':' ws list_item <Indented>
|
228
|
+
':' ws item:list_item <Indented>
|
229
|
+
/
|
230
|
+
':' ws content:phrase <Indented>
|
231
|
+
end
|
232
|
+
|
233
|
+
# a series of list items, so they can be coalesced
|
234
|
+
rule arg_safe_list
|
235
|
+
h:arg_safe_list_item ws rnl r:arg_safe_list <List>
|
120
236
|
/
|
121
|
-
|
237
|
+
h:arg_safe_list_item ws <List>
|
122
238
|
end
|
123
239
|
|
240
|
+
# for matching any list item
|
241
|
+
rule arg_safe_list_item
|
242
|
+
arg_safe_bulleted
|
243
|
+
/
|
244
|
+
arg_safe_numbered
|
245
|
+
/
|
246
|
+
arg_safe_indented
|
247
|
+
/
|
248
|
+
arg_safe_definition
|
249
|
+
end
|
250
|
+
|
251
|
+
rule arg_safe_bulleted
|
252
|
+
'*' ws item:arg_safe_list_item <Bulleted>
|
253
|
+
/
|
254
|
+
'*' ws content:arg_safe_phrase <Bulleted>
|
255
|
+
end
|
256
|
+
|
257
|
+
rule arg_safe_numbered
|
258
|
+
'#' ws item:arg_safe_list_item <Numbered>
|
259
|
+
/
|
260
|
+
'#' ws content:arg_safe_phrase <Numbered>
|
261
|
+
end
|
262
|
+
|
263
|
+
rule arg_safe_indented
|
264
|
+
':' ws item:arg_safe_list_item <Indented>
|
265
|
+
/
|
266
|
+
':' ws content:arg_safe_phrase <Indented>
|
267
|
+
end
|
268
|
+
|
269
|
+
##### definition lists
|
270
|
+
|
124
271
|
rule definition
|
125
|
-
';' ws term:
|
272
|
+
';' ws term:definition_term_phrase ws term_delimiter ws definition:phrase <Definition>
|
126
273
|
/
|
127
274
|
';' ws term:phrase <Definition>
|
128
275
|
end
|
129
276
|
|
130
|
-
rule
|
131
|
-
h:definition_term_word ws r:
|
277
|
+
rule definition_term_phrase
|
278
|
+
h:definition_term_word ws r:definition_term_phrase <Phrase>
|
132
279
|
/
|
133
280
|
h:definition_term_word "" <Phrase>
|
134
281
|
end
|
@@ -167,6 +314,50 @@ module Marker
|
|
167
314
|
arg_delimiter
|
168
315
|
end
|
169
316
|
|
317
|
+
rule arg_safe_definition
|
318
|
+
';' ws term:arg_safe_definition_term_phrase ws term_delimiter ws definition:arg_safe_phrase <Definition>
|
319
|
+
/
|
320
|
+
';' ws term:arg_safe_phrase <Definition>
|
321
|
+
end
|
322
|
+
|
323
|
+
rule arg_safe_definition_term_phrase
|
324
|
+
h:arg_safe_definition_term_word ws r:arg_safe_definition_term_phrase <Phrase>
|
325
|
+
/
|
326
|
+
h:arg_safe_definition_term_word "" <Phrase>
|
327
|
+
end
|
328
|
+
|
329
|
+
rule arg_safe_definition_term_word
|
330
|
+
arg_safe_bold
|
331
|
+
/
|
332
|
+
arg_safe_italic
|
333
|
+
/
|
334
|
+
arg_safe_link
|
335
|
+
/
|
336
|
+
template
|
337
|
+
/
|
338
|
+
url
|
339
|
+
/
|
340
|
+
plain_word
|
341
|
+
/
|
342
|
+
bold_toggle
|
343
|
+
/
|
344
|
+
italic_toggle
|
345
|
+
/
|
346
|
+
heading_toggle
|
347
|
+
/
|
348
|
+
internal_link_start
|
349
|
+
/
|
350
|
+
internal_link_end
|
351
|
+
/
|
352
|
+
external_link_start
|
353
|
+
/
|
354
|
+
external_link_end
|
355
|
+
/
|
356
|
+
template_start
|
357
|
+
end
|
358
|
+
|
359
|
+
##### verbatim areas
|
360
|
+
|
170
361
|
rule verbatim_area
|
171
362
|
h:verbatim rnl r:verbatim_area <VerbatimArea>
|
172
363
|
/
|
@@ -175,29 +366,80 @@ module Marker
|
|
175
366
|
|
176
367
|
rule verbatim
|
177
368
|
" " v:(!"\n" .)* <Verbatim>
|
369
|
+
/
|
370
|
+
"\t" v:(!"\n" .)* <Verbatim>
|
371
|
+
end
|
372
|
+
|
373
|
+
rule arg_safe_verbatim_area
|
374
|
+
h:arg_safe_verbatim rnl r:arg_safe_verbatim_area <VerbatimArea>
|
375
|
+
/
|
376
|
+
h:arg_safe_verbatim "" <VerbatimArea>
|
178
377
|
end
|
179
378
|
|
379
|
+
rule arg_safe_verbatim
|
380
|
+
" " v:( !("\n" / arg_delimiter / template_end) .)* <Verbatim>
|
381
|
+
/
|
382
|
+
"\t" v:(!"\n" .)* <Verbatim>
|
383
|
+
end
|
384
|
+
|
385
|
+
##### horizontal rules
|
386
|
+
|
387
|
+
# horizontal rules deviate from how MediaWiki does things.
|
388
|
+
# consider this:
|
389
|
+
# ---- a para-
|
390
|
+
# graph
|
391
|
+
# MediaWiki renders (newlines removed): <hr /><p>a para-</p><p>graph</p>
|
392
|
+
# this renders (newlines removed): <hr /><p>a para-graph</p>
|
393
|
+
|
180
394
|
rule horiz_rule
|
181
|
-
|
182
|
-
|
183
|
-
|
184
|
-
|
185
|
-
|
186
|
-
|
395
|
+
"----" "-"* ws t:known_text_paragraph <HorizRule>
|
396
|
+
/
|
397
|
+
"----" "-"* <HorizRule>
|
398
|
+
end
|
399
|
+
|
400
|
+
rule arg_safe_horiz_rule
|
401
|
+
"----" "-"* ws t:arg_safe_known_text_paragraph <HorizRule>
|
187
402
|
/
|
188
403
|
"----" "-"* <HorizRule>
|
189
404
|
end
|
190
405
|
|
191
406
|
##### combination rules
|
192
407
|
|
408
|
+
rule paragraph
|
409
|
+
text "" <Paragraph>
|
410
|
+
end
|
411
|
+
|
412
|
+
rule arg_safe_paragraph
|
413
|
+
text:arg_safe_text "" <Paragraph>
|
414
|
+
end
|
415
|
+
|
193
416
|
# a block of normal text, including single newlines but not a special line
|
194
417
|
rule text
|
195
|
-
!
|
418
|
+
!(heading / list / verbatim_area / horiz_rule) h:phrase ws rnl r:text <TextBlock>
|
419
|
+
/
|
420
|
+
!(heading / list / verbatim_area / horiz_rule) h:phrase ws <TextBlock>
|
421
|
+
end
|
422
|
+
|
423
|
+
# arg-safe variant
|
424
|
+
rule arg_safe_text
|
425
|
+
!(arg_safe_heading / arg_safe_list / arg_safe_verbatim_area / arg_safe_horiz_rule) h:arg_safe_phrase ws rnl r:arg_safe_text <TextBlock>
|
196
426
|
/
|
197
|
-
!
|
427
|
+
!(arg_safe_heading / arg_safe_list / arg_safe_verbatim_area / arg_safe_horiz_rule) h:arg_safe_phrase ws <TextBlock>
|
428
|
+
end
|
429
|
+
|
430
|
+
# paragraph wrapper for known text
|
431
|
+
rule known_text_paragraph
|
432
|
+
text:known_text "" <Paragraph>
|
433
|
+
end
|
434
|
+
|
435
|
+
# paragraph wrapper for arg-safe known text
|
436
|
+
rule arg_safe_known_text_paragraph
|
437
|
+
text:arg_safe_known_text "" <Paragraph>
|
198
438
|
end
|
199
439
|
|
200
|
-
# a block of text that does not require a !line check
|
440
|
+
# "known" text a block of text that does not require a !line check
|
441
|
+
#
|
442
|
+
# in other words, it is known that this block is text and not line-anchored
|
201
443
|
#
|
202
444
|
# for situations where text may follow structures that are normally on a
|
203
445
|
# single line---e.g., horizontal rules--- the text should be handled like a
|
@@ -205,12 +447,20 @@ module Marker
|
|
205
447
|
# the text starts like a special line: cases like this: "---- ----".
|
206
448
|
# Because it follows a structure without a new line, it is safe to assume
|
207
449
|
# the text is a phrase.
|
208
|
-
rule
|
209
|
-
h:phrase ws rnl r:text <
|
450
|
+
rule known_text
|
451
|
+
h:phrase ws rnl r:text <TextBlock>
|
210
452
|
/
|
211
|
-
h:phrase ws <
|
453
|
+
h:phrase ws <TextBlock>
|
212
454
|
end
|
213
455
|
|
456
|
+
rule arg_safe_known_text
|
457
|
+
h:arg_safe_phrase ws rnl r:arg_safe_text <TextBlock>
|
458
|
+
/
|
459
|
+
h:arg_safe_phrase ws <TextBlock>
|
460
|
+
end
|
461
|
+
|
462
|
+
##### general words and phrases
|
463
|
+
|
214
464
|
# a phrase of words without newlines
|
215
465
|
rule phrase
|
216
466
|
h:word ws r:phrase <Phrase>
|
@@ -218,6 +468,7 @@ module Marker
|
|
218
468
|
h:word "" <Phrase>
|
219
469
|
end
|
220
470
|
|
471
|
+
# words not inside any container
|
221
472
|
rule word
|
222
473
|
bold
|
223
474
|
/
|
@@ -231,10 +482,70 @@ module Marker
|
|
231
482
|
/
|
232
483
|
plain_word
|
233
484
|
/
|
234
|
-
|
485
|
+
bold_toggle
|
486
|
+
/
|
487
|
+
italic_toggle
|
488
|
+
/
|
489
|
+
heading_toggle
|
490
|
+
/
|
491
|
+
internal_link_start
|
492
|
+
/
|
493
|
+
internal_link_end
|
494
|
+
/
|
495
|
+
external_link_start
|
496
|
+
/
|
497
|
+
external_link_end
|
498
|
+
/
|
499
|
+
template_start
|
500
|
+
/
|
501
|
+
template_end
|
502
|
+
/
|
503
|
+
arg_delimiter
|
504
|
+
/
|
505
|
+
term_delimiter
|
235
506
|
end
|
236
507
|
|
237
|
-
|
508
|
+
# a phrase of arg-safe words without newlines
|
509
|
+
rule arg_safe_phrase
|
510
|
+
h:arg_safe_word ws r:arg_safe_phrase <Phrase>
|
511
|
+
/
|
512
|
+
h:arg_safe_word "" <Phrase>
|
513
|
+
end
|
514
|
+
|
515
|
+
# words inside templates but not other markup
|
516
|
+
rule arg_safe_word
|
517
|
+
arg_safe_bold
|
518
|
+
/
|
519
|
+
arg_safe_italic
|
520
|
+
/
|
521
|
+
arg_safe_link
|
522
|
+
/
|
523
|
+
template
|
524
|
+
/
|
525
|
+
url
|
526
|
+
/
|
527
|
+
plain_word
|
528
|
+
/
|
529
|
+
bold_toggle
|
530
|
+
/
|
531
|
+
italic_toggle
|
532
|
+
/
|
533
|
+
heading_toggle
|
534
|
+
/
|
535
|
+
internal_link_start
|
536
|
+
/
|
537
|
+
internal_link_end
|
538
|
+
/
|
539
|
+
external_link_start
|
540
|
+
/
|
541
|
+
external_link_end
|
542
|
+
/
|
543
|
+
template_start
|
544
|
+
/
|
545
|
+
term_delimiter
|
546
|
+
end
|
547
|
+
|
548
|
+
##### bold
|
238
549
|
|
239
550
|
rule bold
|
240
551
|
bold_toggle ws bold_enclosed_text ws bold_toggle <Bold>
|
@@ -277,6 +588,45 @@ module Marker
|
|
277
588
|
term_delimiter
|
278
589
|
end
|
279
590
|
|
591
|
+
rule arg_safe_bold
|
592
|
+
bold_toggle ws bold_enclosed_text:arg_safe_bold_enclosed_text ws bold_toggle <Bold>
|
593
|
+
end
|
594
|
+
|
595
|
+
rule arg_safe_bold_enclosed_text
|
596
|
+
h:arg_safe_bold_enclosed_word ws r:arg_safe_bold_enclosed_text <Phrase>
|
597
|
+
/
|
598
|
+
h:arg_safe_bold_enclosed_word "" <Phrase>
|
599
|
+
end
|
600
|
+
|
601
|
+
# things that can be inside arg-safe bold areas
|
602
|
+
rule arg_safe_bold_enclosed_word
|
603
|
+
arg_safe_italic
|
604
|
+
/
|
605
|
+
arg_safe_link
|
606
|
+
/
|
607
|
+
template
|
608
|
+
/
|
609
|
+
url
|
610
|
+
/
|
611
|
+
plain_word
|
612
|
+
/
|
613
|
+
heading_toggle
|
614
|
+
/
|
615
|
+
internal_link_start
|
616
|
+
/
|
617
|
+
internal_link_end
|
618
|
+
/
|
619
|
+
external_link_start
|
620
|
+
/
|
621
|
+
external_link_end
|
622
|
+
/
|
623
|
+
template_start
|
624
|
+
/
|
625
|
+
term_delimiter
|
626
|
+
end
|
627
|
+
|
628
|
+
##### italics
|
629
|
+
|
280
630
|
rule italic
|
281
631
|
italic_toggle ws italic_enclosed_text ws italic_toggle_no_lookahead <Italic>
|
282
632
|
end
|
@@ -318,23 +668,71 @@ module Marker
|
|
318
668
|
term_delimiter
|
319
669
|
end
|
320
670
|
|
671
|
+
rule arg_safe_italic
|
672
|
+
italic_toggle ws italic_enclosed_text:arg_safe_italic_enclosed_text ws italic_toggle_no_lookahead <Italic>
|
673
|
+
end
|
674
|
+
|
675
|
+
rule arg_safe_italic_enclosed_text
|
676
|
+
h:arg_safe_italic_enclosed_word ws r:arg_safe_italic_enclosed_text <Phrase>
|
677
|
+
/
|
678
|
+
h:arg_safe_italic_enclosed_word "" <Phrase>
|
679
|
+
end
|
680
|
+
|
681
|
+
# things that can be inside arg-safe italic areas
|
682
|
+
rule arg_safe_italic_enclosed_word
|
683
|
+
# this is equivalent to:
|
684
|
+
# !(template_end / arg_delimiter) italic_enclosed_word
|
685
|
+
arg_safe_bold
|
686
|
+
/
|
687
|
+
arg_safe_link
|
688
|
+
/
|
689
|
+
template
|
690
|
+
/
|
691
|
+
url
|
692
|
+
/
|
693
|
+
plain_word
|
694
|
+
/
|
695
|
+
heading_toggle
|
696
|
+
/
|
697
|
+
internal_link_start
|
698
|
+
/
|
699
|
+
internal_link_end
|
700
|
+
/
|
701
|
+
external_link_start
|
702
|
+
/
|
703
|
+
external_link_end
|
704
|
+
/
|
705
|
+
template_start
|
706
|
+
/
|
707
|
+
term_delimiter
|
708
|
+
end
|
709
|
+
|
710
|
+
##### links
|
711
|
+
|
321
712
|
rule link
|
322
713
|
internal_link
|
323
714
|
/
|
324
715
|
external_link
|
325
716
|
end
|
326
717
|
|
327
|
-
|
718
|
+
rule arg_safe_link
|
719
|
+
arg_safe_internal_link
|
720
|
+
/
|
721
|
+
arg_safe_external_link
|
722
|
+
end
|
723
|
+
|
724
|
+
##### internal links
|
725
|
+
|
328
726
|
# [[ link target | link label ]]
|
329
727
|
# * can contain white space
|
330
728
|
# * cannot contain new lines
|
331
|
-
|
729
|
+
|
332
730
|
rule internal_link
|
333
|
-
internal_link_start ws t:
|
731
|
+
internal_link_start ws t:plain_phrase ws a:arg_delimiter ws l:internal_link_enclosed_text ws internal_link_end <InternalLink>
|
334
732
|
/
|
335
|
-
internal_link_start ws t:
|
733
|
+
internal_link_start ws t:plain_phrase ws a:arg_delimiter ws internal_link_end <InternalLink>
|
336
734
|
/
|
337
|
-
internal_link_start ws t:
|
735
|
+
internal_link_start ws t:plain_phrase ws internal_link_end <InternalLink>
|
338
736
|
end
|
339
737
|
|
340
738
|
rule internal_link_enclosed_text
|
@@ -373,7 +771,50 @@ module Marker
|
|
373
771
|
term_delimiter
|
374
772
|
end
|
375
773
|
|
376
|
-
|
774
|
+
rule arg_safe_internal_link
|
775
|
+
internal_link_start ws t:plain_phrase ws a:arg_delimiter ws l:arg_safe_internal_link_enclosed_text ws internal_link_end <InternalLink>
|
776
|
+
/
|
777
|
+
internal_link_start ws t:plain_phrase ws a:arg_delimiter ws internal_link_end <InternalLink>
|
778
|
+
/
|
779
|
+
internal_link_start ws t:plain_phrase ws internal_link_end <InternalLink>
|
780
|
+
end
|
781
|
+
|
782
|
+
rule arg_safe_internal_link_enclosed_text
|
783
|
+
h:arg_safe_internal_link_enclosed_word ws r:arg_safe_internal_link_enclosed_text <Phrase>
|
784
|
+
/
|
785
|
+
h:arg_safe_internal_link_enclosed_word "" <Phrase>
|
786
|
+
end
|
787
|
+
|
788
|
+
rule arg_safe_internal_link_enclosed_word
|
789
|
+
# this is equivalent to:
|
790
|
+
# !(template_end / arg_delimiter) internal_link_enclosed_word
|
791
|
+
arg_safe_bold
|
792
|
+
/
|
793
|
+
arg_safe_italic
|
794
|
+
/
|
795
|
+
template
|
796
|
+
/
|
797
|
+
plain_word
|
798
|
+
/
|
799
|
+
bold_toggle
|
800
|
+
/
|
801
|
+
italic_toggle
|
802
|
+
/
|
803
|
+
heading_toggle
|
804
|
+
/
|
805
|
+
internal_link_start
|
806
|
+
/
|
807
|
+
external_link_start
|
808
|
+
/
|
809
|
+
external_link_end
|
810
|
+
/
|
811
|
+
template_start
|
812
|
+
/
|
813
|
+
term_delimiter
|
814
|
+
end
|
815
|
+
|
816
|
+
##### external links
|
817
|
+
|
377
818
|
# [ url label ]
|
378
819
|
# * can contain white space
|
379
820
|
# * cannot contain new lines
|
@@ -385,6 +826,7 @@ module Marker
|
|
385
826
|
#
|
386
827
|
# also use this rule to match bad internal link syntax [[ url | label ]]
|
387
828
|
# because the rendering rules are the same
|
829
|
+
|
388
830
|
rule external_link
|
389
831
|
# external_link_start ws t:url rws l:external_link_enclosed_text ws external_link_end <ExternalLink>
|
390
832
|
# /
|
@@ -439,39 +881,34 @@ module Marker
|
|
439
881
|
term_delimiter
|
440
882
|
end
|
441
883
|
|
442
|
-
rule
|
443
|
-
|
884
|
+
rule arg_safe_external_link
|
885
|
+
# external_link_start ws t:url rws l:external_link_enclosed_text ws external_link_end <ExternalLink>
|
886
|
+
# /
|
887
|
+
# external_link_start ws t:url ws external_link_end <ExternalLink>
|
888
|
+
# /
|
889
|
+
external_link_start ws t:implicit_url rws l:arg_safe_external_link_enclosed_text ws external_link_end <ExternalLink>
|
444
890
|
/
|
445
|
-
|
446
|
-
end
|
447
|
-
|
448
|
-
rule arg_list
|
449
|
-
h:arg aws arg_delimiter aws r:arg_list <Arguments>
|
891
|
+
external_link_start ws t:implicit_url ws external_link_end <ExternalLink>
|
450
892
|
/
|
451
|
-
|
452
|
-
end
|
453
|
-
|
454
|
-
# a plain (positional) argument or a named (name=text) argument
|
455
|
-
rule arg
|
456
|
-
name:plain_text aws heading_toggle aws val:arg_list_enclosed_text <Argument>
|
893
|
+
internal_link_start ws t:implicit_url ws arg_delimiter ws l:arg_safe_internal_link_enclosed_text ws internal_link_end <ExternalLink>
|
457
894
|
/
|
458
|
-
|
895
|
+
internal_link_start ws t:implicit_url rws l:arg_safe_internal_link_enclosed_text ws internal_link_end <ExternalLink>
|
459
896
|
/
|
460
|
-
|
897
|
+
internal_link_start ws t:implicit_url ws internal_link_end <ExternalLink>
|
461
898
|
end
|
462
899
|
|
463
|
-
rule
|
464
|
-
h:
|
900
|
+
rule arg_safe_external_link_enclosed_text
|
901
|
+
h:arg_safe_external_link_enclosed_word ws r:arg_safe_external_link_enclosed_text <Phrase>
|
465
902
|
/
|
466
|
-
h:
|
903
|
+
h:arg_safe_external_link_enclosed_word "" <Phrase>
|
467
904
|
end
|
468
905
|
|
469
|
-
rule
|
470
|
-
|
906
|
+
rule arg_safe_external_link_enclosed_word
|
907
|
+
# this is equivalent to:
|
908
|
+
# !(template_end / arg_delimiter) external_link_enclosed_word
|
909
|
+
arg_safe_bold
|
471
910
|
/
|
472
|
-
|
473
|
-
/
|
474
|
-
link
|
911
|
+
arg_safe_italic
|
475
912
|
/
|
476
913
|
template
|
477
914
|
/
|
@@ -479,7 +916,7 @@ module Marker
|
|
479
916
|
/
|
480
917
|
plain_word
|
481
918
|
/
|
482
|
-
bold_toggle
|
919
|
+
bold_toggle
|
483
920
|
/
|
484
921
|
italic_toggle
|
485
922
|
/
|
@@ -491,25 +928,25 @@ module Marker
|
|
491
928
|
/
|
492
929
|
external_link_start
|
493
930
|
/
|
494
|
-
external_link_end
|
495
|
-
/
|
496
931
|
template_start
|
497
932
|
/
|
498
933
|
term_delimiter
|
499
934
|
end
|
500
935
|
|
936
|
+
##### plain words and phrases
|
937
|
+
|
501
938
|
# a phrase of plain words
|
502
|
-
rule
|
503
|
-
h:plain_word ws r:
|
939
|
+
rule plain_phrase
|
940
|
+
h:plain_word ws r:plain_phrase <Phrase>
|
504
941
|
/
|
505
942
|
h:plain_word "" <Phrase>
|
506
943
|
end
|
507
944
|
|
508
945
|
# avoids using a white-list so that utf-8 characters are accepted
|
509
946
|
#
|
510
|
-
# matches anything
|
947
|
+
# matches anything other than a delimiter or whitespace
|
511
948
|
rule plain_word
|
512
|
-
(!delimiter
|
949
|
+
(!delimiter [^ \t\r\n])+ <Word>
|
513
950
|
end
|
514
951
|
|
515
952
|
##### delimiters
|