markascend 0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: da8df76e1fec2a44dfd1dcaa2b8b27d7f12a86fe
4
+ data.tar.gz: bfd019319d0b249bb1bfca43a3aea1b8dd9943f9
5
+ SHA512:
6
+ metadata.gz: d53e8a4039c10b9f74eebd2be669a48b9b328859b53f2397e7b356bfece745ec5f15183959a50856713fa0cc15aad04a4569e596d94de3500b0d49b3b300e156
7
+ data.tar.gz: f6199c6c3052ad1fd5db6d7001c193b0be5550e6b8c0592a6dee94e48d07a142557b05bbb2a6749eb0b7ece9fb07fa898a1bafbb656faa143920efe657caffc6
data/Gemfile ADDED
@@ -0,0 +1,9 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gem 'rake'
4
+ gem 'slim', '~> 2.0.0.pre.6'
5
+ gem 'nokogiri'
6
+ gem 'pry'
7
+ gem 'pygments.rb'
8
+ gem 'compass'
9
+ gem 'filemagic'
data/copying ADDED
@@ -0,0 +1,18 @@
1
+ Copyright (C) 2013 by Zete Lui (BSD)
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy of
4
+ this software and associated documentation files (the "Software"), to deal in
5
+ the Software without restriction, including without limitation the rights to
6
+ use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
7
+ the Software, and to permit persons to whom the Software is furnished to do so,
8
+ subject to the following conditions:
9
+
10
+ The above copyright notice and this permission notice shall be included in all
11
+ copies or substantial portions of the Software.
12
+
13
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
15
+ FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
16
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
17
+ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
18
+ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
@@ -0,0 +1,65 @@
1
+ \hi(ruby)
2
+ h2 Compile
3
+
4
+ |
5
+ Markascend.compile src, options
6
+
7
+ h2 Options
8
+
9
+ - `:autolink`, default value is `%w[http https ftp mailto]`
10
+ - `:inline_img`, compile image into inlined base64, default = `false`
11
+ - `:macros`, specify the names of enabled macros. Other macros will be treated as plain text. The default value is `Markascend::DEFAULT_MACROS`.
12
+ - `:line_units`, specify the inline parsers to be used (be careful with the order!). The default value is `Markascend::DEFAULT_LINE_UNITS`.
13
+ - `:sandbox`, a hybrid option to tweak the syntax to be generally safe for user inputs. `false` by default. When set to `true`, footnotes are disabled, header anchors are ignored, and enabled macros are set to `Markascend::SANDBOX_MACROS`. The sandbox macro list can be overriden by the `:macros` option.
14
+ - `:toc`, whether generate table of contents. `false` by default. Header anchors can be customized or generated in the form of `"-#{N}"`. Note that there's no "permalink" generator for headers, but you can implement one with simple javascript.
15
+
16
+ h2 Customizing macros
17
+
18
+ More macro processors can be added by
19
+
20
+ |
21
+ class Markascend::Macro
22
+ def parse_fancy_macro
23
+ ... compose result string with: env, content, inline
24
+ end
25
+ end
26
+
27
+ Macro names are limited to names like ruby methods.
28
+
29
+ |
30
+ Markascend.compile src, macros: %w[fancy_macro del]
31
+
32
+ h2 Customizing line-unit parsers
33
+
34
+ More line-unit parsers can be added by
35
+
36
+ |
37
+ class Markascend::LineUnit
38
+ def parse_at
39
+ ... compose result string with: env, line, block, linenum
40
+ end
41
+ end
42
+
43
+ The list of inline parsers can be changed, or reordered
44
+
45
+ |
46
+ Markascend.compile src, line_units: Markascend::DEFAULT_LINE_UNITS + %w[at]
47
+
48
+ h2 Notes on `\slim`
49
+
50
+ You need to install slim \hi(bash)(`gem ins slim`) and require it before using the \hi(ma)`\slim` macro:
51
+ \hi(ruby)
52
+ |
53
+ require 'slim'
54
+
55
+ It is disabled in sandbox mode.
56
+
57
+ h2 Notes on `\dot`
58
+
59
+ You need to install [graphviz](graphviz.org) before using the \hi(ma)`\dot` macro.
60
+
61
+ It is disabled in sandbox mode.
62
+
63
+ h2 Notes on output format
64
+
65
+ The output is valid HTML5, but not XHTML.
@@ -0,0 +1,23 @@
1
+ h2 Introduction
2
+
3
+ Markascend is an extensible markdown-like, macro-powered html document syntax and processor.
4
+ Can be used as blog-generator, post-formatter or for literate programing.
5
+
6
+ h2 Install
7
+
8
+ Requires Ruby 2.0.0+
9
+
10
+ |sh
11
+ gem ins markascend
12
+
13
+ h2 Cheat Sheet
14
+
15
+ h2 Document
16
+
17
+ - [Syntax in Details](syntax)
18
+ - [API in Details](api)
19
+ - [Editor Support (Textmate/Sublime)](https://github.com/luikore/Markascend.tmbundle)
20
+
21
+ h2 License
22
+
23
+ [BSD](https://github.com/luikore/markascend/tree/master/copying)
@@ -0,0 +1,264 @@
1
+ \options
2
+ title: Syntax
3
+
4
+ h2 Design principles
5
+
6
+ - A simpler *Markdown* should be better.
7
+ - Since a markup language usually need to process several different syntaces in one article, language composition becomes important in syntax design.
8
+ - TeX macro syntax is nearly perfect for extending the language, but not very visual intuitive when there are too much backslashes and bracers.
9
+ - Layout-based grammar (indentation sensitive) shows a cleaner way for language composition and cleaning up macros.
10
+
11
+ \hi(ma)
12
+
13
+ h2 Inline elements
14
+ *italic*
15
+ |
16
+ *italic*
17
+
18
+ **bold**
19
+ |
20
+ **bold**
21
+
22
+ Italic and bold elements can interpolate
23
+ |
24
+ ***bold and italic*bold**
25
+
26
+ `code` suject to the same rule as in markdown, no escape chars
27
+ |
28
+ `code`
29
+ ``to include "`", use more backticks as delimiter``
30
+ `` ` space before and after to disambig backtick number ``
31
+
32
+ $math$, inside which `\\` and `\$` are treated as atomic elements, no escape chars
33
+ |
34
+ $math$
35
+ $\\\$$ <- the content is parsed to latex processer as '\\\$'
36
+ $\\$$ <- NOTE this is a math elem with a pending dollar
37
+
38
+ Link:
39
+ |
40
+ url in lexical parens: [a](example.com/use/\\/for/backslash/and/\)/for/right/paren)
41
+ url in recursive braces: [a]{example.com/allow/{nested{braces}}/and/(parens)/in/url}
42
+ no need to escape backslashes in braces: [a]{file:\\\C:\windows}
43
+ link to anchor: [a](#anchor)
44
+
45
+ Footnotes:
46
+ |
47
+ [.](this is shown in footnote and replaced by a number)
48
+ [.*](this is shown in footnote and replaced by the acronym: *)
49
+
50
+ Using already defined footnote ("one more dot to use"):
51
+ |
52
+ [:1]
53
+ [:link by acronym]
54
+
55
+ Except math and code, inline elements can be nested.
56
+
57
+ By default, text starting with `http://`, `https://` or `mailto://` will be auto-linked, you can change supported protocols or disable all with compile options.
58
+
59
+ NOTE: Not very different from markdown here, just less, math and footnotes added. Images are too complex for a simple syntax, it is made into a macro.
60
+
61
+ h2 Headers
62
+ |
63
+ h1 title
64
+ h2#anchor-name title
65
+
66
+ NOTE: Anchors are disabled in sandbox mode.
67
+ h2 Lists
68
+
69
+ `-` is the bullet for unordered list
70
+ |
71
+ - an entry
72
+ - another entry
73
+
74
+ `+` is the bullet for ordered list
75
+ |
76
+ + entry 1
77
+ + entry 2
78
+
79
+ h2 Quote
80
+
81
+ Just one `>` is enough for a quote containing many lines (NOTE: use the similar rule as code block and allow quoter name?)
82
+ |
83
+ > You
84
+ can
85
+ input
86
+ new line
87
+ on twitter
88
+ now
89
+
90
+ h2 Escape characters
91
+ |
92
+ \h3
93
+ \\
94
+ \$
95
+ \*
96
+ \[
97
+ \%
98
+ \@
99
+ \#
100
+ ...
101
+
102
+ All HTML4 entities are supported
103
+ |
104
+ &amp;
105
+ &#2382;
106
+ &#x003F;
107
+
108
+ h2 Built-in macros
109
+
110
+ The following macros explains themselves:
111
+ |
112
+ \del{deleted}
113
+ \underline{underline}
114
+ \sub{subscript}
115
+ \sup{supscript}
116
+ \img{hello/a.png alt="an image" href="a.b.c"}
117
+
118
+ Unlike in *Mardown*, all html tags in *Markascend* are escaped. If you write
119
+ |ma
120
+ <div></div>
121
+
122
+ You get HTML like this
123
+ |html
124
+ &lt;div&gt;&lt;/div&gt;
125
+
126
+ To put HTML in, you can use the `\html` macro
127
+ |
128
+ \html{<hr>}
129
+
130
+ *Slim* is a template engine which makes html much cleaner:
131
+ |
132
+ \slim{a href="/" Home}
133
+
134
+ Note that with `\slim`, you can embed a bunch of other template engines, or write ruby logic, or do all kinds of evil things:
135
+ |
136
+ \slim
137
+ - 3.times do
138
+ markdown:
139
+ # h1 hello world
140
+ sass:
141
+ h1
142
+ color: red
143
+ coffee:
144
+ alert "hello world"
145
+
146
+ To build a table from a CSV
147
+ |
148
+ \csv
149
+ name,type
150
+ "Ahri",Mage
151
+ "Miss Fortune",DPS
152
+
153
+ A table from CSV, without header
154
+ |
155
+ \headless_csv
156
+ "Ahri",Mage
157
+ "Miss Fortune",DPS
158
+
159
+ For block-styled math
160
+ |
161
+ \latex
162
+ \begin{align} \\
163
+ & \rm{MathJax}\ \LaTeX \ \rm{Example} \\ \\
164
+ \end{align}
165
+
166
+ `\options` is a special macro. The content *YAML* is not processed by *Markascend*, but retrievable with API. This is designed for blogger generators: you can put metadata inside the document.
167
+ |
168
+ \options
169
+ tags: [markascend readme]
170
+
171
+ To change default code syntax highliter (See [below](#syn-hi-macros) for details]:
172
+ |
173
+ \hi(rb)
174
+ \hi(none)
175
+
176
+ h2 Notes on macros
177
+
178
+ Macro name is case sensitive, starts with a word-character but not digit, and contains no symbols or punctuations. The rule expressed in (Onigmo) regexp is `(?!\d)\w+`. For example, `你2` is valid name, but `2你` is invalid.
179
+
180
+ Macros can be of inline form (`\macro{content}`) or block form (`\macro` and the indented block in following lines is the content of the macro).
181
+
182
+ Inline macros can use some different delimiters for convinience, allowed ones are: `\macro(content)` and `\macro{content}`.
183
+
184
+ In the form `\macro(content)`, the content is parsed by lexical rule. The parser handles and only handles 2 escape chars: `\\` and `\)`, very much like single quoted string in Ruby.
185
+ Example: `\macro(\ and \\ are both single-backslash. nest: (\))`
186
+
187
+ In the form `\macro{content}`, you don't escape chars in the content, and can put in recursively nested braces. When you need to put an unmatched `{` or `}` in the content, use the previous form.
188
+ Example: `\macro{\\ is two backslashes. nest: {}}`
189
+
190
+ Design NOTE: There are actual very few symbols left for us... for example `[]`, `$$`, `|` may confuse with link or math or code elements. Symbols like `"`, `'`, `()`, `<>` are not good either, because they are common when composing an article.
191
+
192
+ User NOTE:
193
+ - The parsing rules for `(content)` and `{content}` are the same as in links.
194
+ - Only 1 block-style macro name is allowed inside a line.
195
+ - The basic parsing unit is a line and a block, every parsing starts with this unit --- so don't hand-add space indentations to paragraphs, let css do it.
196
+
197
+ h2 Charting macros
198
+
199
+ You need to install [graphviz](http://graphviz.org/) first.
200
+
201
+ |
202
+ \dot
203
+ digraph G{
204
+ main -> parse;
205
+ }
206
+
207
+ h2 Popular-company macros
208
+
209
+ This is the transient part but provides some convienience. The list will change if some of them dies before *Markascend*.
210
+
211
+ To generate a twitter or weibo link
212
+ |
213
+ \twitter(@night_song)
214
+ \twitter(#HotS)
215
+ \weibo(@yavaeye)
216
+
217
+ To generate a wiki link (though wiki is an organization, not a company...)
218
+ |
219
+ \wiki{ruby(programing_language)}
220
+
221
+ Embed gist
222
+ |
223
+ \gist(12321)
224
+
225
+ Embed video (currently should recognize youtube, vimeo and niconico links, the size is a problem)
226
+ |
227
+ \video(500x400 http://www.youtube.com/watch?v=TGPvtlnwH6E)
228
+ \video(500x400 http://www.youtube.com/watch?v=TGPvtlnwH6E)
229
+ \video(500x400 http://www.youtube.com/watch?v=TGPvtlnwH6E)
230
+
231
+ h2#syn-hi-macros Syntax hilite macros and code blocks
232
+
233
+ Inline code can also have syntax hiliter
234
+
235
+ |
236
+ \hi(rb)`str.size` in ruby is equivalent to \hi(objc)`str.length` in objc
237
+ |
238
+ this part is also hilited in objc
239
+ \hi(none)`\hi(none)` removes hilite state in document
240
+ |
241
+ this part also has no syntax hilite
242
+
243
+ To specify syntax in block form (this won't change default syntax):
244
+ |
245
+ |rb
246
+ puts 'hello world'
247
+
248
+ h2 Combining macro blocks, lists and quotes
249
+
250
+ Lists and quotes are parsed as recursive block elements.
251
+ If the inside contains macros, need to indent more for the content of the macro:
252
+ |
253
+ > Though the \music macro..
254
+ 1232. 1232. 1232.
255
+ is not defined.
256
+
257
+ Another nesting example:
258
+ |
259
+ - an entry
260
+ contains an ordered list
261
+ + entry \latex
262
+ \mathbb{one}
263
+ + entry 2
264
+ - another entry
@@ -0,0 +1,172 @@
1
+ require "strscan"
2
+ require "cgi"
3
+ require "csv"
4
+ require "yaml"
5
+ require "base64"
6
+ require "open3"
7
+ require "pygments"
8
+ require "open-uri"
9
+ require "filemagic"
10
+
11
+ module Markascend
12
+ VERSION = '0.1'
13
+
14
+ DEFAULT_MACROS = Hash.[] %w[
15
+ del underline sub sup
16
+ img html slim
17
+ csv headless_csv
18
+ latex
19
+ options hi
20
+ dot
21
+
22
+ twitter weibo
23
+ wiki
24
+ gist
25
+ video
26
+ ].map{|k| [k, "parse_#{k}"]}
27
+
28
+ SANDBOX_MACROS = DEFAULT_MACROS.dup.delete_if do |k, v|
29
+ %w[html slim options dot].include? k
30
+ end
31
+
32
+ # NOTE on the order:
33
+ # - link/bold/italic can contain char
34
+ # but link need not interpolate with bold or italic, seems too rare cased
35
+ # - bold/italic can interpolate with each other
36
+ # - escapes are processed in the char parser, link/bold/italic can use escape chars
37
+ DEFAULT_LINE_UNITS = %w[
38
+ inline_code
39
+ math
40
+ auto_link
41
+ macro
42
+ link
43
+ bold_italic
44
+ char
45
+ ].map{|k| "parse_#{k}"}
46
+
47
+ class << Markascend
48
+ def compile src, opts={}
49
+ src = src.gsub "\t", ' '
50
+ env = Env.new opts
51
+ res = Parser.new(env, src).parse
52
+
53
+ if env.toc and !env.toc.empty?
54
+ res = (generate_toc(env.toc) << res)
55
+ end
56
+
57
+ if env.footnotes and !env.footnotes.empty?
58
+ res << generate_footnotes(env.footnotes)
59
+ end
60
+
61
+ res
62
+ end
63
+
64
+ attr_accessor :inline_parsers, :macros
65
+
66
+ # escape html
67
+ def escape_html s
68
+ CGI.escape_html s
69
+ end
70
+
71
+ # escape string so that the result can be placed inside double-quoted value of a tag property
72
+ def escape_attr s
73
+ # http://www.w3.org/TR/html5/syntax.html#attributes-0
74
+ s ? (s.gsub /"/, '&quot;') : ''
75
+ end
76
+
77
+ # escape string so that the result can be placed inside a `<pre>` tag
78
+ def escape_pre s
79
+ s.gsub(/(<)|(>)|&/){$1 ? '&lt;' : $2 ? '&gt;' : '&amp;'}
80
+ end
81
+
82
+ # syntax hilite s with lang
83
+ def hilite s, lang, inline=false
84
+ if !lang or lang =~ /\A(ma(rkascend)?)?\z/i or !(::Pygments::Lexer.find lang)
85
+ # TODO ma lexer
86
+ s = inline ? (escape_html s) : (escape_pre s)
87
+ else
88
+ s = Pygments.highlight s, lexer: lang, options: {nowrap: true}
89
+ end
90
+
91
+ # TODO config class
92
+ if inline
93
+ %Q|<code class="highlight">#{s}</code>|
94
+ else
95
+ %Q|<pre><code class="highlight">#{s}</code></pre>|
96
+ end
97
+ end
98
+
99
+ # detect mime type of the buffer
100
+ def mime buffer
101
+ fm = FileMagic.new FileMagic::MAGIC_MIME_TYPE
102
+ res = fm.buffer buffer
103
+ fm.close
104
+ res
105
+ end
106
+
107
+ # strip tags from s
108
+ def strip_tags s
109
+ # deal with html tags only
110
+ s.gsub(/
111
+ \<\s*script\b
112
+ (?:
113
+ (["']).*?\1|[^\>] # properties
114
+ )*
115
+ \>
116
+ .*?
117
+ \<\s*\/\s*script\s*\>
118
+ /x, '').gsub(/
119
+ \<\s*(?:\/\s*)?
120
+ \w+\b # tag name, no need to care xml namespace
121
+ (?:
122
+ (["']).*?\1|[^\>] # properties
123
+ )*
124
+ \>
125
+ /x, '')
126
+ end
127
+
128
+ private
129
+
130
+ def generate_toc toc
131
+ res = '<div id="toc"><ol>'
132
+ levels = toc.values.map(&:first).uniq.sort
133
+ prev_level = 0
134
+ toc.each do |id, (x, header_content)| # x as in "hx"
135
+ level = levels.index(x)
136
+ if level >= prev_level
137
+ (level - prev_level).times do
138
+ res << "<ol>"
139
+ end
140
+ elsif
141
+ (prev_level - level).times do
142
+ res << "</ol>"
143
+ end
144
+ end
145
+ prev_level = level
146
+ title = strip_tags header_content
147
+ res << %Q{<li><a href="\##{id}">#{title}</a></li>}
148
+ end
149
+ prev_level.times do
150
+ res << "</ol>"
151
+ end
152
+ res << "</ol></div>"
153
+ end
154
+
155
+ def generate_footnotes footnotes
156
+ res = '<div id="footnotes"><ol>'
157
+ i = 0
158
+ footnotes.each do |abbrev, detail|
159
+ res << %Q|<li id="footnote-#{i}">#{escape_html detail}</li>|
160
+ i += 1
161
+ end
162
+ res << "</ol></div>"
163
+ end
164
+ end
165
+ end
166
+
167
+ require_relative "markascend/env"
168
+ require_relative "markascend/parser"
169
+ require_relative "markascend/line_unit"
170
+ require_relative "markascend/macro"
171
+ require_relative "markascend/builtin_macros"
172
+ require_relative "markascend/popular_company_macros"