rouge-carve 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: fbeb1c2fa0f572f138c6a89a23bafcae682051c9e10474951b5cba9de3bc0aa3
4
+ data.tar.gz: 7d089e544a959be5c208ef4e1cc523feda2b1d8cf6632b99086ae41c5cb13c31
5
+ SHA512:
6
+ metadata.gz: 439cc10aa425adaec2fed2e19e6af9aab75ffec47b699c3a3dff0632ee2503ca6240d7d4f869db0c93f907fee29f43b351ba9a507f89cff5b0a3a11ddb257e72
7
+ data.tar.gz: b93c04c6c832959f1a58e238efb161b5d6a7283b104cec5deae971e62065f18b8bf88abf03fc61fdf7cc29da54bac7d3cddaa384f267c7e6e91478cd124f8a59
data/CHANGELOG.md ADDED
@@ -0,0 +1,18 @@
1
+ # Changelog
2
+
3
+ ## [0.1.0] - 2026-08-27
4
+
5
+ First release.
6
+
7
+ - Every Carve construct: headings, containers, fenced and raw blocks, tables
8
+ with alignment and per-row attributes, definition lists, footnotes,
9
+ citations, cross-references, attribute blocks, task items, critic markup,
10
+ symbol shortcodes, mentions, tags and typographic runs.
11
+ - Fenced code is delegated to a lexer for its declared language; a `=FORMAT`
12
+ raw block stays opaque, because a format is not a language.
13
+ - Fence widths are closed over per block, so a shorter inner fence does not
14
+ end a wider one.
15
+ - Input preservation is gated: concatenating the token values reproduces the
16
+ source exactly, checked over every document in the Carve corpus.
17
+ - Construct parity with the sibling grammars is gated against the same
18
+ inventory the highlight.js definition answers to.
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Mark Scherer and contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,98 @@
1
+ # rouge-carve
2
+
3
+ A [Rouge](https://github.com/rouge-ruby/rouge) lexer for the
4
+ [Carve](https://markup-carve.github.io/carve/) markup language.
5
+
6
+ ```bash
7
+ gem install rouge-carve
8
+ ```
9
+
10
+ ```ruby
11
+ require 'rouge-carve'
12
+
13
+ Rouge::Lexer.find('carve') # => Rouge::Lexers::Carve
14
+ Rouge::Lexer.guess(filename: 'notes.crv') # => Rouge::Lexers::Carve
15
+ ```
16
+
17
+ Requiring the gem registers the lexer. Anything that already highlights with
18
+ Rouge - GitLab, Jekyll, Redcarpet, most Ruby static site generators - then
19
+ handles `.crv` and `.carve` files, and ` ```carve ` fenced blocks, without
20
+ further configuration.
21
+
22
+ ## Why not just use the Markdown lexer
23
+
24
+ Because it is not close enough to be wrong quietly. Carve deliberately changed
25
+ the delimiters Markdown got backwards, and three of them **invert**:
26
+
27
+ | Carve | means | Markdown reads it as |
28
+ | --- | --- | --- |
29
+ | `*bold*` | strong | emphasis |
30
+ | `/italic/` | emphasis | literal slashes |
31
+ | `_under_` | underline | emphasis |
32
+ | `~strike~` | strikethrough | subscript, or literal |
33
+ | `{=mark=}` | highlight | literal braces |
34
+ | `{^sup^}` | superscript | literal braces |
35
+
36
+ Highlighting a Carve document as Markdown does not degrade to plain text. It
37
+ shows bold where the author wrote emphasis.
38
+
39
+ ## What it covers
40
+
41
+ Every construct in the language: headings, containers (`::: note`), fenced and
42
+ raw blocks, tables with alignment and per-row attributes, definition lists,
43
+ footnotes, citations, cross-references, attribute blocks, task items, critic
44
+ markup, symbol shortcodes, mentions, tags and the typographic runs.
45
+
46
+ **Fenced code is delegated.** A ` ```ruby ` block inside a `.crv` file is lexed
47
+ as Ruby, because that is what a reader expects. A ` ```=html ` raw block is not:
48
+ its info string names an output format, not a language, so the payload stays
49
+ opaque.
50
+
51
+ **Fence widths are respected.** A three-backtick line inside a four-backtick
52
+ fence is content, not a closer. The width is closed over per block rather than
53
+ approximated, which the sibling grammars in other ecosystems cannot do.
54
+
55
+ ## What it cannot say
56
+
57
+ Rouge has no underline token - the vocabulary stops at `Generic::Emph`,
58
+ `Generic::Strong` and `Generic::EmphStrong`. Carve's `_x_` is underline, so its
59
+ content shares `Generic::Emph` with italic. The delimiters stay `Punctuation`,
60
+ so a consumer can still tell the two apart by the delimiter each carries.
61
+
62
+ Block openers are matched at any indent. Carve opens a block at column 0 or at
63
+ an enclosing container's content column and nowhere in between, which needs a
64
+ container model this lexer does not carry. The trade-off is deliberate and
65
+ shared with the Prism, highlight.js and Pygments grammars, so the four agree:
66
+ it over-colours an indented-at-document-level opener rather than
67
+ under-colouring the far more common indented construct inside a list item.
68
+
69
+ ## Testing
70
+
71
+ ```bash
72
+ bundle install
73
+ bundle exec rspec
74
+ ```
75
+
76
+ Two gates, and the second is the one that finds things:
77
+
78
+ - `spec/carve_spec.rb` pins the constructs, including each inverted delimiter.
79
+ - `script/corpus_check.rb` lexes every `.crv` document in the Carve spec
80
+ repository and fails on a single `Error` token. A lexer emitting `Error` is
81
+ telling the reader its own rules ran out. Both bugs found during development -
82
+ a blank line inside a comment fence, and a `---` thematic break opening
83
+ frontmatter that never closed - came from this and not from an example
84
+ anybody thought to write.
85
+
86
+ ## Related
87
+
88
+ - [carve](https://github.com/markup-carve/carve) - the language and its spec
89
+ - [pygments-carve](https://github.com/markup-carve/pygments-carve) - the same
90
+ grammar for Pygments, which this was ported from
91
+ - [carve-grammars](https://github.com/markup-carve/carve-grammars) - TextMate,
92
+ Prism and highlight.js
93
+ - [carve-css](https://github.com/markup-carve/carve-css) - styles for the HTML
94
+ Carve renders
95
+
96
+ ## License
97
+
98
+ MIT.
@@ -0,0 +1,35 @@
1
+ ---
2
+ title: Carve
3
+ ---
4
+
5
+ {#intro}
6
+ # Styling a document
7
+
8
+ Carve swaps the delimiters Markdown got wrong: *bold* is one asterisk,
9
+ /italic/ is a slash, and _underline_ finally has a spelling of its own.
10
+ Sub and sup are braced-only, so H{,2,}O and E = mc{^2^} are unambiguous.
11
+
12
+ {.lead}
13
+ An attribute line binds an id, classes and `data-*` to the block below it.
14
+
15
+ - A bullet, with `code` and a [link](https://markup-carve.github.io/carve/)
16
+ - A task item is marked inline
17
+ - [x] and it carries its own state
18
+
19
+ ::: note "Containers are core syntax"
20
+ A word with no registered handler renders as a generic div, which is what
21
+ makes a whole family of constructs possible without touching the parser.
22
+ :::
23
+
24
+ ``` ruby "example.rb"
25
+ def greet(name) = "Hello, #{name}!"
26
+ ```
27
+
28
+ |= Engine |= Status |
29
+ | carve-js | shipped |
30
+ | carve-rs | shipped |
31
+
32
+ ^ Table 1: the reference implementations.
33
+
34
+ See </#intro> for the opening, cite [@knuth1984] for the argument, and
35
+ %% this trailing comment never reaches the output.
@@ -0,0 +1,586 @@
1
+ # -*- coding: utf-8 -*- #
2
+ # frozen_string_literal: true
3
+
4
+ module Rouge
5
+ module Lexers
6
+ # Carve is a post-Markdown lightweight markup language. Its inline
7
+ # delimiters deliberately differ from Markdown's, which is why lexing a
8
+ # Carve document as Markdown produces actively WRONG output rather than
9
+ # merely plain text:
10
+ #
11
+ # Carve means Markdown would read it as
12
+ # --------- ------------- -------------------------
13
+ # *bold* strong emphasis
14
+ # /italic/ emphasis literal slashes
15
+ # _under_ underline emphasis
16
+ # ~strike~ strikethrough subscript / literal
17
+ # {=mark=} highlight literal braces
18
+ # {^sup^} superscript literal braces
19
+ #
20
+ # WHY THE BLOCK OPENERS ARE NOT ANCHORED AT COLUMN 0. Carve opens a block at
21
+ # column 0, or at an enclosing container's content column - nowhere in
22
+ # between. So ` # H` at document level is a paragraph, while the same
23
+ # opener at a list item's content column is a real heading. Telling those
24
+ # apart needs a container model that tracks the item's content column. A
25
+ # regex lexer's state stack could carry one, but the sibling grammars
26
+ # (Prism, highlight.js, Pygments) do not, and this lexer keeps their
27
+ # trade-off on purpose so they agree: block openers match at any indent and
28
+ # knowingly over-colour the rare indented-at-document-level case, rather
29
+ # than under-colouring the common valid shape of an indented construct
30
+ # inside a list item.
31
+ #
32
+ # WHY AN ATTRIBUTE BLOCK IS ONE TOKEN. `{#id .cls key="v" :lang}` is emitted
33
+ # whole as Name::Attribute rather than split into id, class, key, value and
34
+ # language parts. Splitting reads better in isolation, but an attribute
35
+ # block can carry a brace inside a quoted value, a language tag, and a bare
36
+ # key that is not an attribute at all, and the sibling grammars treat the
37
+ # block as one unit; a consumer asking "is this text inside an attribute
38
+ # block" must get the same answer here as it does there.
39
+ #
40
+ # WHERE ROUGE CANNOT SAY WHAT CARVE MEANS. Carve's `_x_` is UNDERLINE, not
41
+ # emphasis, and Rouge has no underline token - the vocabulary stops at
42
+ # Generic::Emph, Generic::Strong and Generic::EmphStrong. The content takes
43
+ # Generic::Emph so it is at least marked up, and the delimiters stay
44
+ # Punctuation so a consumer can still tell underline from italic by the
45
+ # delimiter it carries. Pygments, which has Generic.Underline, does make
46
+ # the distinction.
47
+ #
48
+ # Spec: https://markup-carve.github.io/carve/
49
+ class Carve < RegexLexer
50
+ title 'Carve'
51
+ desc 'Carve, a post-Markdown lightweight markup language'
52
+
53
+ tag 'carve'
54
+ aliases 'crv'
55
+ filenames '*.crv', '*.carve'
56
+ mimetypes 'text/x-carve'
57
+
58
+ # A leading margin. A byte order mark at the start of a document is not
59
+ # content, so a block opener behind one is still a block opener.
60
+ MARGIN = /[ \t]*/.freeze
61
+
62
+ # One attribute block, brace to brace. Quoted values may contain a brace
63
+ # and an escaped quote, so the value alternatives come before the
64
+ # bare-character one; a single nested brace pair is allowed for a braced
65
+ # span written inside.
66
+ ATTRS = %r/
67
+ \{(?=[.:}'"]|\#[\w-]+[\s}]|[A-Za-z][\w-]*(?:[=\s}]|$))(?:
68
+ "(?:[^"\\\n]|\\.)*"
69
+ |'(?:[^'\\\n]|\\.)*'
70
+ |\{[^{}\n]*\}
71
+ |[^{}\n]
72
+ )*\}
73
+ /x.freeze
74
+
75
+ # The same block as a STANDALONE ATTRIBUTE LINE, which may span lines.
76
+ # The inline form deliberately cannot, because an unclosed inline `{`
77
+ # would otherwise swallow the rest of the document.
78
+ ATTRS_LINE = %r/
79
+ \{(?=[.:}'"]|\#[\w-]+[\s}]|[A-Za-z][\w-]*(?:[=\s}]|$))(?:
80
+ "(?:[^"\\]|\\.)*"
81
+ |'(?:[^'\\]|\\.)*'
82
+ |\{[^{}]*\}
83
+ |[^{}]
84
+ )*\}
85
+ /x.freeze
86
+
87
+ # A bracketed label that may itself contain three levels of brackets. A
88
+ # regex cannot match arbitrarily balanced brackets, and a link label in
89
+ # practice nests a level or two (`[t[z]](/u)`). Bounding the nesting keeps
90
+ # the common shapes matching instead of stopping at the first inner
91
+ # bracket, which is what a naive \[[^\]]*\] does.
92
+ LABEL = begin
93
+ inner = '[^\[\]\\\\\n]|\\\\.'
94
+ pattern = "(?:#{inner})*"
95
+ 3.times { pattern = "(?:#{inner}|\\[#{pattern}\\])*" }
96
+ Regexp.new("\\[#{pattern}\\]").freeze
97
+ end
98
+
99
+ # Characters that can begin an inline construct. A run of anything else is
100
+ # ordinary content and is emitted as ONE token - without this every
101
+ # content character becomes its own token, which is both noisy and
102
+ # unusable to a consumer asking whether a phrase carries a scope.
103
+ INLINE_STARTERS = '\\\\%!`${\\[\\^<:@\\#*\\/_~=.\\-'
104
+
105
+ # A run of content characters, excluding +extra+ as well.
106
+ def self.plain_run(extra = '')
107
+ Regexp.new("[^\n#{INLINE_STARTERS}#{extra}]+")
108
+ end
109
+
110
+ PLAIN = plain_run.freeze
111
+ PLAIN_NO_PIPE = plain_run('|').freeze
112
+ PLAIN_NO_BRACKET = plain_run('\\]').freeze
113
+
114
+ state :root do
115
+ mixin :block
116
+ end
117
+
118
+ # ------------------------------------------------------------------
119
+ # Block level
120
+ # ------------------------------------------------------------------
121
+ state :block do
122
+ # Front matter, only at the very start of the document. \A is what
123
+ # keeps a `---yaml` line mid-document from opening one.
124
+ rule %r/\A(?)(---)([a-zA-Z][\w-]*)?([ \t]*\n)(?=[\s\S]*?^---[ \t]*$)/ do
125
+ groups Text, Punctuation, Keyword::Type, Text
126
+ push :frontmatter
127
+ end
128
+
129
+ # A comment fence (%%% or longer) versus a one-line comment. The fence
130
+ # has to be tried first: %%% also matches the one-line form.
131
+ rule %r/^(#{MARGIN})(%%%+)([^\n]*)(\n)/ do
132
+ groups Text, Comment::Preproc, Comment, Text
133
+ push :commentfence
134
+ end
135
+ rule %r/^(#{MARGIN})(%%)([^\n]*)$/ do
136
+ groups Text, Comment::Preproc, Comment
137
+ end
138
+
139
+ # A raw block: the `=FORMAT` info string routes the payload to that
140
+ # output format verbatim. Emitted as one token including the `=`,
141
+ # because the format word without its sigil is not the construct. The
142
+ # payload is never a language, so it is never delegated.
143
+ rule %r/^(#{MARGIN})(`{3,}|~{3,})([ \t]*)(=[a-zA-Z][\w+.-]*)([^\n]*)/ do |m|
144
+ groups Text, Punctuation, Text, Keyword::Type
145
+ token Text, m[5]
146
+ push_fence(m[2], nil)
147
+ end
148
+
149
+ # A code fence. The first word of the info string names a language, and
150
+ # a lexer for it highlights the payload - the one place this lexer
151
+ # looks past Carve, because a reader of a `.crv` file expects an
152
+ # embedded PHP sample to look like PHP.
153
+ rule %r/^(#{MARGIN})(`{3,}|~{3,})([ \t]*)([a-zA-Z][\w+#.-]*)?([^\n]*)/ do |m|
154
+ groups Text, Punctuation, Text, Name::Builtin
155
+ lex_info_string(m[5])
156
+ push_fence(m[2], m[4])
157
+ end
158
+
159
+ # Container divs. A reserved kind word (note, tip, figure, ...) names a
160
+ # known container; `:::` followed by `|` is the layout form.
161
+ rule %r/^(#{MARGIN})(:{3,})([ \t]*)(\|)/ do
162
+ groups Text, Punctuation, Text, Operator
163
+ end
164
+ rule %r/^(#{MARGIN})(:{3,})([ \t]*)([a-zA-Z][\w-]*)?([^\n]*)/ do |m|
165
+ groups Text, Punctuation, Text, Keyword::Namespace
166
+ lex_info_string(m[5])
167
+ end
168
+
169
+ # A caption line attaches to the block above or below it.
170
+ rule %r/^(#{MARGIN})(\^)([ \t]+)/ do
171
+ groups Text, Punctuation, Text
172
+ push :caption
173
+ end
174
+
175
+ # Headings. Carve has no setext form, so a `#` run is the only spelling
176
+ # and a trailing `{...}` is NOT an attribute block here.
177
+ rule %r/^(#{MARGIN})(\#{1,6})([ \t]+)/ do
178
+ groups Text, Punctuation, Text
179
+ push :heading
180
+ end
181
+
182
+ # Thematic breaks, before the list rules so `---` is not a bullet.
183
+ rule %r/^#{MARGIN}(?:\*[ \t]*){3,}$/, Punctuation
184
+ rule %r/^#{MARGIN}(?:-[ \t]*){3,}$/, Punctuation
185
+ rule %r/^#{MARGIN}(?:_[ \t]*){3,}$/, Punctuation
186
+
187
+ # Definition markers: footnote, link reference, abbreviation. The
188
+ # separator after the colon must START WITH A LITERAL SPACE - a
189
+ # tab-first separator makes the line an ordinary paragraph.
190
+ rule %r/^(#{MARGIN})(\[\^)([^\]\n]+)(\]:)( )/ do
191
+ groups Text, Punctuation, Name::Label, Punctuation, Text
192
+ push :inline
193
+ end
194
+ rule %r/^(#{MARGIN})(\*\[)([^\]\n]+)(\]:)( )/ do
195
+ groups Text, Punctuation, Name::Entity, Punctuation, Text
196
+ push :inline
197
+ end
198
+ rule %r/^(#{MARGIN})(\[)([^\]\n]+)(\]:)( )/ do
199
+ groups Text, Punctuation, Name::Label, Punctuation, Text
200
+ push :linkdest
201
+ end
202
+
203
+ # A definition-list term (`::`) and its definition (`:`).
204
+ rule %r/^(#{MARGIN})(::)([ \t]+)/ do
205
+ groups Text, Punctuation, Text
206
+ push :heading
207
+ end
208
+ rule %r/^(#{MARGIN})(:)(?=[ \t])/ do
209
+ groups Text, Punctuation
210
+ end
211
+
212
+ # A blockquote marker must be followed by a space or end the line;
213
+ # `>foo` is a paragraph.
214
+ rule %r/^(#{MARGIN})(>+)(?=[ \t]|$)/ do
215
+ groups Text, Punctuation
216
+ push :quoteline
217
+ end
218
+
219
+ # Task items before plain bullets, so the state marker is its own
220
+ # token. The state is any single character, not only a space or an x:
221
+ # `[>]` is deferred and `[-]` is dropped.
222
+ rule %r/^(#{MARGIN})((?:[-*+]|\d+[.)]|[A-Za-z]+[.)])(?:#{ATTRS})?)([ \t]+)(\[[^\]\n]\])/ do
223
+ groups Text, Punctuation, Text, Name::Constant
224
+ end
225
+
226
+ # Bullets. A run of markers on one line opens nested lists at once
227
+ # (`- - A`), and attributes may be glued straight onto the marker.
228
+ rule %r/^(#{MARGIN})((?:[-*+][ \t]+)*[-*+](?:#{ATTRS})?)(?=[ \t]|$)/ do
229
+ groups Text, Punctuation
230
+ end
231
+
232
+ # Ordered markers: numeric, alphabetic, roman, and the bare `.` that
233
+ # continues the enclosing sequence.
234
+ rule %r/^(#{MARGIN})((?:\d+|[A-Za-z]+)[.)](?:#{ATTRS})?)(?=[ \t]|$)/ do
235
+ groups Text, Num::Integer
236
+ end
237
+ rule %r/^(#{MARGIN})(\.(?:#{ATTRS})?)(?=[ \t]|$)/ do
238
+ groups Text, Num::Integer
239
+ end
240
+
241
+ # Tables. The header marker, the alignment run and the separator row
242
+ # are their own tokens; cell content is lexed inline.
243
+ rule %r/^(#{MARGIN})(\|=[<>^v~]*)/ do
244
+ groups Text, Operator
245
+ push :tablerow
246
+ end
247
+ rule %r/^(#{MARGIN})(\|)/ do
248
+ groups Text, Punctuation
249
+ push :tablerow
250
+ end
251
+
252
+ # A standalone attribute block, which may span lines.
253
+ rule %r/^(#{MARGIN})(#{ATTRS_LINE})/ do
254
+ groups Text, Name::Attribute
255
+ end
256
+
257
+ mixin :inline
258
+ end
259
+
260
+ state :frontmatter do
261
+ rule %r/^(---)([ \t]*)$/ do
262
+ groups Punctuation, Text
263
+ pop!
264
+ end
265
+ rule %r/\n/, Comment::Special
266
+ rule %r/[^\n]+\n?/, Comment::Special
267
+ end
268
+
269
+ state :commentfence do
270
+ rule %r/^([ \t]*)(%%%+)([ \t]*)$/ do
271
+ groups Text, Comment::Preproc, Text
272
+ pop!
273
+ end
274
+ rule %r/\n/, Comment
275
+ rule %r/[^\n]+\n?/, Comment
276
+ end
277
+
278
+ state :heading do
279
+ rule %r/$/, Text, :pop!
280
+ mixin :inlinecontent
281
+ rule PLAIN, Generic::Heading
282
+ rule %r/[^\n]/, Generic::Heading
283
+ end
284
+
285
+ state :quoteline do
286
+ rule %r/$/, Text, :pop!
287
+ mixin :inlinecontent
288
+ rule PLAIN, Generic::Emph
289
+ rule %r/[^\n]/, Generic::Emph
290
+ end
291
+
292
+ state :caption do
293
+ rule %r/$/, Text, :pop!
294
+ mixin :inlinecontent
295
+ rule PLAIN, Generic::Subheading
296
+ rule %r/[^\n]/, Generic::Subheading
297
+ end
298
+
299
+ state :tablerow do
300
+ rule %r/$/, Text, :pop!
301
+ # A header cell marker, optionally carrying an alignment run. The pipe
302
+ # is part of it, so `|=` reads as one construct.
303
+ rule %r/\|=[<>^v~]*/, Operator
304
+ # A separator row cell, in the native and the GFM spelling.
305
+ rule %r/(?<=\|)[ \t]*:?-{2,}:?[ \t]*(?=\|)/, Punctuation
306
+ rule %r/(?<=\|)[ \t]*[<>^v~]{1,2}[ \t]*(?=\|)/, Operator
307
+ rule %r/\|/, Punctuation
308
+ mixin :inlinecontent
309
+ rule PLAIN_NO_PIPE, Text
310
+ rule %r/[^\n|]/, Text
311
+ end
312
+
313
+ state :linkdest do
314
+ rule %r/$/, Text, :pop!
315
+ rule %r/<[^>\n]*>/, Name::Tag
316
+ rule %r/"[^"\n]*"/, Str::Double
317
+ rule %r/[^\s\n]+/, Name::Tag
318
+ # Any Unicode whitespace, not only space and tab: a destination can be
319
+ # preceded by U+202F and friends, and a class of two characters leaves
320
+ # the state with nothing to match.
321
+ rule %r/[^\S\n]+/, Text
322
+ end
323
+
324
+ # ------------------------------------------------------------------
325
+ # Inline level
326
+ # ------------------------------------------------------------------
327
+ state :inline do
328
+ mixin :inlinecontent
329
+ rule %r/\n/, Text
330
+ rule %r/./, Text
331
+ end
332
+
333
+ state :inlinecontent do
334
+ # An escape wins over every delimiter that follows it. A backslash at
335
+ # END OF LINE escapes nothing - it is a hard break, and needs its own
336
+ # rule because the general form requires a following character.
337
+ rule %r/\\(?=\n|$)/, Str::Escape
338
+ rule %r/\\[!-\/:-@\[-`{-~]/, Str::Escape
339
+
340
+ # A trailing comment runs to end of line from anywhere on it.
341
+ rule %r/(%%)([^\n]*)$/ do
342
+ groups Comment::Preproc, Comment
343
+ end
344
+
345
+ # Verbatim families first: nothing inside them is markup. The literal
346
+ # form is a `!` PREFIX on a code span, not a trailing attribute, and it
347
+ # has to be tried before plain inline code or the `!` would be read as
348
+ # text and the span as ordinary code.
349
+ rule %r/(!)(`+)([^\n]*?)(\2)/ do
350
+ groups Operator, Punctuation, Literal, Punctuation
351
+ end
352
+ rule %r/(\$\$)(`+)([^\n]*?)(\2)/ do
353
+ groups Operator, Punctuation, Str::Other, Punctuation
354
+ end
355
+ rule %r/(\$)(`+)([^\n]*?)(\2)/ do
356
+ groups Operator, Punctuation, Str::Other, Punctuation
357
+ end
358
+ rule %r/(`+)([^\n]*?)(\1)/ do
359
+ groups Punctuation, Str::Backtick, Punctuation
360
+ end
361
+
362
+ # CriticMarkup substitution and comment, before the forced family:
363
+ # `{~old~>new~}` also matches the forced-strike shape.
364
+ rule %r/(\{~)([^\n]*?)(~>)([^\n]*?)(~\})/ do
365
+ groups Punctuation, Generic::Deleted, Operator, Generic::Inserted, Punctuation
366
+ end
367
+ rule %r/(\{\#)([^\n]*?)(\#\})/ do
368
+ groups Punctuation, Comment, Punctuation
369
+ end
370
+
371
+ # The braced FORCED emphasis family. Braces make a delimiter apply
372
+ # where the bare form would not - `my{_path_}name` is underline inside
373
+ # a word - so the content is emphasis and the braces are its
374
+ # delimiters, not an attribute block.
375
+ rule %r/(\{\*)([^\n]+?)(\*\})/ do
376
+ groups Punctuation, Generic::Strong, Punctuation
377
+ end
378
+ rule %r/(\{\/)([^\n]+?)(\/\})/ do
379
+ groups Punctuation, Generic::Emph, Punctuation
380
+ end
381
+ rule %r/(\{_)([^\n]+?)(_\})/ do
382
+ groups Punctuation, Generic::Emph, Punctuation
383
+ end
384
+ rule %r/(\{~)([^\n]+?)(~\})/ do
385
+ groups Punctuation, Generic::Deleted, Punctuation
386
+ end
387
+
388
+ # Braced inline families. Sup and sub are braced-only in Carve: a bare
389
+ # `^x^` or `,x,` is literal text.
390
+ rule %r/(\{\^)([^\n]+?)(\^\})/ do
391
+ groups Punctuation, Generic::Emph, Punctuation
392
+ end
393
+ rule %r/(\{,)([^\n]+?)(,\})/ do
394
+ groups Punctuation, Generic::Emph, Punctuation
395
+ end
396
+ rule %r/(\{=)([^\n]+?)(=\})/ do
397
+ groups Punctuation, Generic::Inserted, Punctuation
398
+ end
399
+ rule %r/(\{\+)([^\n]+?)(\+\})/ do
400
+ groups Punctuation, Generic::Inserted, Punctuation
401
+ end
402
+ rule %r/(\{-)([^\n]+?)(-\})/ do
403
+ groups Punctuation, Generic::Deleted, Punctuation
404
+ end
405
+ rule %r/(\{>>)([^\n]+?)(<<\})/ do
406
+ groups Punctuation, Comment, Punctuation
407
+ end
408
+ rule %r/(\{%)(.*?)(%\})/ do
409
+ groups Comment::Preproc, Comment, Comment::Preproc
410
+ end
411
+
412
+ # An inline footnote carries content; a reference carries a label.
413
+ rule %r/\^\[/, Punctuation, :inlinefootnote
414
+ rule %r/(\[\^)([^\]\n]+)(\])/ do
415
+ groups Punctuation, Name::Label, Punctuation
416
+ end
417
+
418
+ # A citation before a link: both open with `[`, and `[@key]` or
419
+ # `[+@key]` would otherwise be read as a link label.
420
+ rule %r/(\[)(\+?@[^\]\n]+)(\])/ do
421
+ groups Punctuation, Name::Variable, Punctuation
422
+ end
423
+
424
+ # Image, then link, then a bare span. All three share the `[...]` shape
425
+ # and differ only in the prefix and what follows.
426
+ rule %r/(!#{LABEL})(\()([^)\n]*)(\))/ do
427
+ groups Str::Other, Punctuation, Name::Tag, Punctuation
428
+ end
429
+ rule %r/(!#{LABEL})(#{LABEL})/ do
430
+ groups Str::Other, Name::Label
431
+ end
432
+ rule %r/(#{LABEL})(\()([^)\n]*)(\))/ do
433
+ groups Name::Entity, Punctuation, Name::Tag, Punctuation
434
+ end
435
+ rule %r/(#{LABEL})(#{LABEL})/ do
436
+ groups Name::Entity, Name::Label
437
+ end
438
+ rule %r/#{LABEL}(?=#{ATTRS})/, Name::Entity
439
+
440
+ # A cross-reference to a heading id.
441
+ rule %r/(<\/)(\#[\w-]+)(>)/ do
442
+ groups Punctuation, Name::Namespace, Punctuation
443
+ end
444
+ # An autolink.
445
+ rule %r/(<)([a-zA-Z][\w+.-]*:[^>\s]+|[^>\s@]+@[^>\s]+)(>)/ do
446
+ groups Punctuation, Name::Tag, Punctuation
447
+ end
448
+
449
+ # The `:name[...]` extension form.
450
+ rule %r/(:)([a-zA-Z][\w-]*)(\[)/ do
451
+ groups Punctuation, Name::Function, Punctuation
452
+ push :rolebody
453
+ end
454
+
455
+ # A code callout marker, and a symbol shortcode - whose name may start
456
+ # with a sign, as in `:+1:`.
457
+ rule %r/<\d+>/, Name::Constant
458
+ rule %r/(?<![\w:]):[\w+-]+:(?![\w:])/, Name::Constant
459
+
460
+ # An attribute block attached to the construct before it.
461
+ rule ATTRS, Name::Attribute
462
+
463
+ # Bare emphasis delimiters. Carve's bare set is / * _ ~ = and each
464
+ # needs a non-space inner boundary so `a / b` stays literal.
465
+ rule %r/(\/\*)([^\n]+?)(\*\/)/ do
466
+ groups Punctuation, Generic::Strong, Punctuation
467
+ end
468
+ rule %r/(\*\/)([^\n]+?)(\/\*)/ do
469
+ groups Punctuation, Generic::Strong, Punctuation
470
+ end
471
+ rule %r/(\*)(\S(?:[^\n]*?\S)?)(\*)/ do
472
+ groups Punctuation, Generic::Strong, Punctuation
473
+ end
474
+ rule %r/(\/)(\S(?:[^\n]*?\S)?)(\/)/ do
475
+ groups Punctuation, Generic::Emph, Punctuation
476
+ end
477
+ rule %r/(_)(\S(?:[^\n]*?\S)?)(_)/ do
478
+ groups Punctuation, Generic::Emph, Punctuation
479
+ end
480
+ rule %r/(~)(\S(?:[^\n]*?\S)?)(~)/ do
481
+ groups Punctuation, Generic::Deleted, Punctuation
482
+ end
483
+ rule %r/(=)(\S(?:[^\n]*?\S)?)(=)/ do
484
+ groups Punctuation, Generic::Inserted, Punctuation
485
+ end
486
+
487
+ # A mention and a tag, each one token: the sigil is part of the name
488
+ # rather than punctuation beside it.
489
+ rule %r/(?<![\w\/])@[\w][\w.-]*/, Name::Variable::Magic
490
+ rule %r/(?<![\w&])\#[\w][\w-]*/, Name::Variable::Instance
491
+
492
+ # Typographic runs, longest first: an arrow is not an en dash plus a
493
+ # stray angle bracket, and `---` is not `--` plus `-`.
494
+ rule %r/<-->|<==>|<=>|-->|<--|==>|<==|->|<-/, Operator
495
+ rule %r/(?<!-)---(?!-)|(?<!-)--(?!-)/, Punctuation
496
+ rule %r/\.\.\./, Punctuation
497
+ end
498
+
499
+ state :inlinefootnote do
500
+ rule %r/\]/, Punctuation, :pop!
501
+ mixin :inlinecontent
502
+ rule PLAIN_NO_BRACKET, Generic::Emph
503
+ rule %r/[^\]\n]/, Generic::Emph
504
+ rule %r/\n/, Text
505
+ end
506
+
507
+ state :rolebody do
508
+ rule %r/\]/, Punctuation, :pop!
509
+ mixin :inlinecontent
510
+ rule PLAIN_NO_BRACKET, Name::Function
511
+ rule %r/[^\]\n]/, Name::Function
512
+ rule %r/\n/, Text
513
+ end
514
+
515
+ private
516
+
517
+ # Lex the tail of a fence or container opener line without leaving the
518
+ # current state: a quoted title, a bracketed label and an attribute
519
+ # block, each as its own token.
520
+ def lex_info_string(text)
521
+ return if text.nil? || text.empty?
522
+
523
+ scanner = StringScanner.new(text)
524
+ until scanner.eos?
525
+ if (m = scanner.scan(%r/"[^"\n]*"/))
526
+ token Str::Double, m
527
+ elsif (m = scanner.scan(%r/\[[^\]\n]*\]/))
528
+ token Name::Label, m
529
+ elsif (m = scanner.scan(ATTRS))
530
+ token Name::Attribute, m
531
+ else
532
+ token Text, scanner.getch
533
+ end
534
+ end
535
+ end
536
+
537
+ # Enter a fenced block. The payload is opaque to Carve, so it either goes
538
+ # to a lexer for the declared language or comes out as one string token.
539
+ #
540
+ # The closer must be AT LEAST as long as the opener. A Rouge state cannot
541
+ # carry the opener's width across rules, so the width is closed over here
542
+ # instead of approximated - `~~~~` does not close a ```` ``` ```` block,
543
+ # and a three-backtick line inside a four-backtick fence stays content.
544
+ def push_fence(opener, language)
545
+ char = Regexp.escape(opener[0])
546
+ width = opener.length
547
+ closer = %r/^[ \t]*#{char}{#{width},}[ \t]*$/
548
+
549
+ sublexer = fence_lexer(language)
550
+ sublexer&.reset!
551
+
552
+ push do
553
+ rule closer, Punctuation, :pop!
554
+ if sublexer
555
+ # The opener's own newline, which is not part of the payload and
556
+ # sits mid-line where the line rule below cannot reach it.
557
+ rule %r/\n/, Text
558
+ # WHOLE LINES, never fragments. A delegated lexer carries state
559
+ # across calls, so handing it a piece of a line at a time makes it
560
+ # tokenize the pieces rather than the line.
561
+ rule %r/^.*\n?/ do |m|
562
+ delegate sublexer, m[0]
563
+ end
564
+ else
565
+ # Carve's own callout marker, which only survives where no
566
+ # embedded language owns the payload.
567
+ rule %r/<\d+>/, Name::Constant
568
+ rule %r/[^\n<]+|<|\n/, Str::Backtick
569
+ end
570
+ end
571
+ end
572
+
573
+ # A lexer for the fence's declared language, or nil when there is no
574
+ # language, none is known, or the name is ambiguous. Guessing is
575
+ # deliberately not attempted: a wrong guess colours a sample as the wrong
576
+ # language, which reads worse than leaving it plain.
577
+ def fence_lexer(language)
578
+ return nil if language.nil? || language.empty?
579
+
580
+ Lexer.find_fancy(language)
581
+ rescue StandardError
582
+ nil
583
+ end
584
+ end
585
+ end
586
+ end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'rouge'
4
+ require_relative 'rouge/lexers/carve'
5
+
6
+ # A Rouge lexer for the Carve markup language.
7
+ #
8
+ # Requiring this file is the whole interface: defining the lexer class
9
+ # registers it, so `Rouge::Lexer.find('carve')` and
10
+ # `Rouge::Lexer.guess(filename: 'x.crv')` work from that point on.
11
+ module RougeCarve
12
+ VERSION = '0.1.0'
13
+ end
metadata ADDED
@@ -0,0 +1,78 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: rouge-carve
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Mark Scherer
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2026-08-27 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: rouge
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '3.0'
20
+ - - "<"
21
+ - !ruby/object:Gem::Version
22
+ version: '6.0'
23
+ type: :runtime
24
+ prerelease: false
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - ">="
28
+ - !ruby/object:Gem::Version
29
+ version: '3.0'
30
+ - - "<"
31
+ - !ruby/object:Gem::Version
32
+ version: '6.0'
33
+ description: |-
34
+ Syntax highlighting for Carve (.crv) wherever Rouge is the highlighter -
35
+ GitLab, Jekyll, Redcarpet and most Ruby static site generators. Carve
36
+ swaps several of Markdown's delimiters, so lexing it as Markdown is not
37
+ merely imprecise, it inverts them.
38
+ email:
39
+ executables: []
40
+ extensions: []
41
+ extra_rdoc_files: []
42
+ files:
43
+ - CHANGELOG.md
44
+ - LICENSE
45
+ - README.md
46
+ - lib/rouge-carve.rb
47
+ - lib/rouge/demos/carve
48
+ - lib/rouge/lexers/carve.rb
49
+ homepage: https://github.com/markup-carve/rouge-carve
50
+ licenses:
51
+ - MIT
52
+ metadata:
53
+ homepage_uri: https://github.com/markup-carve/rouge-carve
54
+ source_code_uri: https://github.com/markup-carve/rouge-carve
55
+ bug_tracker_uri: https://github.com/markup-carve/rouge-carve/issues
56
+ changelog_uri: https://github.com/markup-carve/rouge-carve/blob/main/CHANGELOG.md
57
+ documentation_uri: https://markup-carve.github.io/carve/
58
+ rubygems_mfa_required: 'true'
59
+ post_install_message:
60
+ rdoc_options: []
61
+ require_paths:
62
+ - lib
63
+ required_ruby_version: !ruby/object:Gem::Requirement
64
+ requirements:
65
+ - - ">="
66
+ - !ruby/object:Gem::Version
67
+ version: 2.7.0
68
+ required_rubygems_version: !ruby/object:Gem::Requirement
69
+ requirements:
70
+ - - ">="
71
+ - !ruby/object:Gem::Version
72
+ version: '0'
73
+ requirements: []
74
+ rubygems_version: 3.4.19
75
+ signing_key:
76
+ specification_version: 4
77
+ summary: A Rouge lexer for the Carve markup language.
78
+ test_files: []