rouge-carve 0.1.4 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 300be9217acf879b361b092d5bd9f814858ba169272f98f730cdbfa86a2e75bf
-  data.tar.gz: 1b4e02989bd311b9d0423e1021250d5ddac1982420c15016d80600ddb82386b0
+  metadata.gz: 56fbae7b1785bf82049bdeb87ee240391843fd025ae86d383ad7b48b3b3e9562
+  data.tar.gz: bc9a2e6b121c743dec62e18c059105fd2b3959a0d4fc09684849475c071a7e76
 SHA512:
-  metadata.gz: 63d049afb99f11ac50d7b64a2829c8fec7bc220b1501b621cbbe2e6a87160e75cc326bc94f3724f3280650e12e9be2ce0a00bb160f1c9012e717fc2d538916b9
-  data.tar.gz: d3dce00b0304ec63f85dc9e258f78e7c516d7a9048e1ac9e76e8745140c2931a8ee3e0de70abc7a7a804203b511707d901f7ecd44c152ad08bdaeaa3e22dd35d
+  metadata.gz: be5892326629840d6697bb02c14875b1b075b5220d393e88fd5f85af894af8ada36911dd81c90eb3bb8b4d2cb2c7bc182731704a18c6c2d0ef7f4ef459e360a5
+  data.tar.gz: c14089a1b08264317473a18c84fdad82a946499096f5ef0143952b073a74a4d37da745a53d469a635bae2a752c15a94076c16a2abb4429f85c15504581cce2f5

data/CHANGELOG.md

@@ -2,6 +2,15 @@
 
 ## [Unreleased]
 
+## [0.1.5] - 2026-11-12
+
+- Braced cell alignment (`|={>} Price |`, `|={<^ .x} Name |`, `|{~} 1 |`) lexes as an attribute with its alignment run as an operator, like the deprecated `|=>` prefix; cell pipes and escaped pipes stay outside it (#37).
+- `->`, `<-`, `<->` and `=>` are text, and `-->` and `<--` need their exact hyphen run (#38).
+- A mention or tag name needs an ASCII letter (#38).
+- A first-line `---` opens frontmatter through the end of the document when no closer follows; a mid-document `---` never does (#38).
+- A letter or roman marker with a dot needs two spaces, and a definition term is one line (#38).
+- A code fence opener and a `%%%` comment span open without a closer lookahead, and a multiline attribute block continues only on lines indented past its `{` (#38).
+
 ## [0.1.4] - 2026-10-23
 
 - `csv` and `tsv` fence bodies are highlighted: every field has its own token type and the separator is an operator, colored apart from the fields in the default styles (#25, #26, #27, #28).

data/README.md

@@ -1,84 +1,26 @@
 # rouge-carve
 
-A [Rouge](https://github.com/rouge-ruby/rouge) lexer for the
-[Carve](https://markup-carve.github.io/carve/) markup language.
+Rouge lexer for Carve 0.1 syntax. It registers `.crv`, `.carve`, and Carve fenced code for Rouge 3.30 or later. Read the [Carve documentation](https://markup-carve.github.io/carve/) for the language syntax.
 
-```bash
+## Install
+
+```sh
 gem install rouge-carve
 ```
 
+## Usage
+
 ```ruby
 require 'rouge-carve'
-
-Rouge::Lexer.find('carve')                    # => Rouge::Lexers::Carve
-Rouge::Lexer.guess(filename: 'notes.crv')     # => Rouge::Lexers::Carve
+Rouge::Lexer.find('carve')
 ```
 
-It needs Rouge 3.30 or later and is tested against Rouge 3, 4 and 5.
-
-Requiring the gem registers the lexer. Anything that already highlights with
-Rouge - GitLab, Jekyll, Redcarpet, most Ruby static site generators - then
-handles `.crv` and `.carve` files, and ` ```carve ` fenced blocks, without
-further configuration.
-
-## Why not just use the Markdown lexer
-
-Because it is not close enough to be wrong quietly. Carve deliberately changed
-the delimiters Markdown got backwards, and three of them **invert**:
-
-| Carve | means | Markdown reads it as |
-| --- | --- | --- |
-| `*bold*` | strong | emphasis |
-| `/italic/` | emphasis | literal slashes |
-| `_under_` | underline | emphasis |
-| `~strike~` | strikethrough | subscript, or literal |
-| `{=mark=}` | highlight | literal braces |
-| `{^sup^}` | superscript | literal braces |
-
-Highlighting a Carve document as Markdown does not degrade to plain text. It
-shows bold where the author wrote emphasis.
-
-## What it covers
-
-Every construct in the language: headings, containers (`::: note`), fenced and
-raw blocks, tables with alignment and per-row attributes, definition lists,
-footnotes, citations, cross-references, attribute blocks, task items, critic
-markup, symbol shortcodes, mentions, tags and the typographic runs.
-
-**Fenced code is delegated.** A ` ```ruby ` block inside a `.crv` file is lexed
-as Ruby, because that is what a reader expects. A ` ```=html ` raw block is not:
-its info string names an output format, not a language, so the payload stays
-opaque.
-CSV and TSV fences highlight separators, quoted fields, escaped quotes, and unquoted fields (the header record apart from the data rows).
-
-**Fence widths are respected.** A three-backtick line inside a four-backtick
-fence is content, not a closer. The width is closed over per block rather than
-approximated, which the sibling grammars in other ecosystems cannot do.
-
-## What it cannot say
-
-Rouge has no underline token - the vocabulary stops at `Generic::Emph`,
-`Generic::Strong` and `Generic::EmphStrong`. Carve's `_x_` is underline, so its
-content shares `Generic::Emph` with italic. The delimiters stay `Punctuation`,
-so a consumer can still tell the two apart by the delimiter each carries.
-
-Block openers are matched at any indent. Carve opens a block at column 0 or at
-an enclosing container's content column and nowhere in between, which needs a
-container model this lexer does not carry. The trade-off is deliberate and
-shared with the Prism, highlight.js and Pygments grammars, so the four agree:
-it over-colours an indented-at-document-level opener rather than
-under-colouring the far more common indented construct inside a list item.
-
-## Related
+## Documentation
 
-- [carve](https://github.com/markup-carve/carve) - the language and its spec
-- [pygments-carve](https://github.com/markup-carve/pygments-carve) - the same
-  grammar for Pygments, which this was ported from
-- [carve-grammars](https://github.com/markup-carve/carve-grammars) - TextMate,
-  Prism and highlight.js
-- [carve-css](https://github.com/markup-carve/carve-css) - styles for the HTML
-  Carve renders
+- [Carve documentation](https://markup-carve.github.io/carve/)
+- [Documentation index](docs/README.md)
+- [Development](docs/development.md)
 
-## Development
+## License
 
-Contributor setup, testing, and maintenance notes are in the [development guide](docs/development.md).
+[MIT](LICENSE)

data/lib/rouge/demos/carve

@@ -21,13 +21,13 @@ A word with no registered handler renders as a generic div, which is what
 makes a whole family of constructs possible without touching the parser.
 :::
 
-``` ruby "example.rb"
+```ruby "example.rb"
 def greet(name) = "Hello, #{name}!"
 ```
 
-|= Engine   |= Status |
-| carve-js   | shipped |
-| carve-rs   | shipped |
+|= Engine   |={~} Status |
+| carve-js   | shipped    |
+| carve-rs   | shipped    |
 
 ^ Table 1: the reference implementations.
 

data/lib/rouge/lexers/carve.rb

@@ -158,6 +158,10 @@ module Rouge
           |#{LINE_BREAK}
         )*\}
       /x.freeze
+      # The same block as an interrupting line, whose continuations must be
+      # indented (carve#5080). Exact at column 0; an indented block is checked
+      # against its own `{` column in the :block rule.
+      ATTRS_INTERRUPT = Regexp.new(ATTRS_LINE.source.gsub(LINE_BREAK, '\n(?=[ \t]+[^ \t\n])'), Regexp::EXTENDED)
 
       # The prefix of a line inside a blockquote, nested markers included.
       # Each `>` needs a space or the line end after it: `>foo` and `>> x`
@@ -177,8 +181,9 @@ module Rouge
       end
 
       # An ordered marker: a number, one letter, or a roman numeral. `Body.`
-      # is a word, not a marker.
-      ORDERED = '(?:\\d+|[A-Za-z]|[ivxlcdm]+|[IVXLCDM]+)[.)]'
+      # is a word, not a marker, and a letter with `.` needs two spaces, so
+      # `J. R. R. Tolkien` is prose (carve#4811).
+      ORDERED = "(?:\\d+[.)]|(?:[A-Za-z]|[ivxlcdm]+|[IVXLCDM]+)(?:\\)|\\.(?=(?:#{ATTRS})?  )))"
 
       # The states read a column-0 `+` with a flush-left block below as an
       # attachment. Whether a `+` is the marker is settled by CarveContinuation.
@@ -188,34 +193,38 @@ module Rouge
       LIST_MARKER = "(?:[-*]|#{ORDERED}|\\.)(?:#{ATTRS})?(?:[ \\t]|$)"
       OPENER_RUN = "(?:(?:[-*]|#{ORDERED}|\\.)(?:#{ATTRS})? +[ \\t]*|> )+"
 
+      # A cell's alignment symbols: horizontal, then an optional vertical one.
+      CELL_ALIGN = '(?:[<>~][\\^~v]?|\\?[\\^~v])'
+      # A cell's attribute block when it opens with alignment (carve#5091 D3),
+      # glued to its `|` or `|=` and followed by the cell's space.
+      CELL_BRACE = %(({)(#{CELL_ALIGN})((?:[ ]+(?:"(?:[^"\\\\\n]|\\\\.)*"|'(?:[^'\\\\\n]|\\\\.)*'|[^{}"'|\n])*)?)(})(?= ))
+
       # A table row needs its closing pipe: `| not a table` is a paragraph.
       ROW_END = /(?=(?:\\.|[^\\\n])*\|[ \t]*(?:#{ATTRS})?[ \t]*$)/.freeze
 
       # The block openers that interrupt a paragraph. A list marker and a
-      # caption do not, and a fence does only when it is closed - that one is
-      # checked in code, see the :paraline state.
+      # caption do not. A fence opener does without looking for its closer
+      # (carve#4018); a backtick in a backtick info string makes it a code span.
       INTERRUPT = %r/
         \#{1,6}[ \t] | >(?:[ \t]|$) | ::: | ::[ \t] | %%
+        | `{3,}[^`\n]*$ | ~{3,}
         | (?:(?:\*[ \t]*){3,}|(?:-[ \t]*){3,}|(?:_[ \t]*){3,})$
         | \|=?[<>^v~]*#{ROW_END}
         | (?:\[\^|\*\[|\[)[^\]\n]+\]:[ ]
-        | #{ATTRS_LINE}[ \t]*$
+        | #{ATTRS_INTERRUPT}[ \t]*$
       /x.freeze
 
       # Where a bracketed inline body must close, by the block it sits in: a
       # paragraph ends at a blank line or an interrupting block, a list item
       # also at the next marker, and a heading or table row at its line end.
-      # A fence ends either only when it is closed, which is checked in code.
       BODY_ENDS = {
         para: /\n(?=[ \t]*(?:\n|\z)|#{MARGIN}(?:#{INTERRUPT}))/,
         item: /\n(?=[ \t]*(?:\n|\z)|#{MARGIN}(?:#{INTERRUPT}|#{LIST_MARKER}|:[ \t]))/,
         line: /\n/
       }.freeze
-      FENCE_NEXT = /\n(?=#{MARGIN}(`{3,}|~{3,}))/.freeze
-
       # The block states that decide which of those ends applies.
       BODY_SCOPE = {
-        paragraph: :para, listitem: :item, dlterm: :item, heading: :line,
+        paragraph: :para, listitem: :item, dlterm: :line, heading: :line,
         caption: :line, quoteline: :line, tablerow: :line, defline: :line
       }.freeze
 
@@ -259,8 +268,15 @@ module Rouge
       PLAIN_NO_PIPE = plain_run('|').freeze
       PLAIN_NO_BRACKET = plain_run('\\]').freeze
 
+      # The document start. Front matter opens only here: Rouge 3 and 4 scan
+      # without a fixed anchor, so \A alone would also match mid-document.
       state :root do
-        mixin :block
+        rule %r/\A(?)(---)(?:( ?)([A-Za-z0-9]+))?([ \t]*(?:\n|\z))/ do
+          groups Text, Punctuation, Text, Keyword::Type, Text
+          goto :block
+          push :frontmatter
+        end
+        rule(//) { goto :block }
       end
 
       # ------------------------------------------------------------------
@@ -472,6 +488,11 @@ module Rouge
           # Tables. The header marker, the alignment run and the separator row
           # are their own tokens; cell content is lexed inline. A row needs its
           # closing pipe.
+          rule %r/#{bol}(#{MARGIN})(\|=?)#{CELL_BRACE}#{ROW_END}/ do |m|
+            token Text, m[1]
+            braced_cell(m)
+            open_block :tablerow
+          end
           rule %r/#{bol}(#{MARGIN})(\|=[<>^v~]*)#{ROW_END}/ do
             groups Text, Operator
             open_block :tablerow
@@ -487,32 +508,19 @@ module Rouge
       state(:quoteopeners, &block_openers(''))
 
       state :block do
-        # Front matter, only at the very start of the document. \A is what
-        # keeps a `---yaml` line mid-document from opening one.
-        rule %r/\A(?)(---)([a-zA-Z][\w-]*)?([ \t]*\n)(?=[\s\S]*?^---[ \t]*$)/ do
-          groups Text, Punctuation, Keyword::Type, Text
-          push :frontmatter
-        end
-
         # A comment fence (%%% or longer) versus a one-line comment. The fence
         # has to be tried first: %%% also matches the one-line form. Only a
         # run of the SAME length closes it, so the width is closed over the
-        # way #push_fence does it, and an opener with no such closer below it
-        # is the one-line form. Text after either run is part of the comment.
+        # way #push_fence does it. No closer is looked for: an unclosed span
+        # ends at a line left of its opener, which misses the opener's owner,
+        # or at the end of the document. Text after either run is comment.
         rule %r/^(#{MARGIN})(%%%+)([^\n]*)(\n)/ do |m|
           closer = %r/^([ \t]*)(%{#{m[2].length}})(?!%)([^\n]*)$/
-          unless closer_below?(closer, m[2].length)
-            # The tokens of the one-line rule below. Rouge 3 has no
-            # fallthrough! to hand the line on to it.
-            token Text, m[1]
-            token Comment::Preproc, '%%'
-            token Comment, m[2][2..] + m[3]
-            token Text, m[4]
-            next
-          end
+          outside = m[1].empty? ? nil : %r/^(?=[ \t]{0,#{m[1].length - 1}}[^ \t\n])/
 
           groups Text, Comment::Preproc, Comment, Text
           push do
+            rule(outside) { pop! } if outside
             rule closer do
               groups Text, Comment::Preproc, Comment
               pop!
@@ -536,17 +544,20 @@ module Rouge
 
         mixin :blockopeners
 
-        # A standalone attribute block, which may span lines.
-        rule %r/^(#{MARGIN})(#{ATTRS_LINE})/ do
-          groups Text, Name::Attribute
+        # A standalone attribute block, which may span lines indented past its
+        # `{` (carve#5080). One that does not is a paragraph.
+        rule %r/^(#{MARGIN})(#{ATTRS_LINE})/ do |m|
+          width = m[1].length
+          if m[2].split("\n").drop(1).all? { |line| line[/\A[ \t]*/].length > width }
+            groups Text, Name::Attribute
+          else
+            @current_stream.unscan
+            start_paragraph
+          end
         end
 
         # Any other line with content starts a paragraph.
-        rule %r/^(?=#{MARGIN}\S)/ do
-          @lone_image = !@current_stream.check(IMAGE_LINE).nil?
-          @indented = @current_stream.check(/[ \t]/) ? true : false
-          push :paragraph
-        end
+        rule(%r/^(?=#{MARGIN}\S)/) { start_paragraph }
 
         mixin :inline
       end
@@ -567,7 +578,6 @@ module Rouge
           continuation_host_before? ? pop! : continue_paragraph(:paragraph)
         end
         rule(%r/^(?=#{MARGIN}(?:#{INTERRUPT}))/) { pop! }
-        rule(%r/^(?=#{MARGIN}(`{3,}|~{3,}))/) { |m| fence_closed_below?(m[1]) ? pop! : continue_paragraph(:paragraph) }
         rule(%r/^(?=#{MARGIN}\^[ \t])/) { @lone_image ? pop! : continue_paragraph(:paragraph) }
         # An indented paragraph sits in a container, so a list marker below it
         # opens a sublist or the next item rather than continuing it.
@@ -580,11 +590,16 @@ module Rouge
       { listitem: Text, dlterm: Generic::Heading }.each do |name, tok|
         line = :"#{name}line"
         state name do
-          rule(/\n/) { token Text; goto line }
+          if name == :dlterm
+            rule(/\n/) { token Text; pop! }
+          else
+            rule(/\n/) { token Text; goto line }
+          end
           mixin :inlinecontent
           rule PLAIN, tok
           rule(/./) { token plain_token }
         end
+        next if name == :dlterm
 
         state line do
           rule(%r/^(?=[ \t]*(?:\n|\z))/) { pop! }
@@ -595,7 +610,6 @@ module Rouge
             end
           end
           rule(%r/^(?=#{MARGIN}(?:#{INTERRUPT}|#{LIST_MARKER}|:[ \t]))/) { pop! }
-          rule(%r/^(?=#{MARGIN}(`{3,}|~{3,}))/) { |m| fence_closed_below?(m[1]) ? pop! : goto(name) }
           rule(//) { goto name }
         end
       end
@@ -650,6 +664,9 @@ module Rouge
         rule %r/$/, Text, :pop!
         # A header cell marker, optionally carrying an alignment run. The pipe
         # is part of it, so `|=` reads as one construct.
+        # An escaped pipe is consumed by the escape rule first, so a `|` here
+        # always ends a cell.
+        rule(%r/(\|=?)#{CELL_BRACE}/) { |m| braced_cell(m) }
         rule %r/\|=[<>^v~]*/, Operator
         # A separator row cell, in the native and the GFM spelling.
         rule %r/(?<=\|)[ \t]*:?-{2,}:?[ \t]*(?=\|)/, Punctuation
@@ -877,12 +894,12 @@ module Rouge
 
         # A mention and a tag, each one token: the sigil is part of the name
         # rather than punctuation beside it.
-        rule %r/(?<![\w\/])@[\w][\w.-]*/, Name::Variable::Magic
-        rule %r/(?<![\w&])\#[\w][\w-]*/, Name::Variable::Instance
+        rule %r/(?<![\w\/])@(?=[\w.-]*?[A-Za-z])[\w][\w.-]*/, Name::Variable::Magic
+        rule %r/(?<![\w&])\#(?=[\w-]*?[A-Za-z])[\w][\w-]*/, Name::Variable::Instance
 
         # Typographic runs, longest first: an arrow is not an en dash plus a
         # stray angle bracket, and `---` is not `--` plus `-`.
-        rule %r/<-->|<==>|<=>|-->|<--|==>|<==|->|<-/, Operator
+        rule %r/<-->|<--(?!-)|(?<!-)-->|<=>|==>|<==/, Operator
         rule %r/(?<!-)---(?!-)|(?<!-)--(?!-)/, Punctuation
         rule %r/\.\.\./, Punctuation
       end
@@ -1004,15 +1021,6 @@ module Rouge
           state :"#{body}_#{scope}" do
             rule %r/\]/, Punctuation, :pop!
             rule(%r/(?=#{ends.source}|\n?\z)/) { pop! }
-            # The newline is handed back so the block reads the fence line.
-            rule(FENCE_NEXT) do |m|
-              if fence_closed_below?(m[1])
-                m.pos -= 1
-                pop!
-              else
-                token Text
-              end
-            end
             mixin :inlinecontent
             rule PLAIN_NO_BRACKET, tok
             rule %r/[^\]\n]/, tok
@@ -1247,20 +1255,10 @@ module Rouge
         entry[1].bsearch { |at| at >= pos } ? scope : nil
       end
 
-      # Where a body opened at +pos+ must end: the first block end, or the
-      # first closed fence, after it.
+      # Where a body opened at +pos+ must end: the first block end after it.
       def body_end(text, pos, scope)
         ends = BODY_ENDS[scope]
-        last = text.index(ends, pos) || text.length
-        return last if scope == :line
-
-        at = pos
-        while (fence = text.index(FENCE_NEXT, at)) && fence < last
-          return fence if fence_closed_below?($~[1], fence + 1)
-
-          at = fence + 1
-        end
-        last
+        text.index(ends, pos) || text.length
       end
 
       BRACKET_SCAN = /\\.|(?<!`)(`+)(?!`).*?(?<!`)\1(?!`)|\]/.freeze
@@ -1407,9 +1405,9 @@ module Rouge
 
           lines.map! { |line| line.empty? ? line : line[prefix.length..] }
         end
-        return false if lines.count { |line| line.match?(/\A[ \t]*\z/) } > 1
-
-        lines.none? { |line| line.match?(/\A(?:\#{1,6} +|:{3,}|[-*] +|> |\||`{3,}|~{3,}|(?:[-*_][ \t]*){3,}\z)/) }
+        # A term is one line (carve#4894): only a blank may stand between it
+        # and its `: ` line.
+        lines.size <= 1 && lines.all? { |line| line.match?(/\A[ \t]*\z/) }
       end
 
       # A `:` left of the `::` column of every open term above it ends the
@@ -1499,13 +1497,20 @@ module Rouge
         goto name
       end
 
-      # Whether a fence opened by +run+ on the current line is closed further
-      # down. Only a closed fence interrupts a paragraph.
-      def fence_closed_below?(run, line_start = @current_stream.charpos)
-        text = @current_stream.string
-        line_end = text.index("\n", line_start) || text.length
-        closer = %r/^[ \t]*#{Regexp.escape(run[0])}{#{run.length},}[ \t]*$/
-        closer_below?(closer, run, line_end)
+      # The tokens of a cell marker and its braced alignment block, from a
+      # match whose last five groups are the marker and CELL_BRACE.
+      def braced_cell(m)
+        pipe, open, align, rest, close = m.captures.last(5)
+        token pipe == '|' ? Punctuation : Operator, pipe
+        token Name::Attribute, open
+        token Operator, align
+        token Name::Attribute, rest + close
+      end
+
+      def start_paragraph
+        @lone_image = !@current_stream.check(IMAGE_LINE).nil?
+        @indented = @current_stream.check(/[ \t]/) ? true : false
+        push :paragraph
       end
 
       # Lex the tail of a fence or container opener line without leaving the

data/lib/rouge-carve/version.rb

@@ -3,5 +3,5 @@
 # Kept apart from the entry file so the gemspec can read it without loading
 # Rouge, which is not installed yet when Bundler evaluates the gemspec.
 module RougeCarve
-  VERSION = '0.1.4'
+  VERSION = '0.1.5'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rouge-carve
 version: !ruby/object:Gem::Version
-  version: 0.1.4
+  version: 0.1.5
 platform: ruby
 authors:
 - Mark Scherer
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-10-23 00:00:00.000000000 Z
+date: 2026-11-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rouge