codemirror-lang-raku 0.0.0-stage → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,82 @@
+## 0.2.0 (2026-10-07)
+
+### New features
+
+- Regexes are highlighted inside: the literals `/…/`, `rx/…/`, `m/…/` and
+  the pattern of `s/…/…/`, and the body of a `token`, `rule` or `regex`
+  declaration. A `Regex` node has children for character classes (`CharacterClass`),
+  `<…>` assertions (`Assertion`), quoted literals, escapes, quantifiers
+  and anchors, variables, comments, and the blocks of code in it, which
+  are read as code.
+- `multi foo(…) { }`, `proto foo(…) {*}` and `only foo(…) { }`, without the
+  word `sub`, are `RoutineDeclaration` nodes with a `RoutineName`. The
+  node has no declarator keyword, and `multi` is the node before it.
+- Pod blocks have child nodes for their directives (`PodDirective`),
+  headings (`PodHeading`) and formatting codes (`PodStrong`,
+  `PodEmphasis`, `PodCode`, `PodLink`, `PodFormat`), which are highlighted.
+- Variables, blocks and escapes are interpolated in the text of a
+  `qq:to` heredoc. Its `Heredoc` node has the children of a string that
+  interpolates.
+- Variables, blocks and escapes are interpolated in `<<…>>` and `«…»`
+  word lists. Such a list can hold code over several lines, and can be
+  a subscript: `%h«$key»`, `:a<<b $c>>`.
+- Completion offers the names that the document declares: the variables
+  and attributes in scope, routines, classes, enums, subsets and
+  constants, and after a `.` the methods.
+- Completion also offers placeholder variables (`$^a`) in their block,
+  and variables without a sigil (`my \x`).
+- A line that starts with a trait, as in `is export {` under
+  `sub f($a)`, is indented as the continuation of its statement.
+- Toggling a block comment picks brackets that fit the selection:
+  `` #`[ … ] `` when the selection holds an unbalanced parenthesis, and so
+  on. A comment is removed whichever bracket it was written with.
+- A line that continues a statement is indented one unit further: after
+  a line that ends in an operator or a comma, and when it starts with a
+  method call or an infix operator.
+- The adverbs of a quote that switch kinds of interpolation on and off
+  are taken into account: `q:c{…}` interpolates blocks, `qq:!s[…]` does
+  not interpolate `$` variables. This covers `:c`, `:s`, `:a`, `:h`, `:f`,
+  `:b`, `:qq`, `:q` and their long names.
+- An operator referred to as a routine, as in `&infix:<+>`, is a single
+  `VariableName`. Operator names can also be written as `circumfix:<[ ]>`
+  and `infix:['+']`.
+- `q`, `qq`, `Q` and their variants take a bracket after blanks: `q {…}`,
+  `qw <a b>`.
+- An adverb with an argument that is `False` or `0` is switched off:
+  `qq:c(False)[…]`.
+
+### Bug fixes
+
+- A regex with bracket delimiters no longer ends at the `]` of a
+  character class in it: `rx[ <[a..z]> ]`.
+- `multi foo`, `proto foo` and `only foo` declare a sub also when the
+  signature or the body starts on the next line.
+- A long comment right after a type name (`my Foo  # …`) no longer makes
+  every edit before it reparse the document up to that comment.
+- After the built-in terms `pi`, `π`, `tau`, `τ`, `now`, `time` and `rand`, a
+  `/` or `<` is an operator whatever the spacing: `pi /2`.
+- A keyword that is a routine and is called with parentheses, as in
+  `take(1)` or `not($x)`, is a plain name and not a keyword.
+- The body of a regex declaration that is not closed ends before the next
+  `token`, `rule` or `regex` declaration, so that the declarations after
+  it are still found.
+- A heredoc opener that is only mentioned in a string or in a comment no
+  longer starts a heredoc. Heredocs are now found from the quote tokens
+  of the line, not from its text.
+
+## 0.1.0 (2026-10-06)
+
+### New features
+
+First release.
+
+- Syntax highlighting with the standard tags of `@lezer/highlight`:
+  comments and Pod, quotes of all kinds with interpolation, heredocs,
+  regexes, numbers, variables with sigils and twigils, keywords, operators
+  and declarations
+- A syntax tree with nodes for package, routine, regex, enum, subset and
+  constant declarations and their names
+- Indentation, folding and matching of `{ }`, `( )` and `[ ]`, and folding
+  of Pod blocks, heredocs and embedded comments
+- Completion of keywords and of the commonly used built-in types,
+  routines, methods and special variables

package/LICENSE

@@ -0,0 +1,23 @@
+MIT License
+
+Copyright (C) 2026 by Kan Fushihara
+Copyright (C) 2021 by Marijn Haverbeke <marijnh@gmail.com> and others
+(the CodeMirror language package template this project started from)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

package/README.md

@@ -1,3 +1,239 @@
-# Temporary Holding Version
+# codemirror-lang-raku
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![npm version](https://img.shields.io/npm/v/codemirror-lang-raku)](https://www.npmjs.com/package/codemirror-lang-raku)
+[![Test](https://github.com/kan/codemirror-lang-raku/actions/workflows/test.yml/badge.svg)](https://github.com/kan/codemirror-lang-raku/actions/workflows/test.yml)
+[![License: MIT](https://img.shields.io/npm/l/codemirror-lang-raku)](LICENSE)
+
+[Raku](https://raku.org/) (formerly Perl 6) language support for the
+[CodeMirror 6](https://codemirror.net/) code editor: syntax highlighting,
+indentation, code folding, bracket matching and completion, built on a
+[Lezer](https://lezer.codemirror.net/) grammar.
+
+It covers comments and Pod, quotes of all kinds (`'…'`, `"…"`, `q` / `qq` /
+`Q` with any delimiter, heredocs, `<word lists>`) with interpolation,
+regexes, numbers, variables with sigils and twigils, keywords, operators,
+blocks and declarations.
+
+Try it in the [demo](https://kan.github.io/codemirror-lang-raku/), which
+shows the syntax tree next to the editor.
+
+> **Status: 0.x.** The node names of the syntax tree can still change
+> between minor versions. See [Known limitations](#known-limitations) for
+> what is missing.
+
+## Installation
+
+```sh
+npm install codemirror-lang-raku
+```
+
+## Usage
+
+```javascript
+import {EditorView, basicSetup} from "codemirror"
+import {raku} from "codemirror-lang-raku"
+
+new EditorView({
+  doc: 'say "Hello, Raku!";',
+  extensions: [basicSetup, raku()],
+  parent: document.body
+})
+```
+
+Highlighting uses only the standard tags of `@lezer/highlight`, so any
+CodeMirror theme or highlight style applies.
+
+To load the language on demand, the way `@codemirror/language-data` does:
+
+```javascript
+import {LanguageDescription} from "@codemirror/language"
+
+const rakuDescription = LanguageDescription.of({
+  name: "Raku",
+  alias: ["perl6"],
+  extensions: ["raku", "rakumod", "rakutest", "rakudoc", "p6", "pl6", "pm6", "pod6"],
+  load: () => import("codemirror-lang-raku").then(m => m.raku())
+})
+```
+
+## API Reference
+
+<dl>
+<dt><code><strong>raku</strong>() → LanguageSupport</code></dt>
+<dd><p>Raku language support, with completion, and with block comment
+tokens that fit the selection (<code>#`[ … ]</code> around text that holds an
+unbalanced parenthesis).</p></dd>
+<dt><code><strong>rakuLanguage</strong>: LRLanguage</code></dt>
+<dd><p>A language provider based on the Lezer Raku parser, extended with
+highlighting and indentation information.</p></dd>
+<dt><code><strong>rakuCompletion</strong>: Extension</code></dt>
+<dd><p>Completion of Raku keywords, of the commonly used built-in types,
+routines, methods and special variables, and of the variables, routines,
+types and methods that the document declares. <code>raku()</code> includes it. To
+leave it out, use <code>rakuLanguage</code> in place of <code>raku()</code>.</p></dd>
+<dt><code><strong>parser</strong>: LRParser</code></dt>
+<dd><p>The Lezer parser for Raku, without the editor-specific node props.
+Use <code>rakuLanguage.parser</code> for the configured one.</p></dd>
+</dl>
+
+## Syntax tree
+
+The grammar does not parse statements and expressions. It tokenizes the
+input, nests brackets, and recognizes declarations, which is enough for
+highlighting, folding and building an outline, and keeps the tree usable
+on incomplete code.
+
+| Node | Covers |
+|---|---|
+| `PackageDeclaration` | `class`, `role`, `grammar`, `module`, `package`, with a `PackageName` child, up to the body `Block` or the `;` of a `unit` declaration |
+| `RoutineDeclaration` | `sub`, `method`, `submethod`, with a `RoutineName` child (absent for anonymous routines), up to the body `Block`. Also the `foo(…) { }` of `multi foo(…) { }`, which declares a sub without the word `sub`: that node starts with its `RoutineName` |
+| `RegexDeclaration` | `token`, `rule`, `regex`, with a `RegexName` child, up to the body `Block`, which holds one `Regex` |
+| `EnumDeclaration`, `SubsetDeclaration`, `ConstantDeclaration` | The declarator and its `EnumName`, `SubsetName` or `ConstantName` |
+| `Block`, `Parens`, `Brackets` | `{ }`, `( )`, `[ ]` |
+| `VariableName`, `AttributeName`, `SpecialVariable` | `$x` / `$^a`, `$!x` / `$.x`, `$*x` / `$?x` |
+| `StringLiteral` | A quote of any kind. One that interpolates has `Escape`, `Interpolation`, variable, `Brackets`, `Parens` and `MethodCall` children |
+| `Heredoc` | The text of a heredoc, from the line after its `q:to/END/` opener through its terminator. The opener is a `StringLiteral`. The text of a `qq:to` heredoc has the same children as a string that interpolates. The texts of the heredocs that one line opens are a single node |
+| `Regex` | `/…/`, `rx//`, `m//`, `s///`, `tr///`, or the body of a regex declaration. Its children are `CharacterClass` (`<[a..z]>`), `Assertion` (`<name>`, `<?before …>`), `StringLiteral`, `Escape` (`\d`), `Operator` (quantifiers, `\|`, anchors), `VariableName` (`$x`, `$<name>`), `LineComment`, and a `Block` for each `{ }` of code |
+| `LineComment`, `DocComment`, `BlockComment` | `#`, `#|` / `#=`, `` #`( ) `` |
+| `Pod` | `=begin` … `=end` and the other Pod blocks. Its children are `PodDirective` (`=begin`, `=head1`, `=end pod`), `PodHeading` (the text after `=head1`), and the formatting codes `PodStrong` (`B<…>`), `PodEmphasis` (`I<…>`), `PodCode` (`C<…>`), `PodLink` (`L<…>`) and `PodFormat` (the others) |
+
+Each declarator keyword is a child node named after the keyword (`class`,
+`sub`, `token`, …). The words `unit`, `multi`, `proto`, `only`, `my` and
+`our` are not part of the declaration node. They are the nodes right
+before it, also in `multi foo(…) { }`, where the declaration node has no
+declarator keyword.
+
+## Known limitations
+
+Not supported yet:
+
+- Nothing is highlighted in the replacement of `s/…/…/`, nor in
+  `tr/…/…/`.
+- In a regex, a `<…>` is one
+  token, with the code or the nested `<…>` in it, and has to be closed on
+  its line unless it is a character class. Groups (`[ ]`, `( )`) and
+  adverbs (`:i`, `:my`) are plain text, so the text after `:my` is read
+  as regex.
+- A heredoc is interpolated when everything interpolates in it, as in
+  `qq:to`. One that interpolates in part (`q:c:to`, `qq:!s:to`) is not
+  highlighted inside. When a line opens several heredocs, they are
+  interpolated only if all of them are. A `{` that is not closed in the
+  text takes the rest of the heredoc, with its terminator, for code.
+- In Pod, a formatting code is one token: the codes inside it
+  (`B<I<…>>`) are not told apart. Formatting codes are left alone in a
+  block that is code (`=begin code`, `=code`, `=for code`), but not in a
+  code block inside another block, nor in an indented code paragraph.
+  Tables and the configuration after a directive (`:numbered`) are plain
+  text.
+- A `<<…>>` or `«…»` word list that holds `{ }`, `( )` or `;` can span up
+  to 10 lines. A longer one is not taken to be a word list. Nothing is
+  interpolated in a list that holds an unbalanced `{` or its own
+  delimiter: it is a single token.
+- A Pod block is read where a statement or a term can stand, not
+  everywhere that whitespace can: one between `class` and the name of
+  the class is not read as Pod.
+- A quote adverb is switched off by the argument `False` or `0`, as in
+  `qq:c(False)[…]`. With any other argument, such as a variable, it
+  counts as switched on.
+- The brackets of a block comment are picked for the `toggleBlockComment`
+  command. Text that unbalances every bracket that is tried gets
+  `` #`( `` … `)`, which then does not cover exactly that text.
+- A user-defined operator is recognized where it is declared
+  (`sub infix:<+++>`) and as a routine (`&infix:<+++>`), not where it is
+  used as an operator.
+- Completion does not know types. After a `.` it offers the built-in
+  methods and every method that the document declares, whatever the
+  invocant is. Names from other files are not offered.
+- Completion takes a variable to be declared by `my`, `our`, `state` or
+  `has`, in a signature, after the `->` of a pointy block, or by being a
+  placeholder variable (`$^a`, `$:a`). A sigilless variable is offered
+  when it is declared with a backslash (`my \x`, `sub f(\x)`, `-> \x`),
+  not as a term (`my constant x`, `sub term:<x>`).
+- In a string, only the code in `{ }` and in a subscript is completed,
+  not the method name of `"$x.name()"`.
+
+Heuristics that can be wrong:
+
+- A line is indented as the continuation of a statement when the line
+  before it ends in an operator or a comma, or when it starts with a
+  method call, with an operator that has a space after it, or with a
+  trait (`is`, `does`, `returns`, `handles`, `where`, `will`, as in
+  `is export {`). A statement that continues in another way is not
+  indented further. A line that starts with one of these words for
+  another reason, such as a call of the `is` of `Test`, is indented when
+  the line before it does not end in a `;` or a `}`. Inside `( )` and
+  `[ ]`, indentation follows the brackets only.
+- A capitalized word is taken to be a type name.
+- Whether `/` starts a regex and `<` starts a word list is decided from
+  the token before it. After a bare name, spacing decides: `say /x/` and
+  `say <a b>` are a regex and a word list, `foo / 2` and `n < 3` are not.
+  The built-in terms `pi`, `π`, `tau`, `τ`, `now`, `time` and `rand` are
+  followed by an operator whatever the spacing (`pi /2`), also when a
+  routine of your own has one of these names. After a closing brace, a
+  line break decides.
+- A `<…>` right after a term is a subscript only when it holds plain words
+  on one line: `%h<key>`, but not `$a<$b`. A `<<…>>` or `«…»` that touches
+  the term before it is a subscript when it is at most 24 characters
+  long and starts and ends in a word, a variable or a quote: `%h«$key»`,
+  but not the hyper operators of `@a<<+>>@b` and `@a«R-»@b`. A hyper
+  operator that is a word and touches both of its operands, as in
+  `@a<<min>>@b`, is misread as a subscript.
+- A `<<` or `«` where a term is expected starts a word list when a `>>`
+  or `»` follows, within 10 lines if there is a `{ }`, `( )` or `;` in
+  between. One that is being typed takes the code up to there for its
+  content.
+- In a `<<…>>` word list, a `>>` in a subscript or a call that is
+  interpolated ends the list: `<<@a[1 >> 2]>>`. A heredoc that is opened
+  in a block in a word list or in a regex literal is not read as one.
+- `q`, `qq`, `Q`, `m`, `rx`, `s`, `tr` and their variants start a quote
+  when a delimiter follows directly, so a sigilless `s/2` is misread.
+  `q`, `qq` and `Q` also take a bracket after blanks (`q {…}`), so a
+  routine or a sigilless variable of your own named `q`, before a block
+  or a subscript, is misread. The
+  regex words do not: `m {…}` and `s {…}` are calls.
+- `%name` and `&name` are variables, except between two terms with no
+  space on either side: `$a%b`.
+- After `multi`, `proto` or `only`, a name is taken to declare a sub when
+  a `(` or a `{` follows it, on the same line or the next:
+  `multi foo($x) { }`. One that follows a comment or more than 20
+  characters of whitespace is not seen, and a call such as `only foo(1)`, of a
+  routine of your own that is named `only`, is read as a declaration.
+- A word spelled like a keyword is a keyword wherever it appears, except
+  as a method name, as a declared name, before `=>`, and for the
+  keywords that are routines (`take`, `return`, `next`, `not`, …) right
+  before a `(`: `take(1)`. A sigilless variable or a routine of your own
+  that is named like a keyword is highlighted as one.
+- A quote that does not interpolate (`'…'`, `q{…}`), a regex and a
+  heredoc have to be closed. Otherwise their opener is read as ordinary
+  code.
+- A quote that interpolates (`"…`, `qq[…`), a Pod block or an embedded
+  comment that is not closed runs to the end of the document.
+- The body of a regex declaration that is not closed runs up to the next
+  line that starts a `token`, `rule` or `regex` declaration, or to the
+  end of the document.
+
+## Development
+
+```sh
+npm install
+npm test     # builds, then runs the tests in test/
+npm run dev  # serves demo/, an editor next to the syntax tree of its content
+```
+
+The demo loads the sources, so a change to the grammar shows up on reload.
+
+To release a version:
+
+1. Run `npm version 1.2.3 --no-git-tag-version`, and turn the
+   `## Unreleased` heading of `CHANGELOG.md` into `## 1.2.3 (2026-01-31)`.
+   Commit and push.
+2. Run `npm publish` from that commit. It first runs the tests and checks
+   that `CHANGELOG.md` has that section and that nothing is uncommitted.
+3. Tag that commit `v1.2.3` and push the tag. A workflow then checks that
+   the tag matches `package.json` and that npm has the version, published
+   from that commit, and creates the GitHub release from the section of
+   `CHANGELOG.md`.
+
+## License
+
+[MIT](LICENSE)