@op-ted/efx-syntax-highlighter 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/README.md +15 -10
  2. package/dist/efx1/efx1Highlighting.js +45 -0
  3. package/dist/efx1/generated/lexer.js +1772 -0
  4. package/dist/efx1/lexer.js +26 -0
  5. package/dist/efx1/parser.js +23 -0
  6. package/dist/efx1/parser.terms.js +118 -0
  7. package/dist/{completion → efx2/completion}/CompletionItems.js +1 -1
  8. package/dist/{completion → efx2/completion}/EfxCompletions.js +2 -2
  9. package/dist/{completion → efx2/completion}/ExpectedTokens.js +14 -14
  10. package/dist/{efxHighlighting.js → efx2/efx2Highlighting.js} +5 -5
  11. package/dist/{generated → efx2/generated}/completions.js +1 -1
  12. package/dist/efx2/generated/lexer.js +3880 -0
  13. package/dist/efx2/generated/words.js +213 -0
  14. package/dist/efx2/lexer.js +49 -0
  15. package/dist/efx2/parser.js +23 -0
  16. package/dist/efx2/parser.terms.js +318 -0
  17. package/dist/efxLanguage.js +66 -17
  18. package/dist/lexer/AntlrLexer.js +427 -0
  19. package/dist/lexer/EditorLexer.js +134 -0
  20. package/dist/lexer/EditorParser.js +69 -0
  21. package/dist/lexer/Lexing.js +238 -0
  22. package/dist/lexer/ParseRanges.js +22 -0
  23. package/package.json +5 -4
  24. package/dist/efx-parser.js +0 -26
  25. package/dist/efx-parser.terms.js +0 -319
  26. package/dist/generated/words.js +0 -215
  27. package/dist/lexer/ExpressionMode.js +0 -44
  28. package/dist/lexer/FormatMode.js +0 -31
  29. package/dist/lexer/IdentifierMode.js +0 -12
  30. package/dist/lexer/LabelMode.js +0 -27
  31. package/dist/lexer/LexerContext.js +0 -76
  32. package/dist/lexer/LexerInput.js +0 -188
  33. package/dist/lexer/LexerMode.js +0 -24
  34. package/dist/lexer/LexerModes.js +0 -50
  35. package/dist/lexer/LineMode.js +0 -85
  36. package/dist/lexer/ReservedWords.js +0 -75
  37. package/dist/lexer/StageHeaderMode.js +0 -12
  38. package/dist/lexer/TemplateMode.js +0 -77
  39. package/dist/lexer/TemplateStartMode.js +0 -25
  40. /package/dist/{completion → efx2/completion}/Variables.js +0 -0
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # efx-syntax-highlighter
2
2
 
3
- EFX support for [CodeMirror 6](https://codemirror.net/) editors: syntax highlighting and completions for the EFX language of the eForms SDK 2. It reads the three kinds of EFX text: template files (view templates), rules files (validation rules) and single expressions.
3
+ EFX support for [CodeMirror 6](https://codemirror.net/) editors: syntax highlighting and completions for EFX-2, the EFX language of the eForms SDK 2, and syntax highlighting for EFX-1, the language of the SDK 1. It reads template files (view templates), rules files (validation rules, EFX-2 only), single expressions with their context, and bare expressions, whose context the application keeps apart.
4
4
 
5
- The parser is a [Lezer](https://lezer.codemirror.net/) grammar that follows the SDK's ANTLR grammar (`efx-grammar/Efx.g4` and `EfxLexer.g4`) rule for rule. Its words and the completion list are generated from the SDK and the EFX Toolkit at the commits pinned in `package.json` (`config`), and a test compares its tokens with those of the ANTLR lexer over the SDK's view templates and rules files. What each release changes is in [CHANGELOG.md](CHANGELOG.md).
5
+ Both parsers read the tokens of the SDK's own ANTLR lexer (`efx-grammar/EfxLexer.g4`): `npm run sync` takes the lexer's compiled table from the lexer ANTLR generates, and a small engine (`src/lexer/`) runs it in the editor as ANTLR runs it, over the whole text, mode by mode. The EFX-2 parser is a [Lezer](https://lezer.codemirror.net/) grammar that follows the SDK's ANTLR grammar (`efx-grammar/Efx.g4`) rule for rule; its completion list comes from the EFX Toolkit. The EFX-1 parser gives expressions no type (`grammar/efx1.grammar`). All of it is taken at the commits pinned in `package.json` (`config`). Tests compare the editor's tokens with ANTLR's lexer's, over the SDK's view templates and rules files and over texts made at random, the EFX-2 trees with ANTLR's parser's, and the EFX-1 grouping of operators with ANTLR's parser's. What each release changes is in [CHANGELOG.md](CHANGELOG.md).
6
6
 
7
7
  ## Install
8
8
 
@@ -36,7 +36,8 @@ const view = new EditorView({
36
36
 
37
37
  The package offers four things:
38
38
 
39
- - **`efx(kind)`** gives an editor what it needs to read one kind of EFX text: its parser, the EFX colours and the completions. The kind is `'template'` for a template file, `'rules'` for a rules file, or `'expression'` for a single expression with its context, such as `WITH ND-Root COMPUTE BT-01-notice is present`. Any other kind is refused with a `RangeError`. `efx(kind).language` is the CodeMirror language, for an application that needs it on its own.
39
+ - **`efx(kind)`** gives an editor what it needs to read one kind of EFX-2 text: its parser, the EFX colours and the completions. The kind is `'template'` for a template file, `'rules'` for a rules file, `'expression'` for a single expression with its context, such as `WITH ND-Root COMPUTE BT-01-notice is present`, or `'bare-expression'` for an expression with no context in the text, such as `BT-01-notice is present`, read as the expression of `WITH … COMPUTE`. Any other kind is refused with a `RangeError`. `efx(kind).language` is the CodeMirror language, for an application that needs it on its own.
40
+ - **`efx(kind, { version: 1 })`** does the same for EFX-1 text, without completions. The kinds are `'template'`, `'expression'`, such as `{ND-Root} ${BT-01-notice is present}`, and `'bare-expression'`; EFX-1 has no rules files. Any other kind or version is refused with a `RangeError`.
40
41
  - **`efxSymbols`** gives the editor the field and node ids it offers: `efxSymbols.of({ fields, nodes })`, each a list of ids, for example those of the SDK's `fields.json`. The ids of several such extensions add up.
41
42
  - **`efxHighlightStyle`** is the EFX colours. `efx(kind)` already uses them as the editor's default colours; an application that gives its editor its own colours with `syntaxHighlighting(...)` replaces them.
42
43
  - **`efxTheme`** is a light look for the editor itself (white background, black cursor). Add it if you want it.
@@ -54,19 +55,23 @@ The application turns completion on, with `autocompletion()` from `@codemirror/a
54
55
 
55
56
  In template text any word can go, so there the list opens only when asked for (Ctrl-Space). Nothing is offered inside a comment or a string.
56
57
 
57
- Known limits: after the label shorthand `#{name|` no field ids are offered; after a `/` the `@` of an attribute is not offered; after a syntax error nothing is offered until the error is fixed.
58
+ Known limits: after a `/` the `@` of an attribute is not offered; after a syntax error nothing is offered until the error is fixed.
58
59
 
59
60
  ## Develop
60
61
 
61
62
  ```bash
62
- npm run build # generates the parser from grammar/efx.grammar and writes the package to dist/
63
- npm test # the package, tree, conformance, style and completion tests, on the build
64
- npm run demo # serves a demo page at http://localhost:8080/demo/ (after npm run build)
65
- npm run site # gathers the demo page and what it loads into site/ (after npm run build)
66
- npm run sync # copies the words, the corpus, the ANTLR oracle and the completion list again
63
+ npm run sync # writes the lexer tables, the words and the completion list (committed), and the SDK's EFX files and ANTLR grammars that the tests and the demo read (not committed)
64
+ npm run build # generates the parsers from grammar/efx2.grammar and grammar/efx1.grammar and writes the package to dist/
65
+ npm test # the package, lexer, conformance, style, EFX-2 tree and completion, EFX-1 entry and incremental tests, on the build
66
+ npm run demo # serves a demo page at http://localhost:8080/demo/ (after npm run sync and npm run build)
67
+ npm run site # gathers the demo page and what it loads into site/ (after npm run sync and npm run build)
67
68
  ```
68
69
 
69
- `npm run sync` reads the SDK and the toolkit from GitHub at the commits pinned in `package.json`, or pins new ones first: `npm run sync -- <owner>/<repository>@<commit> <owner>/<repository>@<commit>` (the SDK, then the toolkit). It needs Node 22.13 or later and network access; behind a proxy, set `NODE_USE_ENV_PROXY=1`. A `GITHUB_TOKEN`, when set, is sent to the GitHub API only. Build and test need neither network nor Java. The Check workflow builds, tests and syncs every push, and fails if a generated file differs from the committed one. The Pages workflow publishes `site/` to the repository's GitHub Pages site on every push to `develop`.
70
+ Each version of EFX has its own places: its grammar in `grammar/efx2.grammar` or `grammar/efx1.grammar`, the modules its parser loads in `src/efx2/` or `src/efx1/` (its lexer table and the lexer's options in `lexer.js`, its highlight tags, and for EFX-2 its completions), and its tests, its copy of the SDK's ANTLR grammar (`oracle/`) and of the SDK's EFX files (`corpus/`) in `test/efx2/` or `test/efx1/`. What both share is at the top of `src/` and `test/`: the entry point and the language (`src/efxLanguage.js`), the lexer engine (`src/lexer/`), the ids (`src/efxSymbols.js`), the colours (`src/efxTheme.js`), and the package, lexer, conformance, style and incremental tests. The build writes each parser beside its modules, in `dist/efx2/` and `dist/efx1/`.
71
+
72
+ The lexer reads the text on its own, as ANTLR's does, whatever the parser makes of the tokens, so the editor shows the tokens ANTLR's lexer reads, errors included. As a token depends on all the text before it, through the lexer's modes, the editor reuses its earlier tree only before the first change to the text, and the lexer reads the parts of the document a parse reads as one text, as ANTLR would (`src/lexer/EditorParser.js`). A comment that a token holds (the `;` that ends a template line, the path of an include) is shown as a comment.
73
+
74
+ `npm run sync` reads the SDK, the toolkit and the SDK of EFX-1 from GitHub at the commits pinned in `package.json`, or pins new ones first: `npm run sync -- <owner>/<repository>@<commit> <owner>/<repository>@<commit> [<owner>/<repository>@<commit>]` (the SDK, then the toolkit, then, if it changes, the SDK of EFX-1). It needs Node 22.13 or later and network access; behind a proxy, set `NODE_USE_ENV_PROXY=1`. A `GITHUB_TOKEN`, when set, is sent to the GitHub API only. Building needs neither network nor Java: the lexer tables, the words and the completion list it builds from are committed. The SDK's EFX files and ANTLR grammars are not: the sync writes them into `corpus/` and `oracle/` of `test/efx2/` and `test/efx1/`, which git ignores, so run `npm run sync` once after cloning and again after moving a pin; the tests say so when they are missing. The Check workflow syncs, builds and tests every push, and fails if a generated file differs from the committed one. The Pages workflow publishes `site/` to the repository's GitHub Pages site on every push to `develop`.
70
75
 
71
76
  ## Publishing
72
77
 
@@ -0,0 +1,45 @@
1
+ // The highlight tags of the EFX-1 grammar (grammar/efx1.grammar, @external propSource), by the same
2
+ // rules as those of EFX-2 (src/efx2/efx2Highlighting.js), so that both read alike with the same colours.
3
+ import { styleTags, tags as t } from "@lezer/highlight";
4
+
5
+ export const EFX1_STYLES = {
6
+ // The words of expressions.
7
+ "And Or Not Is In Like Present Empty Unique Every Some Satisfies If Then Else For Return Notice Axis": t.className,
8
+ // The words chosen from a fixed list: asset and label types.
9
+ "AssetType LabelType": t.className,
10
+
11
+ // Types, which EFX-1 writes with their colon.
12
+ "BooleanTypeCast NumericTypeCast TextTypeCast CodeTypeCast DateTypeCast TimeTypeCast DurationTypeCast": t.typeName,
13
+ ContextTypeCast: t.typeName,
14
+
15
+ // The built-in functions.
16
+ "FunctionCall/Not CountFunction SubstringFunction StringFunction NumberFunction ContainsFunction": t.function(t.keyword),
17
+ "StartsWithFunction EndsWithFunction StringLengthFunction SumFunction FormatNumberFunction ConcatFunction": t.function(t.keyword),
18
+ "DateFunction TimeFunction DayTimeDurationFunction YearMonthDurationFunction AddMeasure SubtractMeasure": t.function(t.keyword),
19
+ "DistinctValuesFunction UnionFunction IntersectFunction ExceptFunction SequenceEqualFunction": t.function(t.keyword),
20
+
21
+ // Literals.
22
+ "STRING UUIDV4 INTEGER DECIMAL DATE TIME DayTimeDurationLiteral YearMonthDurationLiteral": t.literal,
23
+ "True False Always Never": t.literal,
24
+
25
+ // The data of a notice: fields, nodes and attributes.
26
+ "FieldId NodeId BtId AttributeReference/Identifier": t.propertyName,
27
+
28
+ // Variables, and the field value and label of the context in a template.
29
+ "Variable ShorthandFieldValueReferenceFromContextField ShorthandIndirectLabelReferenceFromContextField": t.variableName,
30
+
31
+ // Names of assets in a label.
32
+ "OtherAssetId AssetId/AssetType AssetId/LabelType": t.special(t.name),
33
+
34
+ // Template text.
35
+ FreeText: t.special(t.string),
36
+
37
+ "Comparison Plus Minus Star Slash Percent ColonColon": t.operator,
38
+ "OpenParenthesis CloseParenthesis OpenBracket CloseBracket Pipe": t.punctuation,
39
+ SlashAt: t.punctuation,
40
+ "StartExpression StartLabel StartSelector EndExpression EndLabel": t.bracket,
41
+ Comment: t.lineComment,
42
+ Identifier: t.name,
43
+ };
44
+
45
+ export const efx1Highlighting = styleTags(EFX1_STYLES);