@gooddata/maql-antlr4ts 0.7.2-58cc7118.0 → 0.7.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -35,8 +35,13 @@ const normalizedAst = normalizer.visit(ast);
35
35
  ```
36
36
 
37
37
  The grammar declares the `caseInsensitive` option and the generated lexer honors it, so MAQL keywords may be
38
- written in any letter case with a plain `CharStream`. The exported `CaseInsensitiveCharStream` is kept for
39
- backward compatibility and can still be used in place of `CharStream.fromString`.
38
+ written in any letter case with a plain `CharStream`.
39
+
40
+ Use the exported `CaseInsensitiveCharStream` instead of `CharStream.fromString` when token positions have to
41
+ match JavaScript string offsets or LSP ranges. The antlr4ng stream indexes by Unicode code point, so a character
42
+ outside the Basic Multilingual Plane (an emoji, for example) shifts every following `start`, `stop` and `column`
43
+ by one against the source string. `CaseInsensitiveCharStream` indexes by UTF-16 code unit, so its positions are
44
+ string offsets.
40
45
 
41
46
  ## Local development
42
47
 
@@ -85,48 +90,31 @@ Use [yalc](https://github.com/wclr/yalc) to link the package to other repositori
85
90
 
86
91
  ## Publishing
87
92
 
88
- The package is published by two GitHub workflows, both run manually (`workflow_dispatch`):
89
-
90
- | workflow | version published | npm dist-tag |
91
- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | ------------ |
92
- | [publish-npm-maql-antlr4ts-prerelease](https://github.com/gooddata/gdc-nas/actions/workflows/publish-npm-maql-antlr4ts-prerelease.yml) | `<package.json version>-<commit sha>.0`, for example `0.7.1-a5e4f136.0` | `prerelease` |
93
- | [publish-npm-maql-antlr4ts-stable](https://github.com/gooddata/gdc-nas/actions/workflows/publish-npm-maql-antlr4ts-stable.yml) | `<package.json version>` as is, for example `0.7.1` | `latest` |
94
-
95
- Neither workflow commits a version back. The version comes from `version` in `package.json`, so to publish a new
96
- stable version, or to raise the base version of prereleases, change it in a PR first.
97
-
98
- Both workflows refuse to publish a version that already exists on npm. For the stable workflow this means `version`
99
- in `package.json` was not bumped since the last stable release. For the prerelease workflow it means the same commit
100
- was already published as a prerelease.
93
+ To publish the package, run the [publish_maql_antlr4ts_npm](https://github.com/gooddata/gdc-nas/actions/workflows/publish_maql_antlr4ts_npm.yml) GitHub workflow manually.
101
94
 
102
- To install a prerelease, pin its exact version, or install `@gooddata/maql-antlr4ts@prerelease`. The `latest`
103
- dist-tag only moves on stable releases.
95
+ The workflow publishes a prerelease version `<package.json version + 1 patch>-<commit sha>.0`, for example `0.5.1-a5e4f136.0`.
96
+ It moves the npm `latest` dist-tag to this version. It does not commit the version back. To raise the base version,
97
+ change `version` in `package.json` in a PR before you run the workflow.
104
98
 
105
- The workflows publish to npm with [trusted publishing](https://docs.npmjs.com/trusted-publishers) (OIDC), so they do not need an npm token.
106
- Trusted publishing does not support self-hosted runners, so the workflows run on GitHub-hosted runners.
99
+ The workflow publishes to npm with [trusted publishing](https://docs.npmjs.com/trusted-publishers) (OIDC), so it does not need an npm token.
100
+ Trusted publishing does not support self-hosted runners, so the workflow runs on a GitHub-hosted runner.
107
101
 
108
102
  ### Trusted publisher setup
109
103
 
110
- A trusted publisher on npm is bound to a workflow file name, so each workflow needs its own entry.
111
- npm supports [multiple trusted publishing configurations](https://github.blog/changelog/2026-09-03-multiple-trusted-publishing-configurations-for-npm) per package.
112
- If you rename a workflow file, add an entry for the new name and remove the old one.
113
- This section records how the entries were set up, so that you can change them or set them up again.
104
+ The trusted publisher for the package is set once on npm.
105
+ This section records how it was set up, so that you can change it or set it up again.
114
106
 
115
- You can set them up in one of these ways:
107
+ You can set it up in one of these ways:
116
108
 
117
109
  - **In the npm UI** at https://www.npmjs.com/package/@gooddata/maql-antlr4ts/access.
118
- - **With the npm CLI.** Use npm 11.15.0 or newer, which has the `npm trust` command.
110
+ - **With the npm CLI.** Use npm 11.10.0 or newer, which has the `npm trust` command.
119
111
  Log in to npm (`npm login`) as user `gooddata-ci`.
120
112
  The user must have publish access to `@gooddata/maql-antlr4ts` and two-factor authentication.
121
- Then run, once per workflow:
113
+ Then run:
122
114
 
123
115
  ```sh
124
116
  npm trust github @gooddata/maql-antlr4ts \
125
- --file publish-npm-maql-antlr4ts-prerelease.yml \
126
- --repo gooddata/gdc-nas \
127
- --allow-publish
128
- npm trust github @gooddata/maql-antlr4ts \
129
- --file publish-npm-maql-antlr4ts-stable.yml \
117
+ --file publish_maql_antlr4ts_npm.yml \
130
118
  --repo gooddata/gdc-nas \
131
119
  --allow-publish
132
120
  ```
@@ -1,13 +1,35 @@
1
- import { CharStreamImpl } from "antlr4ng";
1
+ import { type CharStream, type Interval } from "antlr4ng";
2
2
  /**
3
- * ANTLR4 CharStream extension that sees all characters as lowercase
3
+ * ANTLR4 CharStream over a JavaScript string that indexes by UTF-16 code units and lowercases lookahead.
4
+ *
5
+ * The antlr4ng `CharStream.fromString` stream stores Unicode code points, so a character outside the Basic
6
+ * Multilingual Plane (for example an emoji) takes one position there, while it takes two in a JavaScript string
7
+ * and in LSP ranges. Token `start`, `stop` and `column` values would then drift from the source text after such
8
+ * a character. This stream feeds the lexer UTF-16 code units instead, so every token position is directly a
9
+ * string offset. The MAQL grammar only matches such characters with `.` and `~[...]` sets (in strings, comments
10
+ * and object IDs), which accept the two surrogate halves one by one, so the tokens come out the same.
11
+ *
12
+ * The lexer also honors the grammar's `caseInsensitive` option, so lowercasing in `LA` is kept only for
13
+ * backward compatibility.
4
14
  */
5
- export declare class CaseInsensitiveCharStream extends CharStreamImpl {
6
- constructor(s: string, sourceName?: string);
15
+ export declare class CaseInsensitiveCharStream implements CharStream {
16
+ private readonly text;
17
+ name: string;
18
+ index: number;
19
+ constructor(text: string, sourceName?: string);
20
+ reset(): void;
21
+ consume(): void;
7
22
  /**
8
- * Modified lookahead method.
9
- * Transform the character to lowercase before passing it to the caller.
10
- * The input string is not modified.
23
+ * Lookahead by UTF-16 code unit. The unit is lowercased before it is passed to the caller;
24
+ * the input string is not modified.
11
25
  */
12
26
  LA(offset: number): number;
27
+ mark(): number;
28
+ release(_marker: number): void;
29
+ seek(index: number): void;
30
+ get size(): number;
31
+ getSourceName(): string;
32
+ getTextFromRange(start: number, stop?: number): string;
33
+ getTextFromInterval(interval: Interval): string;
34
+ toString(): string;
13
35
  }
@@ -1,25 +1,74 @@
1
1
  // (C) 2022 GoodData Corporation
2
- import { CharStreamImpl } from "antlr4ng";
2
+ import { IntStream, Token } from "antlr4ng";
3
3
  /**
4
- * ANTLR4 CharStream extension that sees all characters as lowercase
4
+ * ANTLR4 CharStream over a JavaScript string that indexes by UTF-16 code units and lowercases lookahead.
5
+ *
6
+ * The antlr4ng `CharStream.fromString` stream stores Unicode code points, so a character outside the Basic
7
+ * Multilingual Plane (for example an emoji) takes one position there, while it takes two in a JavaScript string
8
+ * and in LSP ranges. Token `start`, `stop` and `column` values would then drift from the source text after such
9
+ * a character. This stream feeds the lexer UTF-16 code units instead, so every token position is directly a
10
+ * string offset. The MAQL grammar only matches such characters with `.` and `~[...]` sets (in strings, comments
11
+ * and object IDs), which accept the two surrogate halves one by one, so the tokens come out the same.
12
+ *
13
+ * The lexer also honors the grammar's `caseInsensitive` option, so lowercasing in `LA` is kept only for
14
+ * backward compatibility.
5
15
  */
6
- export class CaseInsensitiveCharStream extends CharStreamImpl {
7
- constructor(s, sourceName) {
8
- super(s);
9
- if (sourceName !== undefined) {
10
- this.name = sourceName;
16
+ export class CaseInsensitiveCharStream {
17
+ text;
18
+ name;
19
+ index = 0;
20
+ constructor(text, sourceName) {
21
+ this.text = text;
22
+ this.name = sourceName ?? IntStream.UNKNOWN_SOURCE_NAME;
23
+ }
24
+ reset() {
25
+ this.index = 0;
26
+ }
27
+ consume() {
28
+ if (this.index >= this.text.length) {
29
+ throw new Error("cannot consume EOF");
11
30
  }
31
+ this.index += 1;
12
32
  }
13
33
  /**
14
- * Modified lookahead method.
15
- * Transform the character to lowercase before passing it to the caller.
16
- * The input string is not modified.
34
+ * Lookahead by UTF-16 code unit. The unit is lowercased before it is passed to the caller;
35
+ * the input string is not modified.
17
36
  */
18
37
  LA(offset) {
19
- const c = super.LA(offset);
20
- if (c < 0) {
21
- return c;
38
+ if (offset === 0) {
39
+ return 0;
40
+ }
41
+ if (offset < 0) {
42
+ offset += 1;
22
43
  }
23
- return String.fromCodePoint(c).toLowerCase().codePointAt(0) ?? c;
44
+ const pos = this.index + offset - 1;
45
+ if (pos < 0 || pos >= this.text.length) {
46
+ return Token.EOF;
47
+ }
48
+ const c = this.text.charCodeAt(pos);
49
+ return String.fromCharCode(c).toLowerCase().charCodeAt(0);
50
+ }
51
+ // mark/release do nothing; the entire buffer is in memory
52
+ mark() {
53
+ return -1;
54
+ }
55
+ release(_marker) { }
56
+ seek(index) {
57
+ this.index = Math.min(index, this.text.length);
58
+ }
59
+ get size() {
60
+ return this.text.length;
61
+ }
62
+ getSourceName() {
63
+ return this.name;
64
+ }
65
+ getTextFromRange(start, stop = this.text.length - 1) {
66
+ return this.text.substring(start, stop + 1);
67
+ }
68
+ getTextFromInterval(interval) {
69
+ return this.getTextFromRange(interval.start, interval.stop);
70
+ }
71
+ toString() {
72
+ return this.text;
24
73
  }
25
74
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@gooddata/maql-antlr4ts",
3
- "version": "0.7.2-58cc7118.0",
3
+ "version": "0.7.2",
4
4
  "description": "TypeScript lexer and parser for MAQL",
5
5
  "license": "MIT",
6
6
  "repository": {