@markuplint/parser-utils 4.18.0 → 4.18.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (9) hide show
  1. package/CHANGELOG.md +6 -0
  2. package/SKILL.md +1 -0
  3. package/docs/maintenance.ja.md +27 -9
  4. package/docs/maintenance.md +27 -9
  5. package/docs/parser-class.ja.md +29 -10
  6. package/docs/parser-class.md +29 -10
  7. package/lib/parser.d.ts +14 -1
  8. package/lib/parser.js +82 -3
  9. package/package.json +2 -2
package/CHANGELOG.md CHANGED
@@ -3,6 +3,12 @@
3
3
  All notable changes to this project will be documented in this file.
4
4
  See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
5
5
 
6
+ ## [4.18.3](https://github.com/markuplint/markuplint/compare/v4.18.2...v4.18.3) (2026-05-10)
7
+
8
+ ### Bug Fixes
9
+
10
+ - **parser-utils:** handle raw-text element body in parseCodeFragment per HTML LS §13.2.5.1 ([2904465](https://github.com/markuplint/markuplint/commit/2904465893f54153903c8d92c48349dfe62f9cf2)), closes [#3859](https://github.com/markuplint/markuplint/issues/3859) [#3860](https://github.com/markuplint/markuplint/issues/3860) [#3860](https://github.com/markuplint/markuplint/issues/3860) [#3825](https://github.com/markuplint/markuplint/issues/3825)
11
+
6
12
  # [4.18.0](https://github.com/markuplint/markuplint/compare/v4.14.1...v4.18.0) (2026-04-22)
7
13
 
8
14
  ### Bug Fixes
package/SKILL.md CHANGED
@@ -46,6 +46,7 @@ Create a new parser extending the abstract Parser class. Follow recipe #1 in `do
46
46
  2. Implement `tokenize()` -- invoke the language-specific tokenizer on `this.rawCode`
47
47
  3. Implement `nodeize()` -- convert each AST node using visitor methods
48
48
  4. Set constructor options (`endTagType`, `tagNameCaseSensitive`, `ignoreTags`, etc.)
49
+ - The default `rawTextElements` is `['style', 'script']` (HTML LS §13.2.5.1). If your language treats additional elements as raw text, override it; if your language does NOT (e.g., `<script>` body is parsed as language-specific syntax), pass an empty array.
49
50
 
50
51
  ### Step 3: Export the parser module
51
52
 
package/docs/maintenance.ja.md CHANGED
@@ -130,16 +130,16 @@ IDL属性マップは `src/idl-attributes.ts` で定義されています。新
130
130
 
131
131
  ## 下流影響チェックリスト
132
132
 
133
- このパッケージへの変更は、下流の6つのパーサーパッケージすべてに影響を与える可能性があります:
133
+ このパッケージへの変更は下流のすべてのパーサーに影響します:
134
134
 
135
- | パッケージ | 主な依存関係 |
136
- | --------------------------- | --------------------------------------------------------------------- |
137
- | `@markuplint/html-parser` | Parser 基底クラス、researchTags を使用した visitText |
138
- | `@markuplint/jsx-parser` | Parser 基底クラス、quoteSet を使用した visitAttr、detectElementType |
139
- | `@markuplint/vue-parser` | Parser 基底クラス、visitAttr、flattenNodes、detectElementType |
140
- | `@markuplint/svelte-parser` | Parser 基底クラス、visitText、visitPsBlock、visitChildren、ignoreTags |
141
- | `@markuplint/astro-parser` | Parser 基底クラス（html-parser 経由） |
142
- | `@markuplint/pug-parser` | Parser 基底クラス |
135
+ | パッケージ | 主な依存関係 |
136
+ | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
137
+ | `@markuplint/html-parser` | Parser 基底クラス、researchTags を使用した visitText |
138
+ | `@markuplint/jsx-parser` | Parser 基底クラス、quoteSet を使用した visitAttr、detectElementType、parseCodeFragment |
139
+ | `@markuplint/vue-parser` | Parser 基底クラス、visitAttr、flattenNodes、detectElementType |
140
+ | `@markuplint/svelte-parser` | Parser 基底クラス、visitText、visitPsBlock、visitChildren、ignoreTags |
141
+ | `@markuplint/astro-parser` | Parser 基底クラス（直接）、visitElement、**parseCodeFragment に要素全体の raw を渡す唯一の caller — raw-text element 本文の short-circuit 経路を踏む** |
142
+ | `@markuplint/pug-parser` | Parser 基底クラス |
143
143
 
144
144
  Parser クラスを変更する際は、必ずすべてのパーサーパッケージでテストを実行してください:
145
145
 
@@ -149,6 +149,8 @@ yarn test --scope @markuplint/html-parser --scope @markuplint/jsx-parser \
149
149
  --scope @markuplint/astro-parser --scope @markuplint/pug-parser
150
150
  ```
151
151
 
152
+ `parseCodeFragment`（特に raw-text element 分岐）に手を入れる場合は、`astro-parser` が最も影響を受けやすい caller — まずそこから確認してください。
153
+
152
154
  ## トラブルシューティング
153
155
 
154
156
  ### テンプレート式がパースエラーを引き起こす
@@ -174,3 +176,19 @@ yarn test --scope @markuplint/html-parser --scope @markuplint/jsx-parser \
174
176
  **原因:** パースオプションで `ignoreFrontMatter` が有効になっていない。
175
177
 
176
178
  **解決策:** `parse()` を呼び出す際に `options.ignoreFrontMatter` が `true` であることを確認する。注意: Svelte はこれを明示的に無効にしています。
179
+
180
+ ### `<script>` または `<style>` 本文で `parseCodeFragment` が `Invalid tag syntax` を throw する
181
+
182
+ **症状:** 要素全体の raw を `parseCodeFragment()` に渡すサブクラス（例: `astro-parser`）で、script/style 本文に HTML 風部分文字列（`/<br\s*\/?>/gi` や `/* <br = */` など）が含まれると `SyntaxError: Invalid tag syntax: "..."` が throw する。[#3825](https://github.com/markuplint/markuplint/issues/3825) の v4 backport で、[#3860](https://github.com/markuplint/markuplint/issues/3860) として追跡。
183
+
184
+ **原因:** raw-text 認識がないと `parseCodeFragment()` は本文を再トークナイズし、正規表現中の `<br...>` をタグ開始と解釈、続く `\s`（リテラルなバックスラッシュ）を属性名とみなして throw する。
185
+
186
+ **解決策:** 修正は `parseCodeFragment()`（`src/parser.ts`）内に住んでいる。自閉でない開始タグの `nodeName.toLowerCase()` が `rawTextElements` に含まれる場合、本文は次に現れる ASCII 大文字小文字非依存の `</tagName`（タブ/LF/FF/CR/空白/`>` /`/` のいずれかが続く）まで丸ごと消費される（[HTML Living Standard §13.2.5.1](https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions)）。リグレッションが疑われたら:
187
+
188
+ 1. リグレッションスイートを再実行: `npx vitest run packages/@markuplint/astro-parser/src/parser.spec.ts -t "#3860"`。
189
+ 2. `parseCodeFragment()` を確認 — raw-text 分岐が無傷か、`#getRawTextCloseTagPattern()` のキャッシュが参照されているか、close-tag 正規表現がまだ仕様の文字クラス `[\t\n\f\r >/]` を使っているかをチェック。
190
+ 3. `#3860 raw-text element body via JSX expression child` の jsx 防衛テストは上流 invariant の lock-in が目的で、parser-utils の raw-text 分岐自体は経由しない（本文が expression child として処理されるため）。
191
+
192
+ dev (v5 RC) には [#3859](https://github.com/markuplint/markuplint/pull/3859) が先行マージされている。今後の変更を port する際は両 PR を diff 比較すること。
193
+
194
+ **Escapable raw text 要素 (`<title>`, `<textarea>`) について**: 既定の `rawTextElements` には含めていません。HTML LS では escapable raw text として分類され、本文中の文字参照（`&amp;` 等）の展開が要求されますが、本ブランチではその展開を実装していません。意図的に `rawTextElements` に追加する場合は、本文中の文字参照が decode されず raw のまま渡されることを許容してください。
package/docs/maintenance.md CHANGED
@@ -130,16 +130,16 @@ The IDL attribute map is defined in `src/idl-attributes.ts`. To add a new mappin
130
130
 
131
131
  ## Downstream Impact Checklist
132
132
 
133
- Changes to this package can affect all 6 downstream parser packages:
133
+ Changes to this package can affect every downstream parser:
134
134
 
135
- | Package | Key Dependencies |
136
- | --------------------------- | --------------------------------------------------------------------- |
137
- | `@markuplint/html-parser` | Parser base class, visitText with researchTags |
138
- | `@markuplint/jsx-parser` | Parser base class, visitAttr with quoteSet, detectElementType |
139
- | `@markuplint/vue-parser` | Parser base class, visitAttr, flattenNodes, detectElementType |
140
- | `@markuplint/svelte-parser` | Parser base class, visitText, visitPsBlock, visitChildren, ignoreTags |
141
- | `@markuplint/astro-parser` | Parser base class (via html-parser) |
142
- | `@markuplint/pug-parser` | Parser base class |
135
+ | Package | Key Dependencies |
136
+ | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
137
+ | `@markuplint/html-parser` | Parser base class, visitText with researchTags |
138
+ | `@markuplint/jsx-parser` | Parser base class, visitAttr with quoteSet, detectElementType, parseCodeFragment |
139
+ | `@markuplint/vue-parser` | Parser base class, visitAttr, flattenNodes, detectElementType |
140
+ | `@markuplint/svelte-parser` | Parser base class, visitText, visitPsBlock, visitChildren, ignoreTags |
141
+ | `@markuplint/astro-parser` | Parser base class (direct), visitElement, **parseCodeFragment with full element raw — the only caller that exercises the raw-text element body short-circuit** |
142
+ | `@markuplint/pug-parser` | Parser base class |
143
143
 
144
144
  Always run tests across all parser packages when modifying the Parser class:
145
145
 
@@ -149,6 +149,8 @@ yarn test --scope @markuplint/html-parser --scope @markuplint/jsx-parser \
149
149
  --scope @markuplint/astro-parser --scope @markuplint/pug-parser
150
150
  ```
151
151
 
152
+ When touching `parseCodeFragment` (especially the raw-text element branch), `astro-parser` is the most sensitive caller — start there.
153
+
152
154
  ## Troubleshooting
153
155
 
154
156
  ### Template expressions cause parse errors
@@ -174,3 +176,19 @@ yarn test --scope @markuplint/html-parser --scope @markuplint/jsx-parser \
174
176
  **Cause:** `ignoreFrontMatter` is not enabled in parse options.
175
177
 
176
178
  **Solution:** Ensure `options.ignoreFrontMatter` is `true` when calling `parse()`. Note: Svelte explicitly disables this.
179
+
180
+ ### `<script>` or `<style>` body throws `Invalid tag syntax` from `parseCodeFragment`
181
+
182
+ **Symptom:** A subclass that hands the full element raw to `parseCodeFragment()` (e.g., `astro-parser`) throws `SyntaxError: Invalid tag syntax: "..."` when the script/style body contains HTML-like substrings such as `/<br\s*\/?>/gi` or `/* <br = */`. v4 backport of [#3825](https://github.com/markuplint/markuplint/issues/3825), tracked as [#3860](https://github.com/markuplint/markuplint/issues/3860).
183
+
184
+ **Cause:** Without raw-text awareness, `parseCodeFragment()` re-tokenizes the body and tries to parse the regex's `<br...>` as a start tag, hitting the `\s` (literal backslash) where an attribute name is expected.
185
+
186
+ **Solution:** The fix lives in `parseCodeFragment()` (`src/parser.ts`). After parsing a non-self-closing start tag whose `nodeName.toLowerCase()` matches `rawTextElements`, the body is consumed verbatim until the next ASCII-case-insensitive `</tagName` followed by a tab/LF/FF/CR/space/`>` /`/`, per [HTML Living Standard §13.2.5.1](https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions). If a regression is suspected:
187
+
188
+ 1. Re-run the regression suite: `npx vitest run packages/@markuplint/astro-parser/src/parser.spec.ts -t "#3860"`.
189
+ 2. Inspect `parseCodeFragment()` — confirm the raw-text branch is intact, the `#getRawTextCloseTagPattern()` cache is being consulted, and the close-tag regex still uses the spec character class `[\t\n\f\r >/]`.
190
+ 3. The defensive jsx tests under `#3860 raw-text element body via JSX expression child` exist to lock in upstream invariants — they do **not** exercise the parser-utils raw-text branch directly (their bodies are expression children, not raw element source).
191
+
192
+ The dev (v5 RC) implementation landed first via [#3859](https://github.com/markuplint/markuplint/pull/3859) — diff-compare both PRs when porting future changes.
193
+
194
+ **Note on escapable raw text elements (`<title>`, `<textarea>`):** these are NOT in the default `rawTextElements`. HTML LS classifies them as escapable raw text — they require character-reference (`&amp;`) expansion that this branch does not implement. Add them to a parser's `rawTextElements` only if you accept that character refs in their body will be passed through verbatim instead of decoded.
package/docs/parser-class.ja.md CHANGED
@@ -70,16 +70,16 @@ constructor(options?: ParserOptions, defaultState?: State)
70
70
 
71
71
  コンストラクタは `ParserOptions` オブジェクトとオプションのデフォルト状態値を受け取ります。
72
72
 
73
- | オプション | 型 | デフォルト | 説明 |
74
- | ---------------------- | ---------------------- | ------------------------------- | ---------------------------------------------------------------------------------------- |
75
- | `booleanish` | `boolean` | `false` | 省略された属性値を `true` として扱う（例: JSX `<Component aria-hidden />`） |
76
- | `endTagType` | `EndTagType` | `'omittable'` | `'xml'`: 終了タグ必須またはセルフクローズ; `'omittable'`: 省略可; `'never'`: 不要 |
77
- | `ignoreTags` | `readonly IgnoreTag[]` | `[]` | パース前にマスクするコードブロックのパターン（例: テンプレート式） |
78
- | `maskChar` | `string` | `'\uE000'` (MASK_CHAR) | マスクされたコードブロックを置換するために使用する文字 |
79
- | `tagNameCaseSensitive` | `boolean` | `false` | タグ名の比較で大文字小文字を区別するか（例: JSX、Svelte） |
80
- | `selfCloseType` | `SelfCloseType` | `'html'` | `'html'`: void 要素のみセルフクローズ; `'xml'`: スラッシュで判定; `'html+xml'`: いずれか |
81
- | `spaceChars` | `readonly string[]` | `['\t', '\n', '\f', '\r', ' ']` | タグのパース時に空白として扱う文字 |
82
- | `rawTextElements` | `readonly string[]` | `['style', 'script']` | 子要素をトラバースしない要素（生テキストコンテンツ） |
73
+ | オプション | 型 | デフォルト | 説明 |
74
+ | ---------------------- | ---------------------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
75
+ | `booleanish` | `boolean` | `false` | 省略された属性値を `true` として扱う（例: JSX `<Component aria-hidden />`） |
76
+ | `endTagType` | `EndTagType` | `'omittable'` | `'xml'`: 終了タグ必須またはセルフクローズ; `'omittable'`: 省略可; `'never'`: 不要 |
77
+ | `ignoreTags` | `readonly IgnoreTag[]` | `[]` | パース前にマスクするコードブロックのパターン（例: テンプレート式） |
78
+ | `maskChar` | `string` | `'\uE000'` (MASK_CHAR) | マスクされたコードブロックを置換するために使用する文字 |
79
+ | `tagNameCaseSensitive` | `boolean` | `false` | タグ名の比較で大文字小文字を区別するか（例: JSX、Svelte） |
80
+ | `selfCloseType` | `SelfCloseType` | `'html'` | `'html'`: void 要素のみセルフクローズ; `'xml'`: スラッシュで判定; `'html+xml'`: いずれか |
81
+ | `spaceChars` | `readonly string[]` | `['\t', '\n', '\f', '\r', ' ']` | タグのパース時に空白として扱う文字 |
82
+ | `rawTextElements` | `readonly string[]` | `['style', 'script']` | 子要素をトラバースせず、本文を `parseCodeFragment` が再トークナイズしない要素（HTML LS §13.2.5.1 の raw-text element） |
83
83
 
84
84
  ## パースパイプライン
85
85
 
@@ -228,6 +228,25 @@ visitText(
228
228
 
229
229
  テキストノードを作成します。`researchTags` が true の場合、`parseCodeFragment()` を通じてテキストを再パースし、埋め込まれた HTML タグを発見します。`invalidTagAsText` も true の場合、発見された開始タグによりコンテンツ全体が単一のテキストノードとして扱われます。
230
230
 
231
+ ### parseCodeFragment()
232
+
233
+ ```ts
234
+ parseCodeFragment(
235
+ token: ChildToken,
236
+ options?: { namelessFragment?: boolean }
237
+ ): readonly (MLASTTag | MLASTText)[]
238
+ ```
239
+
240
+ タグとテキストが混在した raw fragment を `MLASTTag` / `MLASTText` の列に再トークナイズします。サブクラス（例: `astro-parser` の `visitElement`）が、上流トークナイザーから要素全体に対する単一の raw 文字列しか得られないときに、開始タグ・本文・終了タグへ分解する目的で利用します。
241
+
242
+ **Raw-text element の本文処理**: 自閉でない開始タグの `nodeName.toLowerCase()` が `rawTextElements`（既定値 `['style', 'script']`）に含まれる場合、本文は次に現れる ASCII 大文字小文字を問わない `</tagName`（タブ/LF/FF/CR/空白/`>` /`/` のいずれかが続く）まで丸ごと消費されます（[HTML Living Standard §13.2.5.1](https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions)）。本文は単一の `#text` ノードとなり、**再トークナイズはしません**。これがないと、`<script>` 本文中の `<br\s*\/?>` のような正規表現がタグとして解釈され `Invalid tag syntax` で throw します。
243
+
244
+ このガードは、上流トークナイザーが要素本文中の裸の `<` をすでに弾く parser（TypeScript ESTree を使う `jsx-parser`）では dormant ですが、要素全体の raw を `parseCodeFragment` に渡す parser（例: `astro-parser`）では必須です。
245
+
246
+ **Escapable raw text 要素 (`<title>`, `<textarea>`) は既定の `rawTextElements` に含めていません**。HTML LS では escapable raw text として分類され、本文中の文字参照（`&amp;` 等）の decode が要求されますが、上記 short-circuit はその decode を実装していません。意図的に `rawTextElements` に追加する場合は、本文中の文字参照が decode されず raw のまま渡されることを許容してください。
247
+
248
+ この分岐は v4 では [#3860](https://github.com/markuplint/markuplint/issues/3860)（[#3825](https://github.com/markuplint/markuplint/issues/3825) の backport）で追加された。dev (v5 RC) には [#3859](https://github.com/markuplint/markuplint/pull/3859) が先行マージされている。今後 raw-text 周りに変更を加える際は両 PR を diff 比較すること。
249
+
231
250
  ### visitComment()
232
251
 
233
252
  ```ts
package/docs/parser-class.md CHANGED
@@ -70,16 +70,16 @@ constructor(options?: ParserOptions, defaultState?: State)
70
70
 
71
71
  The constructor accepts a `ParserOptions` object and an optional default state value:
72
72
 
73
- | Option | Type | Default | Description |
74
- | ---------------------- | ---------------------- | ------------------------------- | ------------------------------------------------------------------------------------------ |
75
- | `booleanish` | `boolean` | `false` | Treat omitted attribute values as `true` (e.g., JSX `<Component aria-hidden />`) |
76
- | `endTagType` | `EndTagType` | `'omittable'` | `'xml'`: end tag required or self-close; `'omittable'`: may omit; `'never'`: never need |
77
- | `ignoreTags` | `readonly IgnoreTag[]` | `[]` | Patterns for code blocks to mask before parsing (e.g., template expressions) |
78
- | `maskChar` | `string` | `'\uE000'` (MASK_CHAR) | Character used to replace masked code blocks |
79
- | `tagNameCaseSensitive` | `boolean` | `false` | Whether tag name comparisons are case-sensitive (e.g., JSX, Svelte) |
80
- | `selfCloseType` | `SelfCloseType` | `'html'` | `'html'`: only void elements self-close; `'xml'`: solidus determines; `'html+xml'`: either |
81
- | `spaceChars` | `readonly string[]` | `['\t', '\n', '\f', '\r', ' ']` | Characters treated as whitespace in tag parsing |
82
- | `rawTextElements` | `readonly string[]` | `['style', 'script']` | Elements whose children are not traversed (raw text content) |
73
+ | Option | Type | Default | Description |
74
+ | ---------------------- | ---------------------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
75
+ | `booleanish` | `boolean` | `false` | Treat omitted attribute values as `true` (e.g., JSX `<Component aria-hidden />`) |
76
+ | `endTagType` | `EndTagType` | `'omittable'` | `'xml'`: end tag required or self-close; `'omittable'`: may omit; `'never'`: never need |
77
+ | `ignoreTags` | `readonly IgnoreTag[]` | `[]` | Patterns for code blocks to mask before parsing (e.g., template expressions) |
78
+ | `maskChar` | `string` | `'\uE000'` (MASK_CHAR) | Character used to replace masked code blocks |
79
+ | `tagNameCaseSensitive` | `boolean` | `false` | Whether tag name comparisons are case-sensitive (e.g., JSX, Svelte) |
80
+ | `selfCloseType` | `SelfCloseType` | `'html'` | `'html'`: only void elements self-close; `'xml'`: solidus determines; `'html+xml'`: either |
81
+ | `spaceChars` | `readonly string[]` | `['\t', '\n', '\f', '\r', ' ']` | Characters treated as whitespace in tag parsing |
82
+ | `rawTextElements` | `readonly string[]` | `['style', 'script']` | Elements whose children are not traversed and whose body bypasses `parseCodeFragment` re-tokenization (raw text content per HTML LS §13.2.5.1) |
83
83
 
84
84
  ## Parse Pipeline
85
85
 
@@ -228,6 +228,25 @@ visitText(
228
228
 
229
229
  Creates a text node. When `researchTags` is true, re-parses the text via `parseCodeFragment()` to discover embedded HTML tags. If `invalidTagAsText` is also true, any discovered start tags cause the entire content to be treated as a single text node.
230
230
 
231
+ ### parseCodeFragment()
232
+
233
+ ```ts
234
+ parseCodeFragment(
235
+ token: ChildToken,
236
+ options?: { namelessFragment?: boolean }
237
+ ): readonly (MLASTTag | MLASTText)[]
238
+ ```
239
+
240
+ Re-tokenizes a raw fragment that may contain a mix of tags and text into a sequence of `MLASTTag` and `MLASTText` nodes. Used by subclasses (e.g., `astro-parser`'s `visitElement`) when the upstream tokenizer produces a single raw blob covering an entire element — the subclass needs the start tag, body, and end tag to be split apart.
241
+
242
+ **Raw-text element handling.** After parsing a non-self-closing start tag whose `nodeName.toLowerCase()` matches an entry in `rawTextElements` (default `['style', 'script']`), the body is consumed verbatim until the next ASCII-case-insensitive `</tagName` followed by a tab/LF/FF/CR/space/`>` /`/` (per [HTML Living Standard §13.2.5.1](https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions)). The body becomes a single `#text` node and is **not** re-tokenized — `<br\s*\/?>` inside a script body would otherwise be interpreted as a tag and throw `Invalid tag syntax`.
243
+
244
+ This guard is dormant for parsers whose upstream tokenizer already rejects bare `<` in element body (`jsx-parser` via TypeScript ESTree) but is critical for parsers that hand the full raw element to `parseCodeFragment` (e.g., `astro-parser`).
245
+
246
+ **Escapable raw text elements (`<title>`, `<textarea>`)** are NOT in the default `rawTextElements`. HTML LS classifies them as escapable raw text — they require character-reference (`&amp;` etc.) decoding inside the body, which the short-circuit above does not implement. Add them to `rawTextElements` only if you accept that character references will be passed through verbatim instead of decoded.
247
+
248
+ This branch was added in v4 via [#3860](https://github.com/markuplint/markuplint/issues/3860) (backport of [#3825](https://github.com/markuplint/markuplint/issues/3825)) — the dev (v5 RC) implementation landed first via [#3859](https://github.com/markuplint/markuplint/pull/3859). Diff-compare both PRs when porting future raw-text changes.
249
+
231
250
  ### visitComment()
232
251
 
233
252
  ```ts
package/lib/parser.d.ts CHANGED
@@ -275,11 +275,24 @@ export declare abstract class Parser<Node extends {} = {}, State extends unknown
275
275
  /**
276
276
  * Re-parses a text token to discover embedded HTML/XML tags within it,
277
277
  * splitting the content into a sequence of tag and text AST nodes.
278
- * Handles self-closing detection, depth tracking, and void element recognition.
278
+ * Handles self-closing detection, depth tracking, void element recognition,
279
+ * and the raw-text element body short-circuit (HTML LS §13.2.5.1).
280
+ *
281
+ * Raw-text element handling only covers the elements listed in
282
+ * `rawTextElements` (default `['style', 'script']`). HTML LS *escapable* raw
283
+ * text elements — `<title>` and `<textarea>` — are intentionally NOT in the
284
+ * default. They allow character references (`&amp;` etc.) in their body, and
285
+ * decoding is not implemented in this short-circuit. A subclass that adds
286
+ * them via the `rawTextElements` option will see character references passed
287
+ * through verbatim.
279
288
  *
280
289
  * @param token - The child token containing the code fragment to re-parse
281
290
  * @param options - Controls whether nameless fragments (JSX `<>`) are recognized
282
291
  * @returns An array of tag and text AST nodes discovered in the code fragment
292
+ * @see https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions
293
+ * @see https://github.com/markuplint/markuplint/issues/3860 (v4 backport tracking issue)
294
+ * @see https://github.com/markuplint/markuplint/issues/3825 (original dev report)
295
+ * @see https://github.com/markuplint/markuplint/pull/3859 (dev fix that landed first)
283
296
  */
284
297
  parseCodeFragment(token: ChildToken, options?: {
285
298
  readonly namelessFragment?: boolean;
package/lib/parser.js CHANGED
@@ -9,7 +9,7 @@ var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (
9
9
  if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
10
10
  return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
11
11
  };
12
- var _Parser_instances, _Parser_booleanish, _Parser_defaultState, _Parser_endTagType, _Parser_ignoreTags, _Parser_maskChar, _Parser_tagNameCaseSensitive, _Parser_selfCloseType, _Parser_spaceChars, _Parser_rawTextElements, _Parser_authoredElementName, _Parser_originalRawCode, _Parser_rawCode, _Parser_defaultDepth, _Parser_walkMethodSequentailPrevNode, _Parser_arrayize, _Parser_concatText, _Parser_concatTextNodes, _Parser_convertIntoInvalidNode, _Parser_createOffsetSpaces, _Parser_createRemnantNode, _Parser_exposeRemnantNodes, _Parser_getEndLocation, _Parser_orphanEndTagToBogusMark, _Parser_pairing, _Parser_parseEndTag, _Parser_parseStartTag, _Parser_parseTag, _Parser_removeChild, _Parser_removeDeprecatedNode, _Parser_removeOffsetSpaces, _Parser_reset, _Parser_setRawCode, _Parser_trimText;
12
+ var _Parser_instances, _Parser_booleanish, _Parser_defaultState, _Parser_endTagType, _Parser_ignoreTags, _Parser_maskChar, _Parser_tagNameCaseSensitive, _Parser_selfCloseType, _Parser_spaceChars, _Parser_rawTextElements, _Parser_rawTextCloseTagPatternCache, _Parser_authoredElementName, _Parser_originalRawCode, _Parser_rawCode, _Parser_defaultDepth, _Parser_walkMethodSequentailPrevNode, _Parser_arrayize, _Parser_concatText, _Parser_concatTextNodes, _Parser_convertIntoInvalidNode, _Parser_createOffsetSpaces, _Parser_createRemnantNode, _Parser_exposeRemnantNodes, _Parser_getEndLocation, _Parser_getRawTextCloseTagPattern, _Parser_orphanEndTagToBogusMark, _Parser_pairing, _Parser_parseEndTag, _Parser_parseStartTag, _Parser_parseTag, _Parser_removeChild, _Parser_removeDeprecatedNode, _Parser_removeOffsetSpaces, _Parser_reset, _Parser_setRawCode, _Parser_trimText;
13
13
  import { isVoidElement as detectVoidElement } from '@markuplint/ml-spec';
14
14
  import { v4 as uuid } from 'uuid';
15
15
  import { attrTokenizer } from './attr-tokenizer.js';
@@ -50,6 +50,13 @@ export class Parser {
50
50
  _Parser_selfCloseType.set(this, 'html');
51
51
  _Parser_spaceChars.set(this, defaultSpaces);
52
52
  _Parser_rawTextElements.set(this, ['style', 'script']);
53
+ /**
54
+ * Compiled close-tag patterns for `#rawTextElements`, populated lazily on first
55
+ * use and reused across every starttag in the same parse. Keyed by the original
56
+ * tag name (as authored in source) so a `tagNameCaseSensitive` parser that
57
+ * preserves casing reuses the same `RegExp` for every occurrence.
58
+ */
59
+ _Parser_rawTextCloseTagPatternCache.set(this, new Map());
53
60
  _Parser_authoredElementName.set(this, void 0);
54
61
  _Parser_originalRawCode.set(this, '');
55
62
  _Parser_rawCode.set(this, '');
@@ -661,11 +668,24 @@ export class Parser {
661
668
  /**
662
669
  * Re-parses a text token to discover embedded HTML/XML tags within it,
663
670
  * splitting the content into a sequence of tag and text AST nodes.
664
- * Handles self-closing detection, depth tracking, and void element recognition.
671
+ * Handles self-closing detection, depth tracking, void element recognition,
672
+ * and the raw-text element body short-circuit (HTML LS §13.2.5.1).
673
+ *
674
+ * Raw-text element handling only covers the elements listed in
675
+ * `rawTextElements` (default `['style', 'script']`). HTML LS *escapable* raw
676
+ * text elements — `<title>` and `<textarea>` — are intentionally NOT in the
677
+ * default. They allow character references (`&amp;` etc.) in their body, and
678
+ * decoding is not implemented in this short-circuit. A subclass that adds
679
+ * them via the `rawTextElements` option will see character references passed
680
+ * through verbatim.
665
681
  *
666
682
  * @param token - The child token containing the code fragment to re-parse
667
683
  * @param options - Controls whether nameless fragments (JSX `<>`) are recognized
668
684
  * @returns An array of tag and text AST nodes discovered in the code fragment
685
+ * @see https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions
686
+ * @see https://github.com/markuplint/markuplint/issues/3860 (v4 backport tracking issue)
687
+ * @see https://github.com/markuplint/markuplint/issues/3825 (original dev report)
688
+ * @see https://github.com/markuplint/markuplint/pull/3859 (dev fix that landed first)
669
689
  */
670
690
  parseCodeFragment(token, options) {
671
691
  const nodes = [];
@@ -748,6 +768,53 @@ export class Parser {
748
768
  }
749
769
  nodes.push(tag);
750
770
  }
771
+ /**
772
+ * Raw-text element body short-circuit (HTML Living Standard §13.2.5.1
773
+ * — "Restrictions on the contents of raw text and escapable raw text
774
+ * elements". The spec section anchor is `cdata-rcdata-restrictions`,
775
+ * where RCDATA stands for "Raw text + Character REF data" — the broader
776
+ * class that covers escapable raw text such as `<title>` / `<textarea>`).
777
+ *
778
+ * Per spec, the contents of `<script>` / `<style>` (and any caller-supplied
779
+ * raw-text element) are NOT re-tokenized as HTML — the only thing that
780
+ * terminates the body is `</tagName` followed by a tab/LF/FF/CR/space/`>`
781
+ * /`/`. Without this guard, fragments like `<script>const t = s.replace(
782
+ * /<br\s*\/?>/gi, " ");</script>` would feed the regex to `#parseTag`,
783
+ * which would try to parse `<br\s*\/?>` as a tag and throw on the
784
+ * backslash (#3860 / dev mirror #3825).
785
+ *
786
+ * This is dormant for `jsx-parser` and `mdx-parser` because their upstream
787
+ * tokenizers reject bare `<` in element body before reaching here, but the
788
+ * fix keeps `parseCodeFragment` honest for any future caller.
789
+ *
790
+ * @see https://html.spec.whatwg.org/multipage/syntax.html#cdata-rcdata-restrictions
791
+ * @see https://github.com/markuplint/markuplint/issues/3860
792
+ */
793
+ if (tag.type === 'starttag' && !isSelfClose && __classPrivateFieldGet(this, _Parser_rawTextElements, "f").includes(tag.nodeName.toLowerCase())) {
794
+ const closeTagPattern = __classPrivateFieldGet(this, _Parser_instances, "m", _Parser_getRawTextCloseTagPattern).call(this, tag.nodeName);
795
+ const match = closeTagPattern.exec(raw);
796
+ if (match) {
797
+ const bodyRaw = raw.slice(0, match.index);
798
+ if (bodyRaw) {
799
+ const bodyToken = this.createToken(bodyRaw, startOffset, startLine, startCol);
800
+ const bodyNode = {
801
+ ...bodyToken,
802
+ type: 'text',
803
+ depth,
804
+ nodeName: '#text',
805
+ parentNode: null,
806
+ };
807
+ nodes.push(bodyNode);
808
+ const bodyEnd = __classPrivateFieldGet(this, _Parser_instances, "m", _Parser_getEndLocation).call(this, bodyToken);
809
+ startOffset = bodyEnd.endOffset;
810
+ startLine = bodyEnd.endLine;
811
+ startCol = bodyEnd.endCol;
812
+ }
813
+ raw = raw.slice(match.index);
814
+ }
815
+ // If no matching close tag is found, fall through to the generic loop
816
+ // so the existing "unclosed tag" handling remains in effect.
817
+ }
751
818
  }
752
819
  return nodes;
753
820
  }
@@ -940,7 +1007,7 @@ export class Parser {
940
1007
  Object.assign(parentNode, { childNodes: newChildNodes });
941
1008
  }
942
1009
  }
943
- _Parser_booleanish = new WeakMap(), _Parser_defaultState = new WeakMap(), _Parser_endTagType = new WeakMap(), _Parser_ignoreTags = new WeakMap(), _Parser_maskChar = new WeakMap(), _Parser_tagNameCaseSensitive = new WeakMap(), _Parser_selfCloseType = new WeakMap(), _Parser_spaceChars = new WeakMap(), _Parser_rawTextElements = new WeakMap(), _Parser_authoredElementName = new WeakMap(), _Parser_originalRawCode = new WeakMap(), _Parser_rawCode = new WeakMap(), _Parser_defaultDepth = new WeakMap(), _Parser_walkMethodSequentailPrevNode = new WeakMap(), _Parser_instances = new WeakSet(), _Parser_arrayize = function _Parser_arrayize(nodeTree) {
1010
+ _Parser_booleanish = new WeakMap(), _Parser_defaultState = new WeakMap(), _Parser_endTagType = new WeakMap(), _Parser_ignoreTags = new WeakMap(), _Parser_maskChar = new WeakMap(), _Parser_tagNameCaseSensitive = new WeakMap(), _Parser_selfCloseType = new WeakMap(), _Parser_spaceChars = new WeakMap(), _Parser_rawTextElements = new WeakMap(), _Parser_rawTextCloseTagPatternCache = new WeakMap(), _Parser_authoredElementName = new WeakMap(), _Parser_originalRawCode = new WeakMap(), _Parser_rawCode = new WeakMap(), _Parser_defaultDepth = new WeakMap(), _Parser_walkMethodSequentailPrevNode = new WeakMap(), _Parser_instances = new WeakSet(), _Parser_arrayize = function _Parser_arrayize(nodeTree) {
944
1011
  let nodeList = [];
945
1012
  this.walk(nodeTree, node => {
946
1013
  nodeList.push(node);
@@ -1065,6 +1132,18 @@ _Parser_booleanish = new WeakMap(), _Parser_defaultState = new WeakMap(), _Parse
1065
1132
  endLine: getEndLine(token.raw, token.startLine),
1066
1133
  endCol: getEndCol(token.raw, token.startCol),
1067
1134
  };
1135
+ }, _Parser_getRawTextCloseTagPattern = function _Parser_getRawTextCloseTagPattern(tagName) {
1136
+ const cached = __classPrivateFieldGet(this, _Parser_rawTextCloseTagPatternCache, "f").get(tagName);
1137
+ if (cached) {
1138
+ return cached;
1139
+ }
1140
+ const escapedName = tagName.replaceAll(/[$()*+.?[\\\]^{|}]/g, '\\$&');
1141
+ // `regexp/strict` cannot statically verify a pattern built from the
1142
+ // dynamic `escapedName` interpolation; the escape above is the contract
1143
+ // that makes this safe for any caller-supplied `rawTextElements` value.
1144
+ const pattern = new RegExp(`</${escapedName}(?=[\\t\\n\\f\\r >/])`, 'i');
1145
+ __classPrivateFieldGet(this, _Parser_rawTextCloseTagPatternCache, "f").set(tagName, pattern);
1146
+ return pattern;
1068
1147
  }, _Parser_orphanEndTagToBogusMark = function _Parser_orphanEndTagToBogusMark(nodeList) {
1069
1148
  const newNodeList = [];
1070
1149
  for (let node of nodeList) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@markuplint/parser-utils",
3
- "version": "4.18.0",
3
+ "version": "4.18.3",
4
4
  "description": "Utility module for markuplint parser plugin",
5
5
  "repository": "git@github.com:markuplint/markuplint.git",
6
6
  "author": "Yusuke Hirao <yusukehirao@me.com>",
@@ -40,5 +40,5 @@
40
40
  "devDependencies": {
41
41
  "@typescript-eslint/typescript-estree": "8.59.0"
42
42
  },
43
- "gitHead": "1885af6349def3f19df975b9e9c399dd47361de1"
43
+ "gitHead": "c53026c2c600bb304f2088589f67303479e92e62"
44
44
  }