@markuplint/pug-parser 4.6.22 → 4.6.23
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.ja.md +436 -0
- package/ARCHITECTURE.md +436 -0
- package/CHANGELOG.md +3 -3
- package/SKILL.md +118 -0
- package/docs/maintenance.ja.md +188 -0
- package/docs/maintenance.md +188 -0
- package/lib/index.d.ts +6 -0
- package/lib/index.js +6 -0
- package/lib/parser.d.ts +38 -0
- package/lib/parser.js +43 -0
- package/lib/pug-parser/index.d.ts +9 -0
- package/lib/pug-parser/index.js +99 -3
- package/lib/types.d.ts +29 -0
- package/lib/utils/get-offset-from-line-and-col.d.ts +10 -0
- package/lib/utils/get-offset-from-line-and-col.js +10 -0
- package/package.json +5 -5
package/ARCHITECTURE.md
ADDED
|
@@ -0,0 +1,436 @@
|
|
|
1
|
+
# @markuplint/pug-parser
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
`@markuplint/pug-parser` is the Pug template parser for markuplint. It transforms Pug (formerly Jade) indentation-based template syntax into the unified markuplint AST format (`MLASTDocument`). The package uses `pug-lexer` and `pug-parser` as upstream tokenizer/parser, then runs a custom AST optimization pass (`optimizeAST`) to enrich every node with accurate source offsets, raw text slices, and end positions before the main `PugParser` class converts each node into markuplint AST items. It handles inline HTML, tag interpolation (`#[...]`), shorthand attributes (`#id` / `.class`), `&attributes` spread syntax, mixins, conditionals, each loops, includes, extends, filters, and all other Pug-specific constructs.
|
|
6
|
+
|
|
7
|
+
## Directory Structure
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
src/
|
|
11
|
+
├── index.ts — Re-exports parser instance
|
|
12
|
+
├── parser.ts — HtmlInPugParser, PugParser class, visitAttr, visitElement
|
|
13
|
+
├── types.ts — Optimized AST types (ASTNode, ASTBlock, etc.) and PugAST namespace
|
|
14
|
+
├── pug-parser/
|
|
15
|
+
│ └── index.ts — pugParse(), optimizeAST(), helper functions
|
|
16
|
+
└── utils/
|
|
17
|
+
└── get-offset-from-line-and-col.ts — Multi-byte-safe offset calculator
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Architecture Diagram
|
|
21
|
+
|
|
22
|
+
```mermaid
|
|
23
|
+
flowchart TD
|
|
24
|
+
subgraph upstream ["Upstream"]
|
|
25
|
+
pugLexer["pug-lexer\n(Tokenizer)"]
|
|
26
|
+
pugParserLib["pug-parser\n(AST Builder)"]
|
|
27
|
+
mlAst["@markuplint/ml-ast\n(AST types)"]
|
|
28
|
+
parserUtils["@markuplint/parser-utils\n(Abstract Parser class)"]
|
|
29
|
+
htmlParser["@markuplint/html-parser\n(HtmlParser)"]
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
subgraph pkg ["@markuplint/pug-parser"]
|
|
33
|
+
pugParseFn["pugParse()\npug-lexer → pug-parser → optimizeAST"]
|
|
34
|
+
optimizeAST["optimizeAST()\nEnrich nodes with offsets/raw"]
|
|
35
|
+
pugParser["PugParser\nextends Parser‹ASTNode›"]
|
|
36
|
+
htmlInPug["HtmlInPugParser\nextends HtmlParser"]
|
|
37
|
+
visitAttr["visitAttr()\nAttribute processing"]
|
|
38
|
+
types["types.ts\nOptimized AST types"]
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
subgraph downstream ["Downstream"]
|
|
42
|
+
mlCore["@markuplint/ml-core\n(MLASTDocument → MLDOM)"]
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
pugLexer -->|"Token[]"| pugParseFn
|
|
46
|
+
pugParserLib -->|"PugAST.Block"| pugParseFn
|
|
47
|
+
pugParseFn -->|"ASTBlock"| optimizeAST
|
|
48
|
+
optimizeAST -->|"enriched nodes"| pugParser
|
|
49
|
+
mlAst -->|"AST types"| pugParser
|
|
50
|
+
parserUtils -->|"Parser base class"| pugParser
|
|
51
|
+
htmlParser -->|"extends"| htmlInPug
|
|
52
|
+
htmlInPug -->|"inline HTML parsing"| pugParser
|
|
53
|
+
pugParser -->|"visitAttr"| visitAttr
|
|
54
|
+
pugParser -->|"MLASTDocument"| mlCore
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## HtmlInPugParser
|
|
58
|
+
|
|
59
|
+
`HtmlInPugParser` is an internal class that extends `HtmlParser` from `@markuplint/html-parser`. It is used exclusively to parse **inline HTML content** embedded within Pug templates (text nodes containing `<` or `#[`).
|
|
60
|
+
|
|
61
|
+
### Constructor
|
|
62
|
+
|
|
63
|
+
```ts
|
|
64
|
+
class HtmlInPugParser extends HtmlParser {
|
|
65
|
+
constructor() {
|
|
66
|
+
super({
|
|
67
|
+
ignoreTags: [
|
|
68
|
+
{
|
|
69
|
+
type: 'tag-interpolation',
|
|
70
|
+
start: '#[',
|
|
71
|
+
end: ']',
|
|
72
|
+
},
|
|
73
|
+
],
|
|
74
|
+
});
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
The `ignoreTags` option masks `#[...]` tag interpolation sequences so that the HTML parser treats them as preprocessor-specific blocks (`#ps:tag-interpolation`) rather than attempting to parse them as HTML. These blocks are later recursively parsed by a new `PugParser` instance.
|
|
80
|
+
|
|
81
|
+
## PugParser Class
|
|
82
|
+
|
|
83
|
+
### Inheritance
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
Parser<ASTNode> (from @markuplint/parser-utils)
|
|
87
|
+
└── PugParser (this package)
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### Constructor
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
class PugParser extends Parser<ASTNode> {
|
|
94
|
+
constructor() {
|
|
95
|
+
super({
|
|
96
|
+
endTagType: 'never',
|
|
97
|
+
});
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
`endTagType: 'never'` tells the base parser that Pug never produces explicit closing tags — Pug uses indentation-based nesting instead.
|
|
103
|
+
|
|
104
|
+
### Override Methods
|
|
105
|
+
|
|
106
|
+
| Method | Purpose |
|
|
107
|
+
| --------------------- | ----------------------------------------------------------------------------------------------------------------------- |
|
|
108
|
+
| `tokenize()` | Calls `pugParse()` to produce an optimized Pug AST |
|
|
109
|
+
| `parseError()` | Converts pug-lexer/pug-parser errors (with `msg`, `line`, `column`, `src`) into `ParserError` |
|
|
110
|
+
| `nodeize()` | Dispatches each Pug AST node type to the appropriate visitor method |
|
|
111
|
+
| `afterFlattenNodes()` | Calls `super.afterFlattenNodes()` with `exposeInvalidNode: false` and `exposeWhiteSpace: false` |
|
|
112
|
+
| `visitElement()` | Constructs an `MLASTElement` start tag with pre-parsed attributes and visits child nodes |
|
|
113
|
+
| `visitSpreadAttr()` | Returns `null` (spread attributes are handled inline in the `Tag` case, not through the base class spread attr visitor) |
|
|
114
|
+
| `visitAttr()` | Handles Pug-specific attribute syntax (shorthand, quoted names, unescaped, script values) |
|
|
115
|
+
|
|
116
|
+
## tokenize()
|
|
117
|
+
|
|
118
|
+
```ts
|
|
119
|
+
tokenize(options?: ParseOptions) {
|
|
120
|
+
const offsetOffset = options?.offsetOffset ?? 0;
|
|
121
|
+
const ast = pugParse(this.rawCode, offsetOffset >= 1).nodes;
|
|
122
|
+
return {
|
|
123
|
+
ast: [...ast],
|
|
124
|
+
isFragment: true,
|
|
125
|
+
};
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
- Calls `pugParse()` with the raw Pug source code
|
|
130
|
+
- The `useOffset` parameter (set to `true` when `offsetOffset >= 1`) filters out `indent` and `outdent` tokens from the lexer output — this is necessary when parsing sub-templates (e.g., tag interpolation content) at a non-zero offset, because the indentation context is inherited from the parent
|
|
131
|
+
- Always returns `isFragment: true` since Pug templates are always treated as fragments
|
|
132
|
+
|
|
133
|
+
## nodeize() Details
|
|
134
|
+
|
|
135
|
+
The `nodeize()` method is the central dispatch that converts each optimized Pug AST node into markuplint AST items. It first determines the parent namespace, then slices the source fragment using the node's computed offsets.
|
|
136
|
+
|
|
137
|
+
### Doctype
|
|
138
|
+
|
|
139
|
+
```ts
|
|
140
|
+
case 'Doctype':
|
|
141
|
+
return this.visitDoctype({ ...token, depth, parentNode, name: originNode.raw ?? '', publicId: '', systemId: '' });
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Delegates to `visitDoctype()` with the raw doctype string. Public and system IDs are empty since Pug doctypes use shorthand syntax (`doctype html`).
|
|
145
|
+
|
|
146
|
+
### Text
|
|
147
|
+
|
|
148
|
+
Text nodes have three processing paths:
|
|
149
|
+
|
|
150
|
+
1. **Empty text** (`raw.trim() === ''`): Returns empty array (ignored)
|
|
151
|
+
2. **Simple text** (no `<` or `#[`): Delegates to `visitText()` directly
|
|
152
|
+
3. **Text containing HTML or tag interpolation**: Parsed through `HtmlInPugParser`:
|
|
153
|
+
- Creates a new `HtmlInPugParser` instance and parses the text content with offset/line/column context
|
|
154
|
+
- Iterates over the resulting node list
|
|
155
|
+
- Nodes named `#ps:tag-interpolation` have their `#[` prefix and `]` suffix stripped, then the inner content is recursively parsed by a new `PugParser` instance
|
|
156
|
+
- All other nodes are passed through as-is
|
|
157
|
+
|
|
158
|
+
This recursive parsing chain allows Pug tag interpolation (`#[strong bold text]`) to be fully resolved into markuplint nodes.
|
|
159
|
+
|
|
160
|
+
### Comment / BlockComment
|
|
161
|
+
|
|
162
|
+
- **Comment**: Single-line Pug comments (`//- comment` or `// comment`). Delegates to `visitComment()` with `isBogus: false`
|
|
163
|
+
- **BlockComment**: Multi-line block comments. Computes the end offset from the last child block node, then delegates to `visitComment()`
|
|
164
|
+
|
|
165
|
+
### Tag
|
|
166
|
+
|
|
167
|
+
Tag processing is the most complex path:
|
|
168
|
+
|
|
169
|
+
1. **Namespace resolution**: Calls `getNamespace()` from `@markuplint/html-parser` with the tag name and parent namespace
|
|
170
|
+
2. **Regular attributes**: Each attribute from `originNode.attrs` is processed:
|
|
171
|
+
- Offset/endOffset are computed via `this.getOffsetsFromCode()`
|
|
172
|
+
- For shorthand attributes (`#id` / `.class`), the attribute has `offset === endOffset` in the Pug AST, so `endOffset` is recalculated as `attr.offset + attr.val.length - 1`
|
|
173
|
+
- Each attribute token is passed to `this.visitAttr()`
|
|
174
|
+
3. **`&attributes` spread syntax**: Each `attributeBlock` is processed:
|
|
175
|
+
- The `&attributes(` prefix length is added to the column to skip it
|
|
176
|
+
- A token is created from the inner expression
|
|
177
|
+
- The result is typed as `{ type: 'spread', nodeName: '#spread' }`
|
|
178
|
+
4. **Element creation**: Calls `this.visitElement()` with the tag token, child block nodes, and the combined attributes array (regular + spread)
|
|
179
|
+
|
|
180
|
+
### Default (Pug-specific constructs)
|
|
181
|
+
|
|
182
|
+
All other node types — `Conditional`, `Code`, `Each`, `Mixin`, `MixinBlock`, `Include`, `RawInclude`, `Extends`, `NamedBlock`, `Case`, `When`, `While`, `Filter`, `YieldBlock`, `InterpolatedTag`, `FileReference` — are mapped to preprocessor-specific blocks via `visitPsBlock()`.
|
|
183
|
+
|
|
184
|
+
For nodes with a `file` property (e.g., `Include`, `Extends`), the token is extended to include the file path in the raw source by computing the file reference offset from the node's end position.
|
|
185
|
+
|
|
186
|
+
Child nodes are extracted from either `block.nodes` or `nodes`, depending on the node type.
|
|
187
|
+
|
|
188
|
+
## Attribute Processing (visitAttr)
|
|
189
|
+
|
|
190
|
+
`visitAttr()` handles the full range of Pug attribute syntax:
|
|
191
|
+
|
|
192
|
+
### Shorthand Attributes
|
|
193
|
+
|
|
194
|
+
When the raw attribute starts with `#` or `.`:
|
|
195
|
+
|
|
196
|
+
```ts
|
|
197
|
+
if (token.raw[0] === '#' || token.raw[0] === '.') {
|
|
198
|
+
// Parse as value-only (AttrState.BeforeValue)
|
|
199
|
+
// Set potentialName: '#' → 'id', '.' → 'class'
|
|
200
|
+
// isDuplicatable: true for class (multiple classes allowed)
|
|
201
|
+
}
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
- `#id-value` is parsed as: `potentialName: 'id'`, `potentialValue: 'id-value'`
|
|
205
|
+
- `.class-name` is parsed as: `potentialName: 'class'`, `potentialValue: 'class-name'`, `isDuplicatable: true`
|
|
206
|
+
- The `startState: AttrState.BeforeValue` tells the parser that the entire token is a value (no name=value structure)
|
|
207
|
+
- `quoteSet: []` and `endOfUnquotedValueChars: []` disable quote detection
|
|
208
|
+
|
|
209
|
+
### Regular Attributes
|
|
210
|
+
|
|
211
|
+
For non-shorthand attributes:
|
|
212
|
+
|
|
213
|
+
- `quoteSet: []` — Pug attributes don't use HTML-style quotes for the attribute itself
|
|
214
|
+
- `noQuoteValueType: 'script'` — unquoted values are treated as JavaScript expressions
|
|
215
|
+
- `endOfUnquotedValueChars: []` — no specific end-of-value delimiter characters
|
|
216
|
+
- If the attribute name is `class`, `isDuplicatable` is set to `true`
|
|
217
|
+
|
|
218
|
+
### Quoted Attribute Names
|
|
219
|
+
|
|
220
|
+
```ts
|
|
221
|
+
if (attr.name.raw.startsWith("'") && attr.name.raw.endsWith("'")) {
|
|
222
|
+
this.updateAttr(attr, { potentialName: attr.name.raw.slice(1, -1) });
|
|
223
|
+
}
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Pug allows attribute names to be wrapped in single quotes (e.g., `'data-value'="foo"`). The quotes are stripped to get the actual attribute name.
|
|
227
|
+
|
|
228
|
+
### Unescaped Attributes
|
|
229
|
+
|
|
230
|
+
```ts
|
|
231
|
+
if (attr.name.raw.endsWith('!')) {
|
|
232
|
+
this.updateAttr(attr, { potentialName: attr.name.raw.slice(0, -1) });
|
|
233
|
+
}
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
Pug's `!` suffix on attribute names (e.g., `href!="/url"`) indicates the value should not be HTML-escaped. The `!` is removed from the potential name.
|
|
237
|
+
|
|
238
|
+
### Value Type Parsing
|
|
239
|
+
|
|
240
|
+
The attribute value is analyzed using `scriptParser()` from `@markuplint/parser-utils`:
|
|
241
|
+
|
|
242
|
+
| scriptParser Token Type | Result |
|
|
243
|
+
| ----------------------- | ------------------------------------------------------------------------------------------------- |
|
|
244
|
+
| `Numeric` | `valueType: 'number'` |
|
|
245
|
+
| `Boolean` | `valueType: 'boolean'` |
|
|
246
|
+
| `String` / `Template` | Re-parsed with `super.visitAttr()` to extract quotes and value; `valueType: 'code'` if `!` suffix |
|
|
247
|
+
| Multiple tokens | `isDynamicValue: true`, `valueType: 'code'` (complex JavaScript expression) |
|
|
248
|
+
|
|
249
|
+
## Pug AST Optimization (pug-parser/index.ts)
|
|
250
|
+
|
|
251
|
+
### pugParse()
|
|
252
|
+
|
|
253
|
+
The entry point for Pug template parsing:
|
|
254
|
+
|
|
255
|
+
```
|
|
256
|
+
Pug source → pug-lexer → [optional indent/outdent filter] → pug-parser → optimizeAST → ASTBlock
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
1. **Lexing**: `lexer(pug)` produces a `Token[]` array
|
|
260
|
+
2. **Indent filtering**: When `useOffset` is `true`, `indent` and `outdent` tokens are removed to prevent indentation errors when parsing sub-templates
|
|
261
|
+
3. **Cloning**: Tokens are cloned via `structuredClone()` because both the parser and optimization pass need independent token references
|
|
262
|
+
4. **Parsing**: `parser(lexOrigin)` produces a raw `PugAST.Block`
|
|
263
|
+
5. **Optimization**: `optimizeAST(originAst, lex, pug)` enriches every node with computed offsets and raw source
|
|
264
|
+
|
|
265
|
+
### optimizeAST()
|
|
266
|
+
|
|
267
|
+
Recursively transforms the raw pug-parser AST into an optimized AST. For each node:
|
|
268
|
+
|
|
269
|
+
1. **Offset computation**: Computes the character offset from line/column using `getOffsetsFromLines()`
|
|
270
|
+
2. **End location**: Finds the matching lexer token via `getLocationFromToken()` to determine end line/column/offset
|
|
271
|
+
3. **Raw source**: Slices the original source: `pug.slice(offset, endOffset)`
|
|
272
|
+
4. **Type-specific processing**:
|
|
273
|
+
|
|
274
|
+
| Node Type | Processing |
|
|
275
|
+
| ----------------- | -------------------------------------------------------------------------------------------------- |
|
|
276
|
+
| `Block` | Recursively optimizes and flattens into parent |
|
|
277
|
+
| `Tag` | `getAttrs()` for attributes, `getEndAttributeLocation()` for tag end, recursive block optimization |
|
|
278
|
+
| `Conditional` | Optimizes consequent block, then `optimizeASTOfConditionalNode()` for else-if/else chains |
|
|
279
|
+
| `Each` | Optimizes child block |
|
|
280
|
+
| `Include` | Optimizes child block |
|
|
281
|
+
| `RawInclude` | Preserves filters |
|
|
282
|
+
| `Mixin` | `getLocationFromToken()` with `['mixin', 'call']` type filter, optional block optimization |
|
|
283
|
+
| `MixinBlock` | Simple enrichment |
|
|
284
|
+
| `NamedBlock` | Re-wraps as `Block` for recursive optimization |
|
|
285
|
+
| `Comment` | Simple enrichment |
|
|
286
|
+
| `BlockComment` | Optimizes child block |
|
|
287
|
+
| `Code` | Optimizes child block |
|
|
288
|
+
| `Text` | `getPipelessText()` check, then `getRawTextAndLocationEnd()` for multi-text handling |
|
|
289
|
+
| `Doctype` | Simple enrichment |
|
|
290
|
+
| `Case` / `When` | Optimizes child block |
|
|
291
|
+
| `Filter` | `getAttrs()` for filter options, `getEndAttributeLocation()` for end, block optimization |
|
|
292
|
+
| `Extends` | Simple enrichment |
|
|
293
|
+
| `FileReference` | Simple enrichment |
|
|
294
|
+
| `IncludeFilter` | Simple enrichment |
|
|
295
|
+
| `InterpolatedTag` | Optimizes child block |
|
|
296
|
+
| `While` | Simple enrichment |
|
|
297
|
+
| `YieldBlock` | Simple enrichment |
|
|
298
|
+
|
|
299
|
+
5. **Text merging**: After processing all nodes, `mergeTextNode()` combines consecutive `Text` nodes into single nodes
|
|
300
|
+
|
|
301
|
+
### getOffsetsFromLines()
|
|
302
|
+
|
|
303
|
+
Builds a cumulative offset lookup table from the source string:
|
|
304
|
+
|
|
305
|
+
```ts
|
|
306
|
+
function getOffsetsFromLines(pug: string): number[] {
|
|
307
|
+
const lines = pug.split(/\n/);
|
|
308
|
+
let chars = 0;
|
|
309
|
+
return lines.map(line => {
|
|
310
|
+
chars += line.length + 1; // +1 for newline character
|
|
311
|
+
return chars;
|
|
312
|
+
});
|
|
313
|
+
}
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
Each entry `offsets[i]` contains the cumulative character count through line `i+1` (including the newline). Used as: `lineOffset = offsets[line - 2]` to get the offset of the start of the target line.
|
|
317
|
+
|
|
318
|
+
### mergeTextNode()
|
|
319
|
+
|
|
320
|
+
Combines consecutive `Text` nodes by extending the first node's `raw`, `endColumn`, `endLine`, and `endOffset` to cover all merged nodes:
|
|
321
|
+
|
|
322
|
+
```ts
|
|
323
|
+
if (prevNode.type === 'Text' && node.type === 'Text') {
|
|
324
|
+
prevNode.raw = pug.slice(prevNode.offset, node.endOffset);
|
|
325
|
+
prevNode.endColumn = node.endColumn;
|
|
326
|
+
prevNode.endLine = node.endLine;
|
|
327
|
+
prevNode.endOffset = node.endOffset;
|
|
328
|
+
}
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
### getAttrs()
|
|
332
|
+
|
|
333
|
+
Enriches attribute data by correlating each attribute from the Pug AST with its corresponding lexer token:
|
|
334
|
+
|
|
335
|
+
1. Computes the attribute's offset from `offsets[attr.line - 2] + attr.column - 1`
|
|
336
|
+
2. Finds the matching lexer token by line/column
|
|
337
|
+
3. Computes the attribute's length from the token's location span
|
|
338
|
+
4. Slices the raw source and creates the enriched `ASTAttr`
|
|
339
|
+
|
|
340
|
+
### getPipelessText()
|
|
341
|
+
|
|
342
|
+
Detects whether a `Text` node is part of a **pipeless text block** — indented text content below a tag without pipe characters:
|
|
343
|
+
|
|
344
|
+
```pug
|
|
345
|
+
p.
|
|
346
|
+
This is pipeless text.
|
|
347
|
+
It spans multiple lines.
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
Searches for `start-pipeless-text` and `end-pipeless-text` tokens in the lexer output. If the text node falls within such a range, returns the full span of the pipeless text block.
|
|
351
|
+
|
|
352
|
+
### getEndAttributeLocation()
|
|
353
|
+
|
|
354
|
+
Determines the end position of a tag including all its attributes by scanning lexer tokens after the tag's position. It tracks tokens until it encounters one that is not `attribute`, `start-attributes`, `end-attributes`, `id`, or `class`, then returns the end position of the last attribute-related token.
|
|
355
|
+
|
|
356
|
+
### getRawTextAndLocationEnd()
|
|
357
|
+
|
|
358
|
+
Handles complex text node processing for multi-line text and piped text:
|
|
359
|
+
|
|
360
|
+
1. Walks through lexer tokens from the text node's start position
|
|
361
|
+
2. Tracks `text` and `text-html` tokens for end position
|
|
362
|
+
3. Monitors `indent` / `outdent` tokens for depth tracking
|
|
363
|
+
4. Detects piped text (lines starting with `|`) and stops processing
|
|
364
|
+
5. Returns an array of `ASTText` nodes with computed location data
|
|
365
|
+
|
|
366
|
+
### optimizeASTOfConditionalNode()
|
|
367
|
+
|
|
368
|
+
Recursively processes `else if` / `else` chains in conditional nodes:
|
|
369
|
+
|
|
370
|
+
1. For `else-if` branches: Finds the `else-if` token in the lexer output, computes its location, and creates a `Conditional` node
|
|
371
|
+
2. For `else` branches (`alternate` of type `Block`): Finds the `else` token, computes location, creates a `Conditional` node
|
|
372
|
+
3. For chained conditionals (`alternate` of type `Conditional`): Recursively calls itself with increased depth
|
|
373
|
+
|
|
374
|
+
## Version Compatibility
|
|
375
|
+
|
|
376
|
+
The package uses `pug-lexer` and `pug-parser` which support the Pug 3 syntax specification. The Pug AST types in `types.ts` are modeled after the [pug-ast-spec](https://github.com/pugjs/pug-ast-spec/blob/master/parser.md) with extensions for attribute blocks and additional location data.
|
|
377
|
+
|
|
378
|
+
## Key Source Files
|
|
379
|
+
|
|
380
|
+
| File | Purpose |
|
|
381
|
+
| ------------------------------------------- | ------------------------------------------------------------------------------- |
|
|
382
|
+
| `src/parser.ts` | `HtmlInPugParser` and `PugParser` classes with all visitor methods |
|
|
383
|
+
| `src/pug-parser/index.ts` | `pugParse()`, `optimizeAST()`, and all AST enrichment helper functions |
|
|
384
|
+
| `src/types.ts` | `ASTNode` union, `ASTBlock`, optimized node types, and `PugAST` namespace types |
|
|
385
|
+
| `src/utils/get-offset-from-line-and-col.ts` | `getOffsetFromLineAndCol()` multi-byte-safe offset calculator |
|
|
386
|
+
| `src/index.ts` | Re-exports the `parser` instance |
|
|
387
|
+
|
|
388
|
+
## External Dependencies
|
|
389
|
+
|
|
390
|
+
| Dependency | Purpose |
|
|
391
|
+
| -------------------------- | -------------------------------------------------------------------------------- |
|
|
392
|
+
| `@markuplint/html-parser` | `HtmlParser` class (extended by `HtmlInPugParser`) and `getNamespace()` function |
|
|
393
|
+
| `@markuplint/ml-ast` | AST type definitions (`MLASTElement`, `MLASTAttr`, `MLASTParentNode`, etc.) |
|
|
394
|
+
| `@markuplint/parser-utils` | Abstract `Parser` class, `ParserError`, `AttrState`, `scriptParser`, utilities |
|
|
395
|
+
| `pug-lexer` | Pug template tokenization |
|
|
396
|
+
| `pug-parser` | Pug token stream to AST conversion |
|
|
397
|
+
|
|
398
|
+
## Integration Points
|
|
399
|
+
|
|
400
|
+
```mermaid
|
|
401
|
+
flowchart TD
|
|
402
|
+
subgraph upstream ["Upstream"]
|
|
403
|
+
pugLexer["pug-lexer"]
|
|
404
|
+
pugParserLib["pug-parser"]
|
|
405
|
+
mlAst["@markuplint/ml-ast\n(AST types)"]
|
|
406
|
+
parserUtils["@markuplint/parser-utils\n(Parser base class)"]
|
|
407
|
+
htmlParser["@markuplint/html-parser\n(HtmlParser)"]
|
|
408
|
+
end
|
|
409
|
+
|
|
410
|
+
subgraph pkg ["@markuplint/pug-parser"]
|
|
411
|
+
parser["PugParser"]
|
|
412
|
+
end
|
|
413
|
+
|
|
414
|
+
subgraph downstream ["Downstream"]
|
|
415
|
+
mlCore["@markuplint/ml-core\n(MLASTDocument → MLDOM)"]
|
|
416
|
+
end
|
|
417
|
+
|
|
418
|
+
upstream -->|"tokenization, parsing, types"| parser
|
|
419
|
+
parser -->|"MLASTDocument"| mlCore
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
### Upstream
|
|
423
|
+
|
|
424
|
+
- **`pug-lexer`** -- Tokenizes Pug source into a token stream
|
|
425
|
+
- **`pug-parser`** -- Converts the token stream into a raw Pug AST
|
|
426
|
+
- **`@markuplint/html-parser`** -- Provides `HtmlParser` (extended by `HtmlInPugParser`) and `getNamespace()` for namespace resolution
|
|
427
|
+
- **`@markuplint/ml-ast`** -- AST type definitions used throughout the parser
|
|
428
|
+
- **`@markuplint/parser-utils`** -- Abstract `Parser` class that `PugParser` extends, plus `ParserError`, `AttrState`, `scriptParser`, and location utilities
|
|
429
|
+
|
|
430
|
+
### Downstream
|
|
431
|
+
|
|
432
|
+
- **`@markuplint/ml-core`** -- Consumes the `MLASTDocument` produced by `PugParser` to build the MLDOM
|
|
433
|
+
|
|
434
|
+
## Documentation Map
|
|
435
|
+
|
|
436
|
+
- [Maintenance Guide](docs/maintenance.md) -- Commands, recipes, and troubleshooting
|
package/CHANGELOG.md
CHANGED
|
@@ -3,13 +3,13 @@
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
|
|
5
5
|
|
|
6
|
-
## [4.6.
|
|
6
|
+
## [4.6.23](https://github.com/markuplint/markuplint/compare/@markuplint/pug-parser@4.6.22...@markuplint/pug-parser@4.6.23) (2026-02-10)
|
|
7
7
|
|
|
8
8
|
**Note:** Version bump only for package @markuplint/pug-parser
|
|
9
9
|
|
|
10
|
+
## [4.6.22](https://github.com/markuplint/markuplint/compare/@markuplint/pug-parser@4.6.21...@markuplint/pug-parser@4.6.22) (2025-11-05)
|
|
10
11
|
|
|
11
|
-
|
|
12
|
-
|
|
12
|
+
**Note:** Version bump only for package @markuplint/pug-parser
|
|
13
13
|
|
|
14
14
|
## [4.6.21](https://github.com/markuplint/markuplint/compare/@markuplint/pug-parser@4.6.20...@markuplint/pug-parser@4.6.21) (2025-08-24)
|
|
15
15
|
|
package/SKILL.md
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Maintenance tasks for @markuplint/pug-parser
|
|
3
|
+
globs:
|
|
4
|
+
- packages/@markuplint/pug-parser/src/**/*.ts
|
|
5
|
+
alwaysApply: false
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# pug-parser-maintenance
|
|
9
|
+
|
|
10
|
+
Perform maintenance tasks for `@markuplint/pug-parser`: add node type handling in nodeize(),
|
|
11
|
+
modify attribute processing in visitAttr(), and update AST optimization in pug-parser/index.ts.
|
|
12
|
+
|
|
13
|
+
## Input
|
|
14
|
+
|
|
15
|
+
`$ARGUMENTS` specifies the task. Supported tasks:
|
|
16
|
+
|
|
17
|
+
| Task | Description |
|
|
18
|
+
| ----------------------------- | ----------------------------------------------------- |
|
|
19
|
+
| `add-node-type-handling` | Add handling for a new Pug AST node type in nodeize() |
|
|
20
|
+
| `modify-attribute-processing` | Modify Pug attribute processing in visitAttr() |
|
|
21
|
+
| `update-ast-optimization` | Update AST optimization in pug-parser/index.ts |
|
|
22
|
+
|
|
23
|
+
If omitted, defaults to `add-node-type-handling`.
|
|
24
|
+
|
|
25
|
+
## Reference
|
|
26
|
+
|
|
27
|
+
Before executing any task, read `docs/maintenance.md` (or `docs/maintenance.ja.md`)
|
|
28
|
+
for the full guide. The recipes there are the source of truth for procedures.
|
|
29
|
+
|
|
30
|
+
Also read:
|
|
31
|
+
|
|
32
|
+
- `ARCHITECTURE.md` -- Package overview, parse pipeline, attribute processing, and AST optimization
|
|
33
|
+
- `src/parser.ts` -- PugParser class (source of truth for nodeize/visitAttr)
|
|
34
|
+
- `src/pug-parser/index.ts` -- AST optimization (source of truth for optimizeAST)
|
|
35
|
+
|
|
36
|
+
## Task: add-node-type-handling
|
|
37
|
+
|
|
38
|
+
Add handling for a new Pug AST node type. Follow recipe #1 in `docs/maintenance.md`.
|
|
39
|
+
|
|
40
|
+
### Step 1: Define the type
|
|
41
|
+
|
|
42
|
+
1. Read `src/types.ts` and add the new optimized AST type extending the `PugAST` namespace type with `AdditionalASTData`
|
|
43
|
+
2. Add the new type to the `ASTNode` union
|
|
44
|
+
|
|
45
|
+
### Step 2: Add optimization
|
|
46
|
+
|
|
47
|
+
1. Read `src/pug-parser/index.ts`
|
|
48
|
+
2. Add a new `case` in `optimizeAST()` for the new node type
|
|
49
|
+
3. Compute `offset`, `endOffset`, `endLine`, `endColumn`, and `raw` using the standard pattern
|
|
50
|
+
4. If the node has a `block`, recursively call `optimizeAST()` on it
|
|
51
|
+
5. If the node has attributes, call `getAttrs()` to enrich them
|
|
52
|
+
|
|
53
|
+
### Step 3: Add nodeize handling
|
|
54
|
+
|
|
55
|
+
1. Read `src/parser.ts`
|
|
56
|
+
2. Add a new `case` in `nodeize()` — decide whether to use `visitElement()`, `visitPsBlock()`, `visitComment()`, `visitText()`, or `visitDoctype()`
|
|
57
|
+
3. Most Pug-specific constructs should use `visitPsBlock()` with child nodes from `block.nodes` or `nodes`
|
|
58
|
+
|
|
59
|
+
### Step 4: Verify
|
|
60
|
+
|
|
61
|
+
1. Build: `yarn build --scope @markuplint/pug-parser`
|
|
62
|
+
2. Test: `yarn test --scope @markuplint/pug-parser`
|
|
63
|
+
3. Add test cases using `nodeListToDebugMaps` assertions
|
|
64
|
+
|
|
65
|
+
## Task: modify-attribute-processing
|
|
66
|
+
|
|
67
|
+
Modify Pug attribute processing in visitAttr(). Follow recipe #2 in `docs/maintenance.md`.
|
|
68
|
+
|
|
69
|
+
### Step 1: Understand the current processing
|
|
70
|
+
|
|
71
|
+
1. Read `src/parser.ts` — the `visitAttr()` method
|
|
72
|
+
2. Understand the three paths: shorthand (`#`/`.`), regular attributes, and value type parsing via `scriptParser()`
|
|
73
|
+
3. Read the base `Parser.visitAttr()` in `@markuplint/parser-utils` for the parent behavior
|
|
74
|
+
|
|
75
|
+
### Step 2: Make the change
|
|
76
|
+
|
|
77
|
+
1. For shorthand attributes: modify the `token.raw[0] === '#' || token.raw[0] === '.'` branch
|
|
78
|
+
2. For regular attributes: modify the options passed to `super.visitAttr()` (quoteSet, noQuoteValueType, etc.)
|
|
79
|
+
3. For value types: modify the `scriptParser()` result handling
|
|
80
|
+
4. Use `this.updateAttr()` to set `potentialName`, `potentialValue`, `isDuplicatable`, `valueType`
|
|
81
|
+
|
|
82
|
+
### Step 3: Verify
|
|
83
|
+
|
|
84
|
+
1. Build: `yarn build --scope @markuplint/pug-parser`
|
|
85
|
+
2. Test: `yarn test --scope @markuplint/pug-parser`
|
|
86
|
+
|
|
87
|
+
## Task: update-ast-optimization
|
|
88
|
+
|
|
89
|
+
Update AST optimization in pug-parser/index.ts. Follow recipe #3 in `docs/maintenance.md`.
|
|
90
|
+
|
|
91
|
+
### Step 1: Understand the optimization pipeline
|
|
92
|
+
|
|
93
|
+
1. Read `src/pug-parser/index.ts`
|
|
94
|
+
2. Understand the flow: `pugParse()` → `lexer()` → `parser()` → `optimizeAST()`
|
|
95
|
+
3. Understand the helper functions: `getOffsetsFromLines()`, `getLocationFromToken()`, `getAttrs()`, `getEndAttributeLocation()`, `mergeTextNode()`, `getPipelessText()`, `getRawTextAndLocationEnd()`
|
|
96
|
+
|
|
97
|
+
### Step 2: Make the change
|
|
98
|
+
|
|
99
|
+
1. For offset computation changes: modify `getOffsetsFromLines()` or the offset calculation in `optimizeAST()`
|
|
100
|
+
2. For attribute enrichment: modify `getAttrs()` or `getEndAttributeLocation()`
|
|
101
|
+
3. For text handling: modify `mergeTextNode()`, `getPipelessText()`, or `getRawTextAndLocationEnd()`
|
|
102
|
+
4. For conditional chains: modify `optimizeASTOfConditionalNode()`
|
|
103
|
+
5. Ensure the token matching logic in `getLocationFromToken()` is correct
|
|
104
|
+
|
|
105
|
+
### Step 3: Verify
|
|
106
|
+
|
|
107
|
+
1. Build: `yarn build --scope @markuplint/pug-parser`
|
|
108
|
+
2. Test: `yarn test --scope @markuplint/pug-parser`
|
|
109
|
+
3. Verify with `pug-parser/index.spec.ts` tests
|
|
110
|
+
|
|
111
|
+
## Rules
|
|
112
|
+
|
|
113
|
+
1. **Use pug-lexer and pug-parser for tokenization** — never parse Pug syntax manually. The `pugParse()` function is the only entry point.
|
|
114
|
+
2. **Optimize AST before converting to markuplint nodes** — all nodes must have `offset`, `endOffset`, `endLine`, `endColumn`, and `raw` computed in `optimizeAST()` before `nodeize()` processes them.
|
|
115
|
+
3. **Test with `nodeListToDebugMaps`** — this is the standard assertion pattern for parser tests.
|
|
116
|
+
4. **Use `HtmlInPugParser` for inline HTML** — never parse inline HTML manually. The `HtmlInPugParser` handles `#[...]` masking.
|
|
117
|
+
5. **Recursively parse tag interpolation** — `#ps:tag-interpolation` nodes must be stripped of `#[` / `]` and re-parsed by a new `PugParser` instance.
|
|
118
|
+
6. **Add JSDoc comments** to all new public methods and properties.
|