parseman 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +167 -126
- package/dist/combinators/choice.d.ts.map +1 -1
- package/dist/combinators/completions.d.ts +17 -0
- package/dist/combinators/completions.d.ts.map +1 -0
- package/dist/combinators/grammar.d.ts +20 -4
- package/dist/combinators/grammar.d.ts.map +1 -1
- package/dist/combinators/guard.d.ts +2 -2
- package/dist/combinators/guard.d.ts.map +1 -1
- package/dist/combinators/keywords.d.ts +22 -0
- package/dist/combinators/keywords.d.ts.map +1 -0
- package/dist/combinators/literal.d.ts.map +1 -1
- package/dist/combinators/node.d.ts +24 -0
- package/dist/combinators/node.d.ts.map +1 -0
- package/dist/combinators/parser.d.ts +7 -7
- package/dist/combinators/parser.d.ts.map +1 -1
- package/dist/combinators/probe.d.ts +13 -0
- package/dist/combinators/probe.d.ts.map +1 -0
- package/dist/combinators/recover.d.ts.map +1 -1
- package/dist/combinators/regex.d.ts.map +1 -1
- package/dist/combinators/repeat.d.ts.map +1 -1
- package/dist/combinators/scanTo.d.ts.map +1 -1
- package/dist/combinators/sequence.d.ts.map +1 -1
- package/dist/combinators/trivia-skip.d.ts +33 -0
- package/dist/combinators/trivia-skip.d.ts.map +1 -0
- package/dist/combinators/withCtx.d.ts +1 -1
- package/dist/compiler/codegen.d.ts +13 -4
- package/dist/compiler/codegen.d.ts.map +1 -1
- package/dist/cst/grammar.d.ts +33 -2
- package/dist/cst/grammar.d.ts.map +1 -1
- package/dist/cst/incremental.d.ts +4 -2
- package/dist/cst/incremental.d.ts.map +1 -1
- package/dist/cst/trivia-index.d.ts +50 -0
- package/dist/cst/trivia-index.d.ts.map +1 -0
- package/dist/cst/types.d.ts +7 -7
- package/dist/cst/types.d.ts.map +1 -1
- package/dist/functional/doc.d.ts +53 -0
- package/dist/functional/doc.d.ts.map +1 -0
- package/dist/index.cjs +953 -221
- package/dist/index.cjs.map +4 -4
- package/dist/index.d.ts +12 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +6519 -226
- package/dist/index.js.map +4 -4
- package/dist/plugin/evaluator.d.ts +26 -6
- package/dist/plugin/evaluator.d.ts.map +1 -1
- package/dist/plugin/index.cjs +1164 -215
- package/dist/plugin/index.cjs.map +4 -4
- package/dist/plugin/index.d.ts +5 -2
- package/dist/plugin/index.d.ts.map +1 -1
- package/dist/plugin/index.js +1164 -215
- package/dist/plugin/index.js.map +4 -4
- package/dist/types.d.ts +51 -6
- package/dist/types.d.ts.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -18,7 +18,9 @@ Measured on Apple M2 Pro. Bars show µs per parse — shorter is faster.
|
|
|
18
18
|
|
|
19
19
|

|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
Parséman has three modes — **interpreter** (zero setup, works anywhere), **macro build** (compiled by the bundler plugin at build time, zero runtime cost), and **`.compile()`** (optional runtime JIT). Most production use lands on one of the first two. The initialization section only shows parsers with a nonzero setup cost: `.compile()` costs 75–650 µs depending on grammar size; Chevrotain always costs 840–1,400 µs. Parsers not listed there start for free.
|
|
22
|
+
|
|
23
|
+
On JSON, CSV, and GraphQL, Parséman macro beats Peggy at every fixture size in the charts above. For CST-building grammars (`node()` rules with trivia capture), the interpreter runs ~2× faster than Chevrotain on the JSON CST fixture while building an equivalent tree.
|
|
22
24
|
|
|
23
25
|
---
|
|
24
26
|
|
|
@@ -110,7 +112,7 @@ If `with { type: 'macro' }` is stripped (older bundlers, test runners), the attr
|
|
|
110
112
|
|
|
111
113
|
### What gets compiled
|
|
112
114
|
|
|
113
|
-
|
|
115
|
+
Combinator trees — `literal`, `regex`, `sequence`, `choice`, `many`, `oneOrMore`, `optional`, `sepBy`, `transform`, `skip`, `not`, `scanTo`, `balanced` — plus `rules()` factories (including mutually recursive ones), `parser({ trivia })` wrappers, and `node()` rules (CST capture and all). A full grammar built as a `rules()` factory of `node()` rules compiles end to end: each rule becomes an independently-callable function, terminal/trivia capture is emitted inline, and every `build`/`transform` callback is inlined with its source span. Grammars with no `node()` emit zero capture code, so they compile byte-identically. Parsers that close over external variables the evaluator can't resolve stay as-is — the plugin compiles what it can and quietly leaves the rest alone.
|
|
114
116
|
|
|
115
117
|
---
|
|
116
118
|
|
|
@@ -119,6 +121,8 @@ Pure combinator trees — `literal`, `regex`, `sequence`, `choice`, `many`, `one
|
|
|
119
121
|
| Combinator | Description |
|
|
120
122
|
|---|---|
|
|
121
123
|
| `literal(s, opts?)` | Exact string match. `opts.caseInsensitive` for locale-aware comparison. |
|
|
124
|
+
| `word(s, boundary?)` | Single keyword with automatic word-boundary guard. |
|
|
125
|
+
| `wordContext(boundary?)` | Returns a keyword factory with a baked-in boundary (e.g. CSS `A-Za-z0-9_-`). |
|
|
122
126
|
| `regex(pattern)` | Match a regex at the current position. Patterns are optimized via `regexp-tree`. |
|
|
123
127
|
| `sequence(...parsers)` | Match all in order; returns a tuple `[v1, v2, ...]`. Skips trivia between terms when trivia is set. |
|
|
124
128
|
| `choice(...parsers)` | Ordered alternatives (PEG — first match wins). Disjoint first chars → O(1) dispatch. |
|
|
@@ -129,13 +133,37 @@ Pure combinator trees — `literal`, `regex`, `sequence`, `choice`, `many`, `one
|
|
|
129
133
|
| `transform(parser, fn)` | Map the result: `fn(value, span) → newValue`. |
|
|
130
134
|
| `skip(main, skipped)` | Match `main` then `skipped`; return `main`'s value. |
|
|
131
135
|
| `rules(factory)` | Named grammar rules — no forward declarations needed, handles mutual recursion. |
|
|
136
|
+
| `node(type, parser, build)` | CST/AST rule: captures `parser`'s terminals into `children`/`rawChildren` and trivia offsets into a flat `triviaLog`, then calls `build(children, rawChildren, span, triviaLog)`. |
|
|
132
137
|
| `ref<T>()` | Low-level forward declaration slot (use `rules()` in most cases). |
|
|
133
138
|
| `not(parser)` | Negative lookahead — succeeds (consuming nothing) when `parser` fails. |
|
|
134
139
|
| `guard(predicate)` | Succeeds only when `predicate(ctx)` returns true; used for context-sensitive rules. |
|
|
135
140
|
| `withCtx(extra, parser)` | Merge `extra` into the user context for the duration of `parser`. |
|
|
136
141
|
| `recover(parser, sentinel)` | On failure, skip input until `sentinel` matches; returns a `CSTError` node. |
|
|
137
|
-
| `scanTo(sentinel,
|
|
138
|
-
| `balanced(open, close)` | Match a balanced
|
|
142
|
+
| `scanTo(sentinel, opts?)` | Scan forward until `sentinel` matches (sentinel not consumed). Pass `opts.skip` to treat certain patterns as opaque blobs that may contain the sentinel character. Pass `opts.orEOF: true` to succeed at end-of-input. |
|
|
143
|
+
| `balanced(open, close, opts?)` | Match a single balanced delimited region — e.g. `(…)` or `[…]` — including the delimiters. Primarily used as an element of `scanTo`'s `opts.skip` list. |
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Performance: collapse opaque shapes into one regex
|
|
148
|
+
|
|
149
|
+
The single biggest grammar-level perf lever is **the number of combinator boundaries on the hot path**. Every `sequence` / `regex` / `oneOrMore` step is a function call plus a result-object allocation plus — in a `node()` rule — a leaf push. Fewer, fatter combinators beat many thin ones.
|
|
150
|
+
|
|
151
|
+
Measured on a repeated 3-shape group (`name1 1px #111 …`, ~29 KB), parsing the same content three ways:
|
|
152
|
+
|
|
153
|
+
| Approach | Interpreted | Compiled |
|
|
154
|
+
|---|---|---|
|
|
155
|
+
| `oneOrMore(sequence(ident, sp, num, sp, hex, sp))` | 0.289 ms | 0.167 ms |
|
|
156
|
+
| same, with **inline** `regex(…)` instead of shared refs | 0.281 ms | 0.183 ms |
|
|
157
|
+
| **one** `regex(/…ident…num…hex…/)` per group | **0.055 ms** | **0.042 ms** |
|
|
158
|
+
|
|
159
|
+
Two takeaways:
|
|
160
|
+
|
|
161
|
+
- **Shared combinator ref vs inline `regex(…)` literal makes no difference.** Both produce the identical runtime structure (one `regex` combinator either way), and the compiler inlines a `regex` test at every use site regardless of sharing — there's no function-call indirection to eliminate (that only exists for `ref()` / `rules()` entries). Factor out shared terminals for readability; it costs nothing.
|
|
162
|
+
- **Collapsing a fixed multi-token shape into a single `regex` is 4–5× faster** in both the interpreter and compiled output, because it erases the per-step call + allocation overhead. `compile()` is a real but smaller win (~1.7×) and **stacks** with collapsing.
|
|
163
|
+
|
|
164
|
+
**When to collapse:** only where the CST treats the group as opaque text — a dimension `\d+px`, a hex color, an `nth` expression, a simple ident-run. A single regex yields **one leaf**, not structured sub-nodes.
|
|
165
|
+
|
|
166
|
+
**When NOT to collapse:** keep the parts as separate combinators wherever the builder needs them as distinct CST children — for per-field source spans (`fieldSpans`/`valueSpans`), for trivia recovered *between* the parts, or for distinct typed nodes. Correctness first; collapse only the genuinely opaque runs.
|
|
139
167
|
|
|
140
168
|
---
|
|
141
169
|
|
|
@@ -168,7 +196,20 @@ const ws = trivia(many(choice(regex(/\s+/), lineComment, blockComment)
|
|
|
168
196
|
|
|
169
197
|
`choice()` uses PEG ordered-choice semantics: first match wins. **Order matters.**
|
|
170
198
|
|
|
171
|
-
For keywords — where `if` should not match the prefix of `ifdef` — use `
|
|
199
|
+
For keywords — where `if` should not match the prefix of `ifdef` — use `word()` or `wordContext()`:
|
|
200
|
+
|
|
201
|
+
```ts
|
|
202
|
+
const kw = wordContext() // bakes in a default identifier boundary once
|
|
203
|
+
|
|
204
|
+
const token = choice(
|
|
205
|
+
kw('if'),
|
|
206
|
+
kw('else'),
|
|
207
|
+
kw('return'),
|
|
208
|
+
regex(/[a-zA-Z_]\w*/), // ident fallback
|
|
209
|
+
)
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Or build the boundary guard by hand with `not()`:
|
|
172
213
|
|
|
173
214
|
```ts
|
|
174
215
|
const wordChar = regex(/\w/)
|
|
@@ -231,7 +272,9 @@ jsonParser.parse('{ "a": 1 }')
|
|
|
231
272
|
|
|
232
273
|
`g.value` is a reference that works anywhere inside the factory regardless of order. Local helpers (`comma`, `pair`, `object`) that don't need to be cross-referenced can be plain `const`. Only put a rule in the returned object if other rules need to reach it as `g.xxx`.
|
|
233
274
|
|
|
234
|
-
> **Macro and `rules()`:** The plugin fully compiles `rules()` factories, including recursive ones. It emits mutually recursive named functions (`_pf0` etc.) so the cycle is broken. Add `with { type: 'macro' }` to your import and the entire grammar — recursive rules included — is inlined at build time.
|
|
275
|
+
> **Macro and `rules()`:** The plugin fully compiles `rules()` factories, including recursive ones. It emits mutually recursive named functions (`_pf0` etc.) so the cycle is broken. Add `with { type: 'macro' }` to your import and the entire grammar — recursive rules included — is inlined at build time. Both binding forms compile: `const { value } = rules(...)` (each rule becomes a top-level function) and `const grammar = rules(...)` (an object literal of compiled rules, so `grammar.value(...)` works).
|
|
276
|
+
>
|
|
277
|
+
> If the plugin meets a macro-imported declaration it can't compile statically (it closes over a runtime value, or isn't a recognized combinator shape), it leaves that declaration for the interpreter, strips the `with { type: 'macro' }` attribute so the import stays valid, and emits a build **warning** (`[parseman] file:line — …`) pointing at it — so a silent fallback never goes unnoticed.
|
|
235
278
|
|
|
236
279
|
### `ref<T>()` — low-level forward declaration
|
|
237
280
|
|
|
@@ -245,127 +288,66 @@ value.define(choice(object, array, str, num, bool, nil))
|
|
|
245
288
|
|
|
246
289
|
---
|
|
247
290
|
|
|
248
|
-
##
|
|
291
|
+
## Grammars that build an AST
|
|
249
292
|
|
|
250
|
-
For grammars that
|
|
293
|
+
For grammars that produce a typed CST/AST, support incremental re-parsing, or care about trivia, wrap each rule in `node(type, parser, build)`. parseman captures the rule's terminals into `children` / `rawChildren` and records trivia as flat `[start, end, insertIdx, …]` triples in `triviaLog`, then hands all four to `build(children, rawChildren, span, triviaLog)`. **Capture is the library's job:** you don't wrap terminals to recover their spans, and you don't reconstruct trivia — it's collected as the parser runs, in both the interpreter and the compiled build. Use `buildTriviaIndex(tree, input)` to turn `triviaLog` into a before/after lookup table.
|
|
251
294
|
|
|
252
295
|
```ts
|
|
253
|
-
import {
|
|
254
|
-
import type {
|
|
255
|
-
|
|
256
|
-
class ExprParser extends Parser {
|
|
257
|
-
ws = regex(/\s*/)
|
|
258
|
-
digits = regex(/[0-9]+/)
|
|
259
|
-
ident = regex(/[a-zA-Z_]\w*/)
|
|
260
|
-
|
|
261
|
-
// Plain initializer when no cross-reference needed
|
|
262
|
-
Str = sequence(literal('"'), regex(/[^"]*/), literal('"'))
|
|
263
|
-
|
|
264
|
-
// Thunk form for forward / mutual references
|
|
265
|
-
Num = (g: Refs<ExprParser>) => g.digits
|
|
266
|
-
Id = (g: Refs<ExprParser>) => g.ident
|
|
267
|
-
Add = (g: Refs<ExprParser>) => sequence(g.Num, many(sequence(literal('+'), g.Num)))
|
|
268
|
-
Expr = (g: Refs<ExprParser>) => choice(g.Add, g.Num, g.Id)
|
|
269
|
-
}
|
|
270
|
-
|
|
271
|
-
const expr = new ExprParser()
|
|
272
|
-
const r = parse(expr.rule('Expr'), '1+2+3')
|
|
273
|
-
// r.value is a CSTNode { _tag: 'node', type: 'Expr', span, children, savedContext }
|
|
274
|
-
```
|
|
275
|
-
|
|
276
|
-
### Inheritance
|
|
277
|
-
|
|
278
|
-
Override any rule by redeclaring it in a subclass — subclass initializers run after the parent's, so the override wins automatically:
|
|
279
|
-
|
|
280
|
-
```ts
|
|
281
|
-
class JSXParser extends ExprParser {
|
|
282
|
-
// replace just the ident rule; everything else stays
|
|
283
|
-
ident = regex(/[a-zA-Z_$][\w$]*/)
|
|
284
|
-
}
|
|
285
|
-
```
|
|
286
|
-
|
|
287
|
-
### Custom AST nodes (`buildNode`)
|
|
288
|
-
|
|
289
|
-
Override `buildNode` to return your own node type instead of the default `CSTNode`:
|
|
290
|
-
|
|
291
|
-
```ts
|
|
292
|
-
import type { CSTLeaf, CSTError, CSTRawChild, Span } from 'parseman'
|
|
296
|
+
import { rules, parser, node, regex, literal, sequence, many, trivia } from 'parseman'
|
|
297
|
+
import type { Combinator } from 'parseman'
|
|
293
298
|
|
|
294
|
-
|
|
299
|
+
// Any node shape works as long as it satisfies NodeLike (see below).
|
|
300
|
+
type N = { _tag: 'node'; type: string; span: { start: number; end: number }; state: unknown; children: unknown[] }
|
|
301
|
+
const ws = trivia(regex(/\s+/))
|
|
295
302
|
|
|
296
|
-
|
|
297
|
-
|
|
303
|
+
export const { Expr, Num } = rules<{ Expr: Combinator<N>; Num: Combinator<N> }>(g => {
|
|
304
|
+
const num = node('Num', regex(/[0-9]+/),
|
|
305
|
+
(children, raw, span, _triviaLog) => ({ _tag: 'node', type: 'Num', span, state: null, children: [...children] }))
|
|
306
|
+
const expr = node('Expr', parser({ trivia: ws }, sequence(g.Num, many(sequence(literal('+'), g.Num)))),
|
|
307
|
+
(children, raw, span, _triviaLog) => ({ _tag: 'node', type: 'Expr', span, state: null, children: [...children] }))
|
|
308
|
+
return { Expr: expr, Num: num }
|
|
309
|
+
})
|
|
298
310
|
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
span: Span,
|
|
302
|
-
children: ReadonlyArray<MyNode | CSTLeaf | CSTError>,
|
|
303
|
-
savedContext: unknown,
|
|
304
|
-
rawChildren: ReadonlyArray<CSTRawChild>,
|
|
305
|
-
): MyNode {
|
|
306
|
-
return { _tag: 'node', type, span, savedContext, children: children as MyNode[], text: '...' }
|
|
307
|
-
}
|
|
308
|
-
}
|
|
311
|
+
Expr.parse('1 + 2 + 3', 0, { trackLines: false })
|
|
312
|
+
// value is a Node whose children are the captured Num sub-nodes and '+' leaves
|
|
309
313
|
```
|
|
310
314
|
|
|
311
|
-
|
|
315
|
+
- **`children`** — structural items in source order: spanned `CSTLeaf` terminals (`{ _tag:'leaf', value, span }`) and sub-nodes (whatever a nested `node()`'s `build` returned). A `build` that returns a bare string is recorded by the parent as a spanned leaf, so single-item "collapsing" rules keep their source span.
|
|
316
|
+
- **`rawChildren`** — structural children only (same items as `children`, without trivia tokens).
|
|
317
|
+
- **`triviaLog`** — flat `[start, end, insertIdx, …]` triples for whitespace/comments consumed between terms. `insertIdx` is the `rawChildren` index before which the trivia was consumed. Pass the tree to `buildTriviaIndex(tree, input)` for a `before`/`after` map of trivia tokens — useful for whitespace-sensitive syntax (e.g. CSS `div p` vs `div.p`).
|
|
312
318
|
|
|
313
|
-
|
|
319
|
+
Each rule returned from the factory is independently callable — `Expr`, `Num` above are the **rule registry** incremental re-parsing needs. Wrap a rule's inner parser in `parser({ trivia })` so trivia-skipping is baked in regardless of which rule you start from; the macro compiles the wrapper (and all capture) away to flat JS.
|
|
314
320
|
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
```ts
|
|
318
|
-
import type { CSTTrivia } from 'parseman'
|
|
319
|
-
|
|
320
|
-
class CssParser extends Parser<SelectorNode> {
|
|
321
|
-
ident = regex(/[a-zA-Z-]+/)
|
|
322
|
-
Selector = (g: Refs<CssParser>) => sequence(g.ident, g.ident)
|
|
323
|
-
|
|
324
|
-
protected buildNode(type, span, children, savedContext, rawChildren) {
|
|
325
|
-
if (type === 'Selector') {
|
|
326
|
-
// rawChildren: [Ident("div"), CSTTrivia(" "), Ident("p")]
|
|
327
|
-
const hasDescendant = rawChildren.some(c => c._tag === 'trivia')
|
|
328
|
-
}
|
|
329
|
-
return ...
|
|
330
|
-
}
|
|
331
|
-
}
|
|
332
|
-
|
|
333
|
-
// Trivia is set on parse() — the whitespace skip happens globally
|
|
334
|
-
parse(css.rule('Stylesheet'), src, { trivia: many(choice(regex(/\s+/), comment)) })
|
|
335
|
-
```
|
|
336
|
-
|
|
337
|
-
`CSTTrivia` nodes only appear in `rawChildren`, never in `children`. Zero-length trivia matches (e.g. `\s*` at a non-whitespace position) are not emitted.
|
|
321
|
+
> `transform(p, fn)` is still the tool for plain value-mapping (no children/trivia). `node()` is for CST/AST rules — it adds the capture `transform` doesn't. Both compile under the macro.
|
|
338
322
|
|
|
339
323
|
### Incremental re-parsing
|
|
340
324
|
|
|
341
|
-
`
|
|
325
|
+
`makeFunctionalDoc(registry, rootRule, input, opts?)` wraps a parse in a document that re-parses incrementally on edits. The `registry` is the object `rules()` returns (rule name → parser fn); the parser functions stay stateless, all incremental state lives in the doc.
|
|
342
326
|
|
|
343
327
|
```ts
|
|
344
|
-
|
|
328
|
+
import { makeFunctionalDoc } from 'parseman'
|
|
345
329
|
|
|
346
|
-
const
|
|
347
|
-
doc
|
|
330
|
+
const registry = { Expr, Num } // straight from rules()
|
|
331
|
+
let doc = makeFunctionalDoc(registry, 'Expr', src)
|
|
332
|
+
doc.tree // your Node root, or null on failure
|
|
348
333
|
doc.errors // ParseFail[], empty on success
|
|
349
334
|
doc.input // the source string
|
|
350
335
|
|
|
351
|
-
//
|
|
352
|
-
//
|
|
353
|
-
|
|
336
|
+
// edit(from, to, replacement) — two byte offsets into the OLD text + the
|
|
337
|
+
// replacement. "Select from→to, type replacement" — the three things every
|
|
338
|
+
// editor knows on each keystroke. Returns a new doc; the old one is untouched.
|
|
339
|
+
doc = doc.edit(changeStart, changeStart + changeLength, newText)
|
|
354
340
|
```
|
|
355
341
|
|
|
356
|
-
`edit(
|
|
342
|
+
`edit()` finds the smallest node containing the change, re-parses just that rule from its start offset using the node's saved `state`, and grafts the result back in when the new span end lands where the edit's delta predicts. Nodes unaffected by the edit are shared by reference between old and new docs. For class-instance ASTs that can't be shallow-spread, pass `opts.rebuild(node, children)` to control how a parent is reconstructed when a child is replaced.
|
|
357
343
|
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
**In an IDE extension**, hold one parser instance per language, one `ParseDoc` per open document. On each keystroke your editor gives you the changed range as byte offsets — pass those straight to `edit()`:
|
|
344
|
+
**In an IDE extension**, keep one registry per language and one doc per open document. Each keystroke gives you the changed range as byte offsets — pass them straight to `edit()`:
|
|
361
345
|
|
|
362
346
|
```ts
|
|
363
|
-
|
|
364
|
-
const parser = new CSSParser()
|
|
365
|
-
const docs = new Map<string, ParseDoc<CSTNode>>()
|
|
347
|
+
const docs = new Map<string, ReturnType<typeof makeFunctionalDoc<Node>>>()
|
|
366
348
|
|
|
367
|
-
vscode.workspace.onDidOpenTextDocument(
|
|
368
|
-
docs.set(
|
|
349
|
+
vscode.workspace.onDidOpenTextDocument(d => {
|
|
350
|
+
docs.set(d.uri.toString(), makeFunctionalDoc(registry, 'Stylesheet', d.getText()))
|
|
369
351
|
})
|
|
370
352
|
|
|
371
353
|
vscode.workspace.onDidChangeTextDocument(event => {
|
|
@@ -375,7 +357,23 @@ vscode.workspace.onDidChangeTextDocument(event => {
|
|
|
375
357
|
doc = doc.edit(change.rangeOffset, change.rangeOffset + change.rangeLength, change.text)
|
|
376
358
|
}
|
|
377
359
|
docs.set(uri, doc)
|
|
378
|
-
// walk doc.tree
|
|
360
|
+
// walk doc.tree for diagnostics, folding ranges, semantic tokens, etc.
|
|
361
|
+
})
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
### Inheritance — by composition
|
|
365
|
+
|
|
366
|
+
Share rules across grammar variants by composition: factor common rules into helpers, and let each variant swap the ones it needs. Because every rule is a plain value, a variant is just different pieces passed into the same `rules()` shape — and it all stays macro-compilable.
|
|
367
|
+
|
|
368
|
+
```ts
|
|
369
|
+
// shared building blocks
|
|
370
|
+
const makeIdent = () => regex(/[a-zA-Z_]\w*/)
|
|
371
|
+
|
|
372
|
+
export const { Expr } = rules<{ Expr: Combinator<Node> }>(g => {
|
|
373
|
+
const ident = makeIdent() // base
|
|
374
|
+
// a JSX variant would use regex(/[a-zA-Z_$][\w$]*/) here instead
|
|
375
|
+
/* ...rules referencing ident... */
|
|
376
|
+
return { Expr: /* ... */ }
|
|
379
377
|
})
|
|
380
378
|
```
|
|
381
379
|
|
|
@@ -383,28 +381,26 @@ vscode.workspace.onDidChangeTextDocument(event => {
|
|
|
383
381
|
|
|
384
382
|
## Context-sensitive parsing
|
|
385
383
|
|
|
386
|
-
`withCtx` and `guard` implement context-sensitive rules without mutating shared state.
|
|
384
|
+
`withCtx` and `guard` implement context-sensitive rules without mutating shared state, and they compose into `rules()` like any other combinator.
|
|
387
385
|
|
|
388
|
-
`withCtx(extra, parser)` merges `extra` into the user context for the duration of `parser`. `guard(predicate)` succeeds only when `predicate(ctx)` returns true,
|
|
386
|
+
`withCtx(extra, parser)` merges `extra` into the user context for the duration of `parser`. `guard(predicate)` succeeds only when `predicate(ctx)` returns true, gating a rule behind runtime context.
|
|
389
387
|
|
|
390
388
|
```ts
|
|
391
|
-
import { withCtx, guard, many, sequence, choice, literal, regex } from 'parseman'
|
|
389
|
+
import { rules, withCtx, guard, many, sequence, choice, literal, regex, trivia, parser } from 'parseman'
|
|
390
|
+
import type { Combinator } from 'parseman'
|
|
392
391
|
|
|
393
|
-
|
|
394
|
-
ws = regex(/\s*/)
|
|
392
|
+
const ws = trivia(regex(/\s*/))
|
|
395
393
|
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
guard((ctx: { inFn?: boolean }) => ctx.inFn === true),
|
|
399
|
-
|
|
400
|
-
)
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
Program = (g: Refs<LangParser>) => many(g.Body)
|
|
404
|
-
}
|
|
394
|
+
export const { Program } = rules<{ Program: Combinator<unknown> }>(g => {
|
|
395
|
+
const expr = regex(/[a-z]+/)
|
|
396
|
+
const ret = sequence(guard((ctx: { inFn?: boolean }) => ctx.inFn === true), literal('return'))
|
|
397
|
+
const stmt = choice(ret, expr)
|
|
398
|
+
const body = withCtx({ inFn: true }, many(sequence(stmt, ws)))
|
|
399
|
+
return { Program: parser({ trivia: ws }, many(body)) }
|
|
400
|
+
})
|
|
405
401
|
```
|
|
406
402
|
|
|
407
|
-
`
|
|
403
|
+
`return` is only reachable inside a body because `guard` rejects it when `inFn` is not set. Incremental `edit()` replays the correct context because each node records the `ctx.state` snapshot (`node.state`) active when it was parsed.
|
|
408
404
|
|
|
409
405
|
---
|
|
410
406
|
|
|
@@ -422,7 +418,35 @@ const stmt = recover(g.Stmt, literal(';'))
|
|
|
422
418
|
const block = scanTo(literal('}'), [balanced(literal('('), literal(')')), balanced(literal('['), literal(']'))])
|
|
423
419
|
```
|
|
424
420
|
|
|
425
|
-
`scanTo
|
|
421
|
+
### `scanTo` vs `balanced` — when to use each
|
|
422
|
+
|
|
423
|
+
Both are **position arithmetic**: pure cursor-advance with zero CST allocation. Neither pushes terminals into the enclosing `node()`'s child list; only the final scanned span appears as a single leaf.
|
|
424
|
+
|
|
425
|
+
**`scanTo(sentinel, opts?)`** — use when you need to consume an open-ended region whose boundary is a specific token. It scans character-by-character, stopping the moment `sentinel` matches. Pass `opts.skip` to declare patterns that should be treated as opaque blobs so their content never accidentally looks like the sentinel:
|
|
426
|
+
|
|
427
|
+
```ts
|
|
428
|
+
// Consume a CSS at-rule prelude up to '{' or ';',
|
|
429
|
+
// but don't stop inside parentheses or strings.
|
|
430
|
+
const prelude = scanTo(choice(literal('{'), literal(';')), {
|
|
431
|
+
skip: [balanced('(', ')'), singleStr, doubleStr],
|
|
432
|
+
})
|
|
433
|
+
```
|
|
434
|
+
|
|
435
|
+
**`balanced(open, close, opts?)`** — use when you want to match a single self-contained delimited region and get its full text back (delimiters included). This is the natural building block for `scanTo`'s `skip` list:
|
|
436
|
+
|
|
437
|
+
```ts
|
|
438
|
+
const parenGroup = balanced('(', ')') // matches (a + b) including the parens
|
|
439
|
+
const bracketExpr = balanced('[', ']') // matches [0] including the brackets
|
|
440
|
+
```
|
|
441
|
+
|
|
442
|
+
`balanced` can itself accept a `skip` list for deeply nested structures:
|
|
443
|
+
|
|
444
|
+
```ts
|
|
445
|
+
// Match (…) allowing strings inside to contain unbalanced parens
|
|
446
|
+
const parenWithStrings = balanced('(', ')', { skip: [singleStr, doubleStr] })
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
**The key difference**: `scanTo` scans *until* a boundary; `balanced` matches *across* a known delimited region. Use `balanced` inside `scanTo`'s `skip` to prevent the scanner from stopping at a sentinel that appears inside a nested structure.
|
|
426
450
|
|
|
427
451
|
---
|
|
428
452
|
|
|
@@ -475,18 +499,35 @@ type Span = {
|
|
|
475
499
|
}
|
|
476
500
|
```
|
|
477
501
|
|
|
478
|
-
###
|
|
502
|
+
### Node and document types
|
|
503
|
+
|
|
504
|
+
Any AST your `transform` callbacks produce participates in incremental re-parsing as long as it satisfies `NodeLike` — that's the whole contract:
|
|
479
505
|
|
|
480
506
|
```ts
|
|
481
|
-
type
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
507
|
+
type NodeLike = {
|
|
508
|
+
readonly _tag: 'node'
|
|
509
|
+
readonly type: string // the rule name — used as the registry key on re-parse
|
|
510
|
+
readonly span: Span
|
|
511
|
+
readonly state: unknown // ctx.state snapshot at parse time; replayed on edit
|
|
512
|
+
readonly children: ReadonlyArray<{ readonly _tag: string }>
|
|
513
|
+
}
|
|
485
514
|
|
|
486
|
-
|
|
487
|
-
type
|
|
515
|
+
// makeFunctionalDoc<N>(registry, rootRule, input, opts?) → FunctionalDoc<N>
|
|
516
|
+
type Registry<N> = Record<string, (input: string, pos: number, ctx: ParseContext) => ParseResult<N>>
|
|
517
|
+
interface FunctionalDoc<N extends NodeLike> {
|
|
518
|
+
readonly tree: N | null
|
|
519
|
+
readonly errors: ParseFail[]
|
|
520
|
+
readonly input: string
|
|
521
|
+
edit(from: number, to: number, replacement: string): FunctionalDoc<N>
|
|
522
|
+
}
|
|
523
|
+
type FunctionalDocOptions<N> = {
|
|
524
|
+
state?: unknown // initial ctx.state for the root parse
|
|
525
|
+
rebuild?: (node: N, children: ReadonlyArray<unknown>) => N // override for class-instance ASTs
|
|
526
|
+
}
|
|
488
527
|
```
|
|
489
528
|
|
|
529
|
+
`children` only needs items carrying a `_tag` so traversal can tell sub-nodes (`_tag: 'node'`) from anything else. The `type` string must match the rule name in the registry so `edit()` can re-parse the right rule.
|
|
530
|
+
|
|
490
531
|
---
|
|
491
532
|
|
|
492
533
|
## Developing
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"choice.d.ts","sourceRoot":"","sources":["../../src/combinators/choice.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,UAAU,EAC+B,QAAQ,EAClD,MAAM,aAAa,CAAA;AAGpB,KAAK,SAAS,CAAC,CAAC,IAAI,CAAC,SAAS,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,GAAG,CAAC,SAAS,UAAU,CAAC,MAAM,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,GAAG,KAAK,CAAA;AACvH,KAAK,SAAS,CAAC,CAAC,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC,EAAE,IAAI;KACrE,CAAC,IAAI,MAAM,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;CAChC,CAAC,MAAM,CAAC,SAAS,UAAU,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,KAAK,CAAA;AAEjD,wBAAgB,MAAM,CAAC,CAAC,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,EAAE,GAAG,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,EACxH,GAAG,IAAI,EAAE,CAAC,GACT,UAAU,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,
|
|
1
|
+
{"version":3,"file":"choice.d.ts","sourceRoot":"","sources":["../../src/combinators/choice.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,UAAU,EAC+B,QAAQ,EAClD,MAAM,aAAa,CAAA;AAGpB,KAAK,SAAS,CAAC,CAAC,IAAI,CAAC,SAAS,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,GAAG,CAAC,SAAS,UAAU,CAAC,MAAM,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,GAAG,KAAK,CAAA;AACvH,KAAK,SAAS,CAAC,CAAC,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC,EAAE,IAAI;KACrE,CAAC,IAAI,MAAM,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;CAChC,CAAC,MAAM,CAAC,SAAS,UAAU,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,KAAK,CAAA;AAEjD,wBAAgB,MAAM,CAAC,CAAC,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,EAAE,GAAG,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,EACxH,GAAG,IAAI,EAAE,CAAC,GACT,UAAU,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAiI1B;AAwFD,uEAAuE;AACvE,wBAAgB,mBAAmB,CAAC,CAAC,EAAE,UAAU,CAAC,OAAO,CAAC,GAAG,MAAM,GAAG,IAAI,CAKzE;AAED,qEAAqE;AACrE,wBAAgB,eAAe,CAAC,CAAC,EAAE,UAAU,CAAC,OAAO,CAAC,GAAG;IAAE,MAAM,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAA;CAAE,GAAG,IAAI,CAKhG"}
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
import type { Combinator } from '../types.ts';
|
|
2
|
+
/**
|
|
3
|
+
* Returns the set of expected tokens at the given cursor offset.
|
|
4
|
+
* Useful for implementing IDE completions.
|
|
5
|
+
*
|
|
6
|
+
* Runs the parser on input truncated at `offset` with a probe that tracks the
|
|
7
|
+
* highest-position failure seen, even when sepBy/many backtrack past the cursor.
|
|
8
|
+
* Returns the expected tokens from that deepest failure.
|
|
9
|
+
*
|
|
10
|
+
* Returns an empty array when the input up to `offset` parses completely
|
|
11
|
+
* with no failures at or before the cursor.
|
|
12
|
+
*
|
|
13
|
+
* The returned strings use the same labels as ParseFail.expected:
|
|
14
|
+
* quoted literals like `"\"{\""` and regex patterns like `"/[0-9]+/"`.
|
|
15
|
+
*/
|
|
16
|
+
export declare function completionsAt(parser: Combinator<unknown>, input: string, offset: number): string[];
|
|
17
|
+
//# sourceMappingURL=completions.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"completions.d.ts","sourceRoot":"","sources":["../../src/combinators/completions.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAA2B,MAAM,aAAa,CAAA;AAEtE;;;;;;;;;;;;;GAaG;AACH,wBAAgB,aAAa,CAC3B,MAAM,EAAE,UAAU,CAAC,OAAO,CAAC,EAC3B,KAAK,EAAE,MAAM,EACb,MAAM,EAAE,MAAM,GACb,MAAM,EAAE,CAYV"}
|
|
@@ -1,8 +1,24 @@
|
|
|
1
|
-
import type { Combinator, ParseResult } from '../types.ts';
|
|
1
|
+
import type { Combinator, ParseContext, ParseResult } from '../types.ts';
|
|
2
2
|
export type ParseOptions = {
|
|
3
|
-
trivia?: Combinator<unknown>;
|
|
4
3
|
trackLines?: boolean;
|
|
4
|
+
/**
|
|
5
|
+
* Enable error recovery. When true, recover() nodes collect their ParseErrors
|
|
6
|
+
* into a side-channel array rather than (only) embedding them in the value tree.
|
|
7
|
+
* The returned ParseOk will have an `errors` field listing all recovered errors.
|
|
8
|
+
* Top-level parse failures (where no recover() node caught the error) still
|
|
9
|
+
* return ParseFail as usual.
|
|
10
|
+
*/
|
|
11
|
+
recover?: boolean;
|
|
12
|
+
};
|
|
13
|
+
export type ParserOptions = ParseOptions & {
|
|
14
|
+
trivia?: Combinator<unknown>;
|
|
15
|
+
/** Record consumed trivia as CSTTrivia tokens in rawChildren. Default: skip. */
|
|
16
|
+
captureTrivia?: boolean;
|
|
5
17
|
};
|
|
6
|
-
export
|
|
7
|
-
|
|
18
|
+
export interface ParsemanParser<T> extends Combinator<T> {
|
|
19
|
+
parse(input: string): ParseResult<T>;
|
|
20
|
+
parse(input: string, pos: number, ctx: ParseContext): ParseResult<T>;
|
|
21
|
+
}
|
|
22
|
+
export declare function parser<T>(opts: ParserOptions, root: Combinator<T>): ParsemanParser<T>;
|
|
23
|
+
export declare function parse<T>(combinator: Combinator<T>, input: string, opts?: ParseOptions): ParseResult<T>;
|
|
8
24
|
//# sourceMappingURL=grammar.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"grammar.d.ts","sourceRoot":"","sources":["../../src/combinators/grammar.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,
|
|
1
|
+
{"version":3,"file":"grammar.d.ts","sourceRoot":"","sources":["../../src/combinators/grammar.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,aAAa,CAAA;AAGxE,MAAM,MAAM,YAAY,GAAG;IACzB,UAAU,CAAC,EAAE,OAAO,CAAA;IACpB;;;;;;OAMG;IACH,OAAO,CAAC,EAAE,OAAO,CAAA;CAClB,CAAA;AAED,MAAM,MAAM,aAAa,GAAG,YAAY,GAAG;IACzC,MAAM,CAAC,EAAE,UAAU,CAAC,OAAO,CAAC,CAAA;IAC5B,gFAAgF;IAChF,aAAa,CAAC,EAAE,OAAO,CAAA;CACxB,CAAA;AAED,MAAM,WAAW,cAAc,CAAC,CAAC,CAAE,SAAQ,UAAU,CAAC,CAAC,CAAC;IACtD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IACpC,KAAK,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,YAAY,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;CACrE;AAED,wBAAgB,MAAM,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CA6BrF;AAED,wBAAgB,KAAK,CAAC,CAAC,EACrB,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EACzB,KAAK,EAAE,MAAM,EACb,IAAI,GAAE,YAAiB,GACtB,WAAW,CAAC,CAAC,CAAC,CAehB"}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type { Combinator } from '../types.ts';
|
|
2
2
|
/**
|
|
3
3
|
* Zero-width assertion: succeeds (consuming nothing) only when `predicate`
|
|
4
|
-
* returns true for `ctx.
|
|
4
|
+
* returns true for `ctx.state`. Fails otherwise.
|
|
5
5
|
*
|
|
6
6
|
* Intended for use inside sequence() to gate subsequent parsing on runtime
|
|
7
7
|
* context set with withCtx().
|
|
@@ -11,5 +11,5 @@ import type { Combinator } from '../types.ts';
|
|
|
11
11
|
* literal('return'), optional(expr)
|
|
12
12
|
* )
|
|
13
13
|
*/
|
|
14
|
-
export declare function guard(predicate: (
|
|
14
|
+
export declare function guard(predicate: (state: unknown) => boolean): Combinator<null>;
|
|
15
15
|
//# sourceMappingURL=guard.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"guard.d.ts","sourceRoot":"","sources":["../../src/combinators/guard.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;AAEpF;;;;;;;;;;;GAWG;AACH,wBAAgB,KAAK,CAAC,SAAS,EAAE,CAAC,
|
|
1
|
+
{"version":3,"file":"guard.d.ts","sourceRoot":"","sources":["../../src/combinators/guard.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;AAEpF;;;;;;;;;;;GAWG;AACH,wBAAgB,KAAK,CAAC,SAAS,EAAE,CAAC,KAAK,EAAE,OAAO,KAAK,OAAO,GAAG,UAAU,CAAC,IAAI,CAAC,CAgB9E"}
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
import type { Combinator } from '../types.ts';
|
|
2
|
+
export type KeywordsOptions = {
|
|
3
|
+
/** Match case-insensitively. */
|
|
4
|
+
caseInsensitive?: boolean;
|
|
5
|
+
/**
|
|
6
|
+
* Character class (regex source, e.g. `'A-Za-z0-9_-'`) that must NOT follow a
|
|
7
|
+
* match — a word boundary. Prevents matching `red` inside `redish`.
|
|
8
|
+
*/
|
|
9
|
+
boundary?: string;
|
|
10
|
+
};
|
|
11
|
+
/**
|
|
12
|
+
* Match any one of a set of fixed keywords, longest-first (so `border` wins over
|
|
13
|
+
* `bord`), with an optional trailing word-boundary guard. Compiles to a single
|
|
14
|
+
* sticky regex — far less error-prone than a hand-maintained alternation, and
|
|
15
|
+
* the common case (keyword sets: CSS colors, units, at-rule names, HTML tags).
|
|
16
|
+
*
|
|
17
|
+
* keywords(CSS_COLOR_NAMES, { caseInsensitive: true, boundary: 'A-Za-z0-9_-' })
|
|
18
|
+
*
|
|
19
|
+
* On success the matched text is recorded as a CSTLeaf (like literal()/regex()).
|
|
20
|
+
*/
|
|
21
|
+
export declare function keywords(words: readonly string[], opts?: KeywordsOptions): Combinator<string>;
|
|
22
|
+
//# sourceMappingURL=keywords.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"keywords.d.ts","sourceRoot":"","sources":["../../src/combinators/keywords.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAmD,MAAM,aAAa,CAAA;AAI9F,MAAM,MAAM,eAAe,GAAG;IAC5B,gCAAgC;IAChC,eAAe,CAAC,EAAE,OAAO,CAAA;IACzB;;;OAGG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAA;CAClB,CAAA;AAMD;;;;;;;;;GASG;AACH,wBAAgB,QAAQ,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,EAAE,IAAI,GAAE,eAAoB,GAAG,UAAU,CAAC,MAAM,CAAC,CA4CjG"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"literal.d.ts","sourceRoot":"","sources":["../../src/combinators/literal.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;
|
|
1
|
+
{"version":3,"file":"literal.d.ts","sourceRoot":"","sources":["../../src/combinators/literal.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;AAYpF,MAAM,MAAM,cAAc,GAAG;IAC3B,eAAe,CAAC,EAAE,OAAO,CAAA;CAC1B,CAAA;AAED,wBAAgB,OAAO,CAAC,KAAK,EAAE,MAAM,EAAE,IAAI,GAAE,cAAmB,GAAG,UAAU,CAAC,MAAM,CAAC,CAgDpF"}
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
import type { Combinator } from '../types.ts';
|
|
2
|
+
/**
|
|
3
|
+
* A CST/AST node rule. Runs `parser` while collecting its terminals (and, when
|
|
4
|
+
* trivia capture is on, the trivia between them) into `children` / `rawChildren`
|
|
5
|
+
* arrays, then calls `build(children, rawChildren, span)` to produce the node.
|
|
6
|
+
*
|
|
7
|
+
* This is the functional replacement for the class `Parser`'s capital-letter
|
|
8
|
+
* rules + `buildNode`: capture is owned by the library (here and in the
|
|
9
|
+
* compiler), so grammar authors don't hand-wrap terminals or reconstruct trivia.
|
|
10
|
+
*
|
|
11
|
+
* - `children` — structural items in source order: spanned CSTLeaf terminals
|
|
12
|
+
* and sub-nodes (whatever `build` returned for inner nodes).
|
|
13
|
+
* - `rawChildren` — the same, plus CSTTrivia tokens for trivia consumed between
|
|
14
|
+
* terms (only when the parse runs with `captureTrivia`).
|
|
15
|
+
*
|
|
16
|
+
* If `build` returns a non-node value (e.g. a bare string for a collapsed rule),
|
|
17
|
+
* the parent records it as a spanned leaf so its source span is still recoverable.
|
|
18
|
+
*/
|
|
19
|
+
export type BuildNode<N> = (children: ReadonlyArray<unknown>, rawChildren: ReadonlyArray<unknown>, span: {
|
|
20
|
+
start: number;
|
|
21
|
+
end: number;
|
|
22
|
+
}) => N;
|
|
23
|
+
export declare function node<N>(type: string, parser: Combinator<unknown>, build: BuildNode<N>): Combinator<N>;
|
|
24
|
+
//# sourceMappingURL=node.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"node.d.ts","sourceRoot":"","sources":["../../src/combinators/node.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;AAEpF;;;;;;;;;;;;;;;;GAgBG;AACH,MAAM,MAAM,SAAS,CAAC,CAAC,IAAI,CACzB,QAAQ,EAAE,aAAa,CAAC,OAAO,CAAC,EAChC,WAAW,EAAE,aAAa,CAAC,OAAO,CAAC,EACnC,IAAI,EAAE;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,KACjC,CAAC,CAAA;AAEN,wBAAgB,IAAI,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,UAAU,CAAC,OAAO,CAAC,EAAE,KAAK,EAAE,SAAS,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,CAoCrG"}
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
import type { Combinator } from '../types.ts';
|
|
2
2
|
/**
|
|
3
|
-
* Define
|
|
3
|
+
* Define named grammar rules without forward declarations.
|
|
4
4
|
*
|
|
5
|
-
* Pass a factory that receives all rule names as
|
|
6
|
-
* and returns a record of
|
|
5
|
+
* Pass a factory that receives all rule names as references (via a Proxy)
|
|
6
|
+
* and returns a record of combinators. rules() handles creating ref()
|
|
7
7
|
* placeholders and wiring them up — the user never sees ref() at all.
|
|
8
8
|
*
|
|
9
|
-
* const { value } =
|
|
9
|
+
* const { value } = rules(g => ({
|
|
10
10
|
* value: choice(g.object, g.array, str, num, bool, nil),
|
|
11
11
|
* object: transform(sequence('{', sepBy(g.pair, ','), '}'), Object.fromEntries),
|
|
12
12
|
* array: transform(sequence('[', sepBy(g.value, ','), ']'), ([, items]) => items),
|
|
@@ -15,11 +15,11 @@ import type { Combinator } from '../types.ts';
|
|
|
15
15
|
*
|
|
16
16
|
* Not every name in the factory must appear in the returned object — local helpers
|
|
17
17
|
* (like `comma`, `key`) can be plain const inside the factory and composed normally.
|
|
18
|
-
* Only names that OTHER
|
|
18
|
+
* Only names that OTHER rules reference via `g.xxx` need to be in the returned record.
|
|
19
19
|
*
|
|
20
20
|
* TypeScript: use an explicit type parameter for full type safety on `g`:
|
|
21
|
-
*
|
|
21
|
+
* rules<{ value: Combinator<JSONValue>; array: Combinator<JSONValue[]> }>(g => ({ ... }))
|
|
22
22
|
* Without it, `g.*` accesses are typed as `any` but the return is still inferred.
|
|
23
23
|
*/
|
|
24
|
-
export declare function
|
|
24
|
+
export declare function rules<T extends Record<string, Combinator<unknown>>>(factory: (self: any) => T): T;
|
|
25
25
|
//# sourceMappingURL=parser.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"parser.d.ts","sourceRoot":"","sources":["../../src/combinators/parser.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AAG7C;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,wBAAgB,
|
|
1
|
+
{"version":3,"file":"parser.d.ts","sourceRoot":"","sources":["../../src/combinators/parser.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AAG7C;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,wBAAgB,KAAK,CAAC,CAAC,SAAS,MAAM,CAAC,MAAM,EAAE,UAAU,CAAC,OAAO,CAAC,CAAC,EACjE,OAAO,EAAE,CAAC,IAAI,EAAE,GAAG,KAAK,CAAC,GACxB,CAAC,CAmCH"}
|