parseman 0.6.2 → 0.7.0

package/README.md

@@ -112,7 +112,7 @@ If `with { type: 'macro' }` is stripped (older bundlers, test runners), the attr
 
 ### What gets compiled
 
-Combinator trees — `literal`, `regex`, `sequence`, `choice`, `many`, `oneOrMore`, `optional`, `sepBy`, `transform`, `skip`, `not`, `scanTo`, `balanced` — plus `rules()` factories (including mutually recursive ones), `parser({ trivia })` wrappers, and `node()` rules (CST capture and all). A full grammar built as a `rules()` factory of `node()` rules compiles end to end: each rule becomes an independently-callable function, terminal/trivia capture is emitted inline, and every `build`/`transform` callback is inlined with its source span. Grammars with no `node()` emit zero capture code, so they compile byte-identically. Parsers that close over external variables the evaluator can't resolve stay as-is — the plugin compiles what it can and quietly leaves the rest alone.
+Combinator trees — `literal`, `regex`, `sequence`, `choice`, `many`, `oneOrMore`, `optional`, `sepBy`, `transform`, `skip`, `not`, `scanTo`, `balanced` — plus `rules()` factories (including mutually recursive ones), `parser({ trivia })` / `noTrivia()` wrappers, and `node()` rules (CST capture and all). A full grammar built as a `rules()` factory of `node()` rules compiles end to end: each rule becomes an independently-callable function, terminal/trivia capture is emitted inline, and every `build`/`transform` callback is inlined with its source span. Grammars with no `node()` emit zero capture code, so they compile byte-identically. Parsers that close over external variables the evaluator can't resolve stay as-is — the plugin compiles what it can and quietly leaves the rest alone.
 
 ---
 
@@ -155,8 +155,9 @@ A combinator reads from the input at the current position and succeeds or fails
 | `trivia(combinator)` | Label a combinator as skippable filler (whitespace, comments). Does not skip by itself — pass the result to `parser({ trivia })` to turn on auto-skipping between tokens. |
 | `makeWord(boundary?)` | Returns `(str) => Combinator` with a fixed word-boundary class. Not a combinator — see [keyword disambiguation](#ordered-choice-and-keyword-disambiguation). |
 | `rules(factory)` | Named, mutually-recursive rule bundle. See [Named and recursive rules](#named-and-recursive-rules). |
-| `parser({ trivia }, combinator)` | Wrap a root combinator with document-level trivia skipping. See [Whitespace and comment skipping](#whitespace-and-comment-skipping). |
-| `triviaEntries(log, labels?, opts?)` | View over a flat trivia log: `.kind(i)`, `.text(i, input)`, `.start(i)`, `.end(i)`. Stride is 2/3 for root `_triviaLog`, 3/4 for per-node `triviaLog` when kinds are enabled. |
+| `parser({ trivia }, combinator)` | Wrap a root combinator with document-level trivia skipping. `trivia: null` clears trivia for that scope (see `noTrivia`). See [Whitespace and comment skipping](#whitespace-and-comment-skipping). |
+| `noTrivia(combinator)` | Run `combinator` with the active trivia cleared — no trivia is skipped between its `sequence`/`many` terms, so they must be **contiguous** in the input. (Trivia is whatever `parser({ trivia })` installed — often whitespace/comments, but it's grammar-defined.) The inverse of `parser({ trivia })`; re-enable trivia in a nested region with `parser({ trivia }, …)`. For a *static* glued token just use one `literal`/`regex` — reach for `noTrivia` when a glued part is itself a structured sub-rule (e.g. a head that must touch a `[subscript]` or `(args)`). See [Contiguous tokens](#contiguous-tokens-no-trivia-between-them). |
+| `triviaEntries(log, labels?, opts?)` | Indexed view over a trivia log — `.start(i)`, `.end(i)`, `.kind(i)`, `.text(i, input)`. The log is a **flat number array** (no per-entry objects); each entry is a fixed-size run of consecutive numbers, so entry `i` starts at `i * stride`. Root `_triviaLog`: `[start, end]` (stride 2). Per-node `triviaLog` (pass `{ nodeLog: true }`): `[start, end, insertIdx]` (stride 3). Passing `labels` appends a kind index per entry (+1 to stride). |
 
 ---
 
@@ -230,6 +231,27 @@ entries.text(0, input)  // slice on demand
 
 When all trivia arms are labeled, each captured chunk appends a **kind index** to the log (stride +1). Root `_triviaLog` entries are `[start, end, kind]`; per-node `triviaLog` entries are `[start, end, insertIdx, kind]`. Without labels, strides are 2 and 3 respectively.
 
+### Contiguous tokens (no trivia between them)
+
+Trivia skipping is ambient: once `parser({ trivia })` installs it, `sequence`/`many`/`choice` skip filler between **every** term. (Filler is whatever you passed as `trivia` — usually whitespace/comments, but it's grammar-defined.) Sometimes you need the opposite — parts that must touch.
+
+If the whole thing is *static*, don't reach for `noTrivia` — just write one `literal`/`regex`: a decimal is `regex(/[0-9]+\.[0-9]+/)`, an operator is `literal('>=')`. `noTrivia` earns its keep only when a glued part is itself a **structured sub-rule** you can't fold into one pattern — usually recursive, or with its own trivia-enabled interior. The classic case is a head that must touch a bracket, wrapping a sub-expression that *does* allow spaces:
+
+```ts
+const ws = trivia(regex(/[ \t\n]+/))
+
+// `arr[i + 1]` — `arr` is glued to `[` (no `arr [i]`), but the bracketed
+// subscript is a full, space-tolerant expression. A regex can't express that:
+// `expr` is recursive. Turn trivia back on for a region by nesting another
+// parser({ trivia }) — the innermost one wins, and reverts on exit.
+const indexed = noTrivia(sequence(
+  name,
+  parser({ trivia: ws }, sequence(literal('['), expr, literal(']'))),
+))
+```
+
+Put the **whole** trivia-tolerant region inside the nested `parser({ trivia })` — including its leading `[`. `sequence` skips trivia only *between* terms, never before its first, so a `[` left *outside* the nested parser would glue fine but then reject a space right after it (`arr[ i ]`). Two rules of thumb: wrap the whole contiguous run in `noTrivia` (an enclosing `sequence` skips trivia *before* a term, so wrapping just the inner part leaks leading trivia), and wrap the whole spaced region in the nested `parser({ trivia })`. `noTrivia(child)` is exactly `parser({ trivia: null }, child)`.
+
 ---
 
 ## Ordered choice and keyword disambiguation

package/dist/combinators/grammar.d.ts

@@ -11,7 +11,14 @@ export type ParseOptions = {
     recover?: boolean;
 };
 export type ParserOptions = ParseOptions & {
-    trivia?: Combinator<unknown>;
+    /**
+     * Trivia parser for this scope. Three states:
+     *   - a Combinator → skip this trivia between sequence/repeat terms
+     *   - `undefined`  → inherit the enclosing scope's trivia (no change)
+     *   - `null`       → CLEAR trivia: no trivia skipped, so terms must be contiguous.
+     *                    Re-enable inside a nested region with another parser({ trivia }).
+     */
+    trivia?: Combinator<unknown> | null;
     /** Record consumed trivia as CSTTrivia tokens in rawChildren. Default: skip. */
     captureTrivia?: boolean;
 };
@@ -20,5 +27,27 @@ export interface ParsemanParser<T> extends Combinator<T> {
     parse(input: string, pos: number, ctx: ParseContext): ParseResult<T>;
 }
 export declare function parser<T>(opts: ParserOptions, root: Combinator<T>): ParsemanParser<T>;
+/**
+ * Run `root` with the active trivia cleared — no trivia is skipped between its
+ * sequence/repeat terms, so they must be contiguous in the input. (Trivia is
+ * whatever parser({ trivia }) installed — often whitespace/comments, but it is
+ * grammar-defined.) The inverse of parser({ trivia }); re-enable trivia in a
+ * nested region with another parser({ trivia }).
+ *
+ * Wrap the WHOLE contiguous run: an enclosing sequence skips trivia *before* a
+ * term runs, so wrapping just the inner part would let leading trivia through.
+ *
+ * For a static glued token just use one literal/regex; reach for noTrivia when a
+ * glued part is a structured sub-rule — e.g. a head glued to a `[subscript]`
+ * whose interior still allows trivia. Turn trivia back on for a region by nesting
+ * another parser({ trivia }) (innermost wins, reverts on exit); put the WHOLE
+ * spaced region — including its leading `[` — inside it, since sequence skips
+ * trivia only between terms, not before its first:
+ *
+ *   // `arr[i + 1]` — `arr` touches `[`, but the subscript is a spaced expr:
+ *   noTrivia(sequence(name,
+ *     parser({ trivia: ws }, sequence(literal('['), expr, literal(']')))))
+ */
+export declare function noTrivia<T>(root: Combinator<T>): ParsemanParser<T>;
 export declare function parse<T>(combinator: Combinator<T>, input: string, opts?: ParseOptions): ParseResult<T>;
 //# sourceMappingURL=grammar.d.ts.map

package/dist/combinators/grammar.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"grammar.d.ts","sourceRoot":"","sources":["../../src/combinators/grammar.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,aAAa,CAAA;AAGxE,MAAM,MAAM,YAAY,GAAG;IACzB,UAAU,CAAC,EAAE,OAAO,CAAA;IACpB;;;;;;OAMG;IACH,OAAO,CAAC,EAAE,OAAO,CAAA;CAClB,CAAA;AAED,MAAM,MAAM,aAAa,GAAG,YAAY,GAAG;IACzC,MAAM,CAAC,EAAE,UAAU,CAAC,OAAO,CAAC,CAAA;IAC5B,gFAAgF;IAChF,aAAa,CAAC,EAAE,OAAO,CAAA;CACxB,CAAA;AAED,MAAM,WAAW,cAAc,CAAC,CAAC,CAAE,SAAQ,UAAU,CAAC,CAAC,CAAC;IACtD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IACpC,KAAK,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,YAAY,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;CACrE;AAED,wBAAgB,MAAM,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAkCrF;AAED,wBAAgB,KAAK,CAAC,CAAC,EACrB,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EACzB,KAAK,EAAE,MAAM,EACb,IAAI,GAAE,YAAiB,GACtB,WAAW,CAAC,CAAC,CAAC,CAehB"}
+{"version":3,"file":"grammar.d.ts","sourceRoot":"","sources":["../../src/combinators/grammar.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,aAAa,CAAA;AAGxE,MAAM,MAAM,YAAY,GAAG;IACzB,UAAU,CAAC,EAAE,OAAO,CAAA;IACpB;;;;;;OAMG;IACH,OAAO,CAAC,EAAE,OAAO,CAAA;CAClB,CAAA;AAED,MAAM,MAAM,aAAa,GAAG,YAAY,GAAG;IACzC;;;;;;OAMG;IACH,MAAM,CAAC,EAAE,UAAU,CAAC,OAAO,CAAC,GAAG,IAAI,CAAA;IACnC,gFAAgF;IAChF,aAAa,CAAC,EAAE,OAAO,CAAA;CACxB,CAAA;AAED,MAAM,WAAW,cAAc,CAAC,CAAC,CAAE,SAAQ,UAAU,CAAC,CAAC,CAAC;IACtD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IACpC,KAAK,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,YAAY,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;CACrE;AAED,wBAAgB,MAAM,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAwCrF;AAED;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,wBAAgB,QAAQ,CAAC,CAAC,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAElE;AAED,wBAAgB,KAAK,CAAC,CAAC,EACrB,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EACzB,KAAK,EAAE,MAAM,EACb,IAAI,GAAE,YAAiB,GACtB,WAAW,CAAC,CAAC,CAAC,CAehB"}

package/dist/combinators/repeat.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"repeat.d.ts","sourceRoot":"","sources":["../../src/combinators/repeat.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;AAkDpF,wBAAgB,IAAI,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,EAAE,CAAC,CAuBlE;AAED,wBAAgB,SAAS,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,EAAE,CAAC,CA2BvE;AAED,wBAAgB,QAAQ,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,GAAG,IAAI,CAAC,CAiB3E;AAED,wBAAgB,KAAK,CAAC,CAAC,EAAE,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,EAAE,CAAC,CAuDhG"}
+{"version":3,"file":"repeat.d.ts","sourceRoot":"","sources":["../../src/combinators/repeat.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAyC,MAAM,aAAa,CAAA;AAkDpF,wBAAgB,IAAI,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,EAAE,CAAC,CAuBlE;AAED,wBAAgB,SAAS,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,EAAE,CAAC,CA2BvE;AAED,wBAAgB,QAAQ,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,GAAG,IAAI,CAAC,CAoB3E;AAED,wBAAgB,KAAK,CAAC,CAAC,EAAE,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,CAAC,EAAE,CAAC,CAyDhG"}

package/dist/compiler/codegen.d.ts

@@ -30,5 +30,14 @@ export type CompiledParser<T> = {
      */
     inlineExpression: string | null;
 };
+/**
+ * Compile a combinator tree into an optimized parse function at runtime.
+ *
+ * Uses `new Function` internally, so it will fail in environments with a strict
+ * Content Security Policy that omits `'unsafe-eval'`. Prefer the interpreter or
+ * macro build plugin in those cases.
+ *
+ * @see https://www.greadme.com/blog/security/what-is-content-security-policy-complete-guide
+ */
 export declare function compile<T>(combinator: Combinator<T>, mapFnSources?: string[]): CompiledParser<T>;
 //# sourceMappingURL=codegen.d.ts.map

package/dist/compiler/codegen.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"codegen.d.ts","sourceRoot":"","sources":["../../src/compiler/codegen.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,KAAK,EAAE,UAAU,EAAuB,WAAW,EAAE,YAAY,EAAE,UAAU,EAAkB,MAAM,aAAa,CAAA;AAuwCzH,MAAM,MAAM,cAAc,CAAC,CAAC,IAAI;IAC9B,KAAK,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,CAAC,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IAClD,kGAAkG;IAClG,gBAAgB,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,YAAY,EAAE,GAAG,CAAC,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IAChF;;;;OAIG;IACH,eAAe,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,CAAC,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,GAAG;QAAE,MAAM,EAAE,UAAU,EAAE,CAAA;KAAE,CAAA;IACvF,iEAAiE;IACjE,MAAM,EAAE,MAAM,CAAA;IACd;;;;;;OAMG;IACH,gBAAgB,EAAE,MAAM,GAAG,IAAI,CAAA;CAChC,CAAA;AAiCD,wBAAgB,OAAO,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EAAE,YAAY,CAAC,EAAE,MAAM,EAAE,GAAG,cAAc,CAAC,CAAC,CAAC,CAgGhG"}
+{"version":3,"file":"codegen.d.ts","sourceRoot":"","sources":["../../src/compiler/codegen.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,KAAK,EAAE,UAAU,EAAuB,WAAW,EAAE,YAAY,EAAE,UAAU,EAAkB,MAAM,aAAa,CAAA;AAm1CzH,MAAM,MAAM,cAAc,CAAC,CAAC,IAAI;IAC9B,KAAK,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,CAAC,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IAClD,kGAAkG;IAClG,gBAAgB,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,YAAY,EAAE,GAAG,CAAC,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAA;IAChF;;;;OAIG;IACH,eAAe,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,CAAC,EAAE,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,GAAG;QAAE,MAAM,EAAE,UAAU,EAAE,CAAA;KAAE,CAAA;IACvF,iEAAiE;IACjE,MAAM,EAAE,MAAM,CAAA;IACd;;;;;;OAMG;IACH,gBAAgB,EAAE,MAAM,GAAG,IAAI,CAAA;CAChC,CAAA;AAiCD;;;;;;;;GAQG;AACH,wBAAgB,OAAO,CAAC,CAAC,EAAE,UAAU,EAAE,UAAU,CAAC,CAAC,CAAC,EAAE,YAAY,CAAC,EAAE,MAAM,EAAE,GAAG,cAAc,CAAC,CAAC,CAAC,CAgGhG"}