@scinorandex/sparse 0.1.1 → 0.1.2

package/.github/workflows/ci.yml

@@ -0,0 +1,41 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  verify:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: yarn
+
+      - run: yarn install --frozen-lockfile
+
+      - name: Typecheck
+        run: yarn tc
+
+      - name: Test
+        run: yarn test
+
+      - name: Build
+        run: yarn build
+
+      - name: Run the examples
+        run: |
+          npm install -g tsx
+          npx tsx example/math/example.ts
+          npx tsx example/kleene-test/example.ts
+          npx tsx example/LoLang/example.ts
+
+      - name: The committed tables are up to date
+        run: |
+          node dist/cli.js --input=example/math/grammar.txt --output=example/math/table.txt --check
+          node dist/cli.js --input=example/LoLang/grammar.txt --output=example/LoLang/table.txt --check

package/AGENTS.md

@@ -1,14 +1,16 @@
 # AGENTS.md
 
 `@scinorandex/sparse` — LR(1) parser generator library (TypeScript, CJS via tsc, yarn 1).
-Single package: `src/` = library + CLI, `example/` = runnable demos. README.md is the user-facing doc.
+Single package: `src/` = library + CLI, `example/` = runnable demos, `test/` = vitest suite.
+README.md is the user-facing doc (grammar syntax, reducer contract, recovery guide, API reference).
 
 ## Commands
-- Typecheck: `yarn tc` (covers src/ and example/)
+- Typecheck: `yarn tc` (covers src/, example/ and test/)
 - Build: `yarn build` → `dist/` (CLI is `dist/cli.js`)
+- Test: `yarn test` (vitest; builds `dist/` first because `test/cli.test.ts` runs the CLI)
 - Run examples: `tsx example/<name>/example.ts` — tsx is global, not a repo dep; `node file.ts` fails (enums)
 - Table CLI: `node dist/cli.js --input=<grammar> --output=<table>` (build first); output matches committed tables exactly
-- `yarn test` / `yarn bench` (vitest) exist but there are no test/bench files — `yarn test` fails with "No test files found". Verify via examples + `yarn tc` instead.
+- `yarn bench` exists but there are no bench files. Verify with `yarn tc && yarn test` plus the examples.
 - `dist/` and `yarn.lock` are gitignored.
 
 ## Self-hosting / codegen (critical, easy to break)
@@ -19,22 +21,40 @@ Single package: `src/` = library + CLI, `example/` = runnable demos. README.md i
   `02-table-grammar.txt`, run `tsx example/selfhosted/example.ts`, then copy the regenerated
   `codegen-*.ts` over the two `src/.../states.ts` files. The old (checked-in) parser must still
   parse the modified grammar (bootstrap constraint).
+- `test/codegen.test.ts` enforces this: it regenerates both state files in memory and compares them to
+  the checked-in ones. It compares objects (not file bytes) so the `cp` step stays manual.
+- Changing the *lexer* rules in `grammarLexerGenerator` (e.g. allowing `-` inside identifiers) does NOT
+  change the generated states — the token types and positions stay the same.
 
 ## Non-obvious behavior
-- `?`, `*`, `+` groups and multi-token alternatives are unrolled into extra productions; `*`/`+`
+- `?`, `*`, `+` groups and `|` alternatives are unrolled into extra productions; `*`/`+`
   create `autogen-N` productions whose reducer is named `autogenerated-kleene` — user reducers must
-  handle it (see `example/kleene-test/`).
+  handle it (see `example/kleene-test/`). `?` on a group that can match nothing produces an empty
+  production and is rejected by `validateProductions`.
+- Modifiers only apply to parenthesized groups: `([A] [B])?`, not `[A]?`.
+- `|` alternatives live inside a single token: `[A | B]`, `<A | B>`.
 - Two construction paths: `Sparse.fromProductions(...)` builds states at runtime (slow, ~30s for
-  150 productions) vs `new Sparse({ productions, states: buildStates(table.txt) })` (fast).
+  150 productions) vs `Sparse.fromGrammarFile(...)` / `new Sparse({ productions, states })` (fast).
 - Reducers are keyed by production name (`<LHS: name>` in the grammar file); nameless productions
-  have `name: null` (use `reducers[name ?? ""]`). Production index 0 is the accept production —
-  the parse loop breaks on it without calling the reducer.
+  have `name: null`. Production index 0 is the accept production —
+  the parse loop breaks on it without calling the reducer. `oldInput.index` is the pre-unrolling index.
 - Table file format: one line per state, comma-separated `[TERMINAL]=sN|rN` and `<VARIABLE>=M`.
+  The reader preserves file order via `getItemsReversed()` (the item lists are built right-recursively).
 - Errors flow as `Result<T>` (`{ success: false, reason, token }`); `buildProductions` /
-  `Sparse.fromProductions` are the throwing wrappers.
+  `Sparse.fromProductions` / `buildStates` are the throwing wrappers.
+- `GeneratorResult.ActionTable` is sparse: a state whose items only move to other states has no entry.
+  `toStates()`/`toTable()` use `Array.from`, not `map`, so state numbering stays aligned; `toTable()`
+  throws for such grammars because the file format cannot express an action-less state.
+- LALR(1) mode merges same-core LR(1) states. Same-core states always agree on their shifts, gotos and
+  reduce targets, so the conflict branch in `mergeStatesForLALR` is currently unreachable.
 
 ## Key files
 - `src/generator.ts` — LR(1) state construction (first/follow sets, item-set expansion), `toTable()` serialization
-- `src/parser.ts` — runtime parser loop + `recover` error-recovery callback API
+- `src/parser.ts` — runtime parser loop + `recover` error-recovery callback API (`insertToken`, `finish`, `crash`)
 - `src/meta/selfhosted.ts` — grammar-file parser + production unrolling
-- `example/` — `math` (prebuilt table), `kleene-test` (`*`/`+`), `LoLang` (large grammar + recovery), `complicated` (grammar only), `selfhosted` (codegen)
+- `src/meta/common.ts` — `Production` type + `validateProductions`
+- `src/table/selfhosted.ts` — table-file parser (`tryBuildStates`), `src/table/validate.ts` — table validation
+- `src/cli.ts` — `--input/--output/--lalr/--check/--stdout/--quiet/--help`
+- `example/` — `math` (prebuilt table), `kleene-test` (`*`/`+`), `LoLang` (large grammar + recovery, exports
+  its lexer for the tests), `selfhosted` (codegen)
+- `test/` — `grammar`, `table`, `parser`, `lalr`, `codegen`, `cli`

package/README.md

@@ -1,20 +1,33 @@
 ## Sparse - Scin's Parsing Library
 
-Sparse allows developers to easily create LR1 parsers and LR1 parsing tables.
+Sparse allows developers to easily create LR(1) parsers and LR(1) parsing tables.
 
 **Features:**
  - Complete Error Handling API - when parsing fails, you have the final say on where parsing stops
- - Deferred Reductions - you create the nodes and Sparse builds the tree 
+ - Deferred Reductions - you create the nodes and Sparse builds the tree
+ - Grammar and table validation - mistakes are reported with the line, column and a window into your file
  - Full [Slex](https://github.com/scinscinscin/slex) integration - define your entire language as a set of DFA and CFG rules
  - Optional Table Output - use Sparse to build your tables for use in other languages
 
-## Getting Started
+## Table of contents
+
+- [Getting started](#getting-started)
+- [Grammar syntax reference](#grammar-syntax-reference)
+- [Building a parser](#building-a-parser)
+- [Reducers](#reducers)
+- [Repetition with `*` and `+`](#repetition-with--and-)
+- [Error recovery](#error-recovery)
+- [Generating and shipping the parsing table](#generating-and-shipping-the-parsing-table)
+- [API reference](#api-reference)
+- [Known limitations](#known-limitations)
+
+## Getting started
 
 1. **Define the CFG of your language.**
 
-The CFG is defined by creating a file containing a list of productions. Variables are identifiers encased in angle brackets like `<STATEMENT>`, while terminals are encased in square-brackets like `[L_COLON]`. 
+The CFG is defined by creating a file containing a list of productions. Variables are identifiers encased in angle brackets like `<STATEMENT>`, while terminals are encased in square-brackets like `[L_COLON]`.
 
-An example production includes: `<IF_STATEMENT> : [IF] [L_PAREN] <EXPRESSION> [R_PAREN] <STATEMENT> <ELSE_IF_STATEMENTS> [ELSE] <STATEMENT>;`. 
+An example production includes: `<IF_STATEMENT> : [IF] [L_PAREN] <EXPRESSION> [R_PAREN] <STATEMENT> [ELSE] <STATEMENT>;`.
 
 An example CFG for a basic MDAS calculator is the following:
 
@@ -22,16 +35,20 @@ An example CFG for a basic MDAS calculator is the following:
 <S>: <PROGRAM>;
 <PROGRAM>: <EXPRESSION> [EOF];
 <PROGRAM>: [EOF];
+
+// Begin parsing arithmetic expressions
 <EXPRESSION>: <TERM_EXPRESSION>;
 <TERM_EXPRESSION>: <FACTOR_EXPRESSION> [PLUS] <TERM_EXPRESSION>;
 <TERM_EXPRESSION>: <FACTOR_EXPRESSION> [MINUS] <TERM_EXPRESSION>;
 <TERM_EXPRESSION>: <FACTOR_EXPRESSION>;
 <FACTOR_EXPRESSION>: <ENDPOINT> [STAR] <FACTOR_EXPRESSION>;
-<FACTOR_EXPRESSION>: <ENDPOINT> [SLASH] <FACTOR_EXPRESSION>;
+<FACTOR_EXPRESSION>: <ENDPOINT> [FORWARD_SLASH] <FACTOR_EXPRESSION>;
 <FACTOR_EXPRESSION>: <ENDPOINT>;
 <ENDPOINT>: [NUMBER];
 ```
 
+The first production has to be the "accept" production: a variable that appears nowhere else, whose body is a single variable. It is the only production whose reducer Sparse never calls.
+
 2. **Create your [Slex](https://github.com/scinscinscin/slex) lexer.**
 
 ```ts
@@ -61,7 +78,7 @@ lexerGenerator.addRule("number_literal", "${float_number}|${decimal_number}", To
 const lexer = lexerGenerator.generate(`2.4 + 3.5 * 1 / 456.789`, () => ({}));
 ```
 
-3. **Define the node representation.** 
+3. **Define the node representation.**
 
 Sparse allows you to build the AST however you want, deferring to your functions when its time to make a reduction, giving you control over the representation.
 
@@ -71,7 +88,7 @@ type StringifiedNode = (StringifiedNode | string)[];
 // It doesn't have to be a class. It just has to be a structure that all nodes
 // in the AST adhere to. Classes allow this to be done easily through subclassing.
 class Node {
-  constructor(public readonly nodes: LR1StackSymbol<TokenType, {}, Node>[]) {}
+  constructor(public readonly nodes: LR1StackSymbol<TokenType, Metadata, Node>[]) {}
   toObject(): StringifiedNode {
     return this.nodes.map((node) => (node.type === "token" 
       ? node.token.lexeme 
@@ -81,53 +98,290 @@ class Node {
 }
 ```
 
-4. **Building the productions and the parser.**
+4. **Building the parser.**
 
-Productions are built using the `buildProductions()` function, which can be passed into Sparse alongside `toStringifiedTokenType`, which converts a numerical TypeScript enum to the name of the terminal.
+With a prebuilt parsing table, `Sparse.fromGrammarFile` reads the grammar, reads the table, and checks that the two belong together:
 
 ```ts
+import { Sparse, enumToString } from "@scinorandex/sparse";
+
+async function main() {
+  const parserGenerator = await Sparse.fromGrammarFile<TokenType, Metadata, Node>({
+    grammarPath: "./example/math/grammar.txt",
+    tablePath: "./example/math/table.txt",
+    toStringifiedTokenType: enumToString<TokenType>(TokenType),
+    onWarning: ({ reason, token }) => console.warn(`${token.line}:${token.column} ${reason}`),
+  });
+
+  const parser = parserGenerator.generate(lexer, {
+    reducer: (_, { input }) => new Node(input),
+  });
+
+  console.log(parser.parse().result!.toObject());
+}
+```
+
+Generating the table at startup instead (fine for small grammars, slow for big ones):
+
+```ts
+import { buildProductions, Sparse, enumToString } from "@scinorandex/sparse";
+
 async function main() {
-  const toStringifiedTokenType = (type: TokenType) => TokenType[type];
   const productions = buildProductions(await fs.readFile("./example/math/grammar.txt", "utf8"));
-  const parserGenerator = Sparse.fromProductions<TokenType, Metadata, Node>({ productions, toStringifiedTokenType });
+  const parserGenerator = Sparse.fromProductions<TokenType, Metadata, Node>({
+    productions,
+    toStringifiedTokenType: enumToString<TokenType>(TokenType),
+  });
 }
 ```
 
-5. **Define your reducer and begin parsing.**
+## Grammar syntax reference
+
+A grammar file is a list of productions. Whitespace is insignificant, `//` starts a line comment, and `/* ... */` spans lines.
+
+| Syntax | Meaning |
+| --- | --- |
+| `<A>: <B> [C];` | `A` is made of one `B` followed by one `C` |
+| `<A: name>: ...;` | The production is *named*: its reducer is looked up under `name` |
+| `[TOK: name]` | Names the terminal on the right hand side so it shows up on the reducer's `bag` |
+| `<VAR: name>` | Same, for a variable |
+| `[A | B]` | Alternatives: unrolled into one production per alternative |
+| `<A | B>` | Same, for variables |
+| `([A] [B])?` | Optional group, unrolled into one production with and one without it |
+| `// comment`, `/* comment */` | Comments |
+
+Identifiers may contain letters, digits, `_` and `-`, and must start with a letter or `_`.
+
+### What Sparse checks for you
+
+Grammar and table problems are reported as a `Result` failure with the offending token, so the CLI and
+`loadGrammar` can print a window into your file:
+
+- a variable used on the right hand side that no production defines
+- an empty right hand side (a production like `<A: a>: ([X])?;` expands to nothing, and Sparse has no support for empty productions)
+- the start symbol appearing on the right hand side
+- two symbols in one production sharing a name (warns: only the last one reaches the `bag`)
+- a production that is never reachable from the start symbol (warns)
+
+```
+$ npx sparse --input=broken.txt --output=table.txt
+Invalid syntax at 2:30: got SEMICOLON (";"), but expected one of [L_ANGLE], [L_BRACKET], [L_PAREN]
+
+2 | <PROGRAM: program>: [NUMBER] (;
+  |                               ~
+3 |
+```
 
-Finally, you can define the parser's reduction function and optionally define the recovery handling function.
+## Reducers
+
+Reducers are keyed by the name in the grammar, and receive two arguments:
 
 ```ts
+type Reducer<TokenType, Metadata, Node> = (
+  newInput: { bag: Record<string, Node | Token<TokenType, Metadata>>; name: string | null },
+  oldInput: { input: LR1StackSymbol<TokenType, Metadata, Node>[]; index: number },
+) => Node;
+```
+
+- `newInput.name` is the production's name, or `null` for an unnamed production.
+- `newInput.bag` holds the right hand side symbols that were named in the grammar, keyed by those names. If two symbols share a name, the last one wins, so reach for `oldInput.input` when that happens.
+- `oldInput.input` is the reduced symbols in order, as `{ type: "token", token }` or `{ type: "node", node }`.
+- `oldInput.index` is the production's index *in the grammar file*, before `*`/`?`/`+`/`|` unrolling. Index 0 is the accept production, whose reducer is never called.
+
+`defineReducers` turns a map of reducers into the reducer that `generate` wants, and tells you exactly which production has no reducer:
+
+```ts
+import { assertReducersCoverGrammar, defineReducers } from "@scinorandex/sparse";
+
+const reducers = {
+  program: ({ bag }) => new Node(bag),
+  expression: ({ bag }, { input }) => new Node(input),
+};
+
+assertReducersCoverGrammar(productions, reducers); // throws if something is missing
+
 const parser = parserGenerator.generate(lexer, {
-  reducer: (_, { input, index }) => new Node(input),
+  reducer: defineReducers(reducers),
 });
+```
 
-console.log(parser.parse().result!.toObject());
+Without the helper the failure looks like this, halfway through a parse:
+
+```
+Error while performing the reduction for production 2 ("expression"): I cannot reduce a null bag
 ```
 
-## Exporting and Loading the Parsing Table
+## Repetition with `*` and `+`
 
-It takes a while for Sparse to build states (around 0.5 seconds seconds for a file containing 150 productions). This can be alleviated by generating the parsing table and loading the states directly instead. This method also allows you to edit the parsing table to resolve parsing conflicts.
+`[A]?`, `A B*`, and `A B+` are unrolled into extra productions. For `*` and `+` that means a new
+production named **`autogenerated-kleene`** (with variables called `autogen-0`, `autogen-1`, ...), which
+you **have** to implement: it is handed the items matched so far, and it has to flatten them into a list.
+The first reduction has no `rest`, later ones do:
 
-To create the the states, you can run `npx @scinorandex/sparse --input=<input> --output=<output>`.
+```ts
+class KleeneNode<T> extends Node {
+  contents: T[] = [];
 
-By default, LR(1) states are generated. You can generate LALR(1) states instead by passing the `--lalr` flag (or setting `mode: "lalr1"` in `Sparse.fromProductions`): the resulting table is never larger than the LR(1) one, but generation fails if the grammar is not LALR(1). Using LALR(1) yields a ~35% performance boost over LR(1) for the same grammar (tested on the LoLang example).
+  constructor(nodes: LR1StackSymbol<TokenType, Metadata, Node>[], bag: T) {
+    super(nodes);
+    this.contents.unshift(bag);
+  }
 
-Afterwards, you can create a parser with pre-built states like the following:
+  add(nodes: LR1StackSymbol<TokenType, Metadata, Node>[], bag: T) {
+    this.nodes.unshift(...nodes.slice(0, nodes.length - 1));
+    this.contents.unshift(bag);
+    return this;
+  }
+}
 
-```ts
-async function main() {
-  const toStringifiedTokenType = (type: TokenType) => TokenType[type];
-  const productions = buildProductions(await fs.readFile("./example/math/grammar.txt", "utf8"));
+const reducers = {
+  "autogenerated-kleene": ({ bag: { rest, ...item } }, { input }) =>
+    rest == null ? new KleeneNode(input, item) : rest.add(input, item),
+  program: (_, { input }) => new Node(input),
+};
+```
+
+See `example/kleene-test` for the whole thing.
 
-  // Notice how the "new Sparse()" constructor is used here instead of "Sparse.fromProductions()"
-  const states = buildStates(await fs.readFile("./example/math/table.txt", "utf8"));
-  const parserGenerator = new Sparse<TokenType, Metadata, Node>({ productions, states, toStringifiedTokenType });
+## Error recovery
+
+By default a syntax error throws `LR1ParserGraveError`, which carries the token that could not be parsed:
+
+```ts
+try {
+  parser.parse();
+} catch (err) {
+  if (err instanceof LR1ParserGraveError) console.error(err.reason, err.currentToken);
 }
 ```
 
+Pass a `recover` function to decide what happens instead. It receives the lexer, both stacks, the table,
+and these helpers:
+
+| Helper | What it does |
+| --- | --- |
+| `addError(reason)` | Records an error, keeps parsing, and returns it in `parse().errors` |
+| `crash(reason)` | Stops parsing and throws a `LR1ParserGraveError` |
+| `finish(options?)` | Stops recovery and carries on parsing from the current state |
+| `insertToken(token)` | Shifts a token the input did not have, e.g. a synthesized `;` |
+| `isSafe()` | True when the current state can consume the next real token |
+
+```ts
+const parser = parserGenerator.generate(lexer, {
+  reducer: (_, { input }) => new Node(input),
+  recover({ lexer, states, statesStack, insertToken, addError, isSafe, finish, crash }) {
+    const token = lexer.peekNextToken();
+
+    // skip the extra semicolons the author left behind
+    while (token.type === TokenType.SEMICOLON) {
+      lexer.getNextToken();
+      if (isSafe()) return finish();
+    }
+
+    // or insert the semicolon they forgot
+    if (states[statesStack.peek()].getTerminalAction("SEMICOLON") != null) {
+      const semicolon = new Token(TokenType.SEMICOLON, ";", new ColumnAndRow(token.line, token.column), {});
+      const inserted = insertToken(semicolon);
+      if (inserted != null) return crash(inserted.reason);
+      addError(`Expected SEMICOLON but received ${TokenType[token.type]}`);
+      if (isSafe()) return finish();
+    }
+
+    return crash("Invalid syntax");
+  },
+});
+
+const { result, errors } = parser.parse();
+```
+
+A parser consumes its lexer, so build a new one per input with `generate(...)`, or call `parser.reset()`
+if you want to reuse the parser with a lexer that still has tokens left.
+
+## Generating and shipping the parsing table
+
+Generating states for a large grammar takes a while (a few seconds for a few hundred productions), so
+generate the table once at build time and commit it:
+
+```
+npx sparse --input=grammar.txt --output=table.txt
+```
+
+| Flag | Meaning |
+| --- | --- |
+| `--input=<file>` | Grammar to read (required) |
+| `--output=<file>` | Table to write; missing directories are created |
+| `--lalr` | Generate LALR(1) states instead of LR(1) |
+| `--check` | Write nothing; verify that `<output>` already matches the grammar |
+| `--stdout` | Print the table instead of writing it |
+| `--quiet` | Hide warnings about suspicious rules |
+| `--help` | Usage |
+
+By default, LR(1) states are generated. You can generate LALR(1) states instead by passing the `--lalr`
+flag (or setting `mode: "lalr1"` in `Sparse.fromProductions`): the resulting table is never larger than
+the LR(1) one. Using LALR(1) yields a ~35% performance boost over LR(1) for the same grammar (tested on
+the LoLang example).
+
+`--check` is useful in CI to prove that a committed table still matches its grammar:
+
+```
+npx sparse --input=example/math/grammar.txt --output=example/math/table.txt --check
+```
+
+Then load the table with `Sparse.fromGrammarFile`, as shown in [Getting started](#getting-started).
+
+## API reference
+
+### Reading and writing grammars
+
+| Function | Returns |
+| --- | --- |
+| `buildProductions(source)` / `tryBuildProductions(source)` | The unrolled productions. Throws / returns a `Result` failure with the offending token |
+| `loadGrammar(path)` | Same, but reads the file, turning missing files into failures too |
+| `buildStates(table)` / `tryBuildStates(table)` | `TableState[]` from a table file, validating action syntax and state numbers |
+| `loadTable(path)` | Same, but reads the file |
+| `generateStates(productions, options?)` | `GeneratorResult`; `options` is `{ mode, onWarning, onProgress }` |
+| `validateProductions(productions, options?)` | Fails on a grammar that cannot generate a correct table |
+| `validateTable(productions, states)` | Fails when a table does not belong to the productions |
+| `validateTableStates(states, options?)` | Fails on a malformed table |
+
+### Building parsers
+
+| Function | Returns |
+| --- | --- |
+| `Sparse.fromGrammarFile({ grammarPath, tablePath, toStringifiedTokenType, onWarning?, validate? })` | Loads a grammar and its prebuilt table, cross-checking them |
+| `Sparse.fromProductions({ productions, toStringifiedTokenType, mode?, quiet? })` | Generates states at startup |
+| `Sparse.tryFromProductions(...)` | Same, as a `Result` |
+| `new Sparse({ productions, states, toStringifiedTokenType, validate?, source? })` | Use when you already hold the states |
+| `generator.generate(lexer, { reducer, recover? })` | A parser |
+| `parser.parse()` | `{ result, errors }` |
+| `parser.reset()` | Clears the stacks and errors |
+
+### Helpers
+
+| Export | Purpose |
+| --- | --- |
+| `enumToString(TokenType)` | The `toStringifiedTokenType` every example used to write by hand |
+| `defineReducers(reducers)` | Turns a map of named reducers into a reducer |
+| `assertReducersCoverGrammar(productions, reducers)` | Fails up front when a named production has no reducer |
+| `missingReducerNames(productions, reducers)` / `missingReducerMessage(...)` | Same, as data |
+| `namedProductions(productions)` | Every production the grammar names |
+| `TableState.fromJSObject(json)` / `GeneratorResult.toJSObject()` | Round trip a table through JSON |
+| `hydrateProduction(json)` / `dehydateProduction(production)` | Round trip productions through JSON |
+| `buildErrorWindow(source, token)` | Renders the window around a token |
+| `Stack` | `push`, `pop`, `peek`, `peekAt`, `size`, `isEmpty`, `toArray` |
+
+## Known limitations
+
+- `?`, `*` and `+` only apply to a parenthesized group, and a group that can match nothing
+  (`<A: a>: ([X])?;`) is rejected because Sparse has no support for empty productions. Write two
+  productions instead.
+- Sparse does not detect grammar conflicts (shift/reduce, reduce/reduce). An ambiguous grammar produces a
+  table with one arbitrary resolution, so if your parser behaves strangely, check your grammar for ambiguity.
+- A state with no actions cannot be written in the table file format; `toTable()` throws for such grammars.
+  Use `Sparse.fromProductions` to keep the states in memory instead.
+
 ## AI Disclaimer
 
 This project was originally written without the use of AI tools, the core LR(1) table generator was written by hand as per the algorithms described in the Dragon Book. Every release prior to v0.1 contained no AI generated code.
 
-OpenCode and Qwen 3.8 27B were to implement performance improvements on the original LR(1) table generator and to implement LALR(1) support.
+OpenCode and Qwen 3.8 27B were to implement performance improvements on the original LR(1) table generator and to implement LALR(1) support.

package/dist/cli.js

@@ -5,43 +5,132 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 const promises_1 = __importDefault(require("fs/promises"));
-const index_1 = require("./index");
-const generator_1 = require("./generator");
-const minimist_1 = __importDefault(require("minimist"));
 const path_1 = __importDefault(require("path"));
-const errorWindowBuilder_1 = require("./utils/errorWindowBuilder");
+const minimist_1 = __importDefault(require("minimist"));
+const index_1 = require("./index");
 const args = (0, minimist_1.default)(process.argv.slice(2));
-function checkFileExists(filepath) {
-    return new Promise((resolve, reject) => {
-        promises_1.default.access(filepath, promises_1.default.constants.F_OK)
-            .then(() => resolve(true))
-            .catch(() => resolve(false));
-    });
+const BINARY_NAME = "@scinorandex/sparse";
+const USAGE = `Usage: npx @scinorandex/sparse --input=<grammar> --output=<table> [options]
+
+Generates an LR(1) parsing table from a grammar file.
+
+Required:
+  --input=<file>        Grammar file to read
+  --output=<file>       Where to write the parsing table (created if the directory is missing)
+
+Options:
+  --lalr                Generate LALR(1) states instead of LR(1) (smaller table, ~35% faster parsing)
+  --check               Do not write anything: verify that <output> already matches the grammar
+  --stdout              Print the table instead of writing it to a file
+  --quiet               Do not print warnings about suspicious grammar rules
+  --help                Show this message
+
+Run with npx ${BINARY_NAME} (the package is @scinorandex/sparse).`;
+function fail(message) {
+    console.error(message);
+    process.exitCode = 1;
+}
+function toTableOrFail(result) {
+    try {
+        return result.toTable();
+    }
+    catch (err) {
+        fail(err instanceof Error ? err.message : String(err));
+        return null;
+    }
 }
 async function main() {
-    if (typeof args.input !== "string" || typeof args.output !== "string") {
-        console.log("Usage: npx @scinorandex/sparse --input=<input> --output=<output> [--lalr]");
+    if (args.help === true || args.h === true) {
+        console.log(USAGE);
+        return;
+    }
+    if (typeof args.input !== "string") {
+        fail(`Missing --input.\n\n${USAGE}`);
+        return;
+    }
+    const useStdout = args.stdout === true;
+    const check = args.check === true;
+    if (typeof args.output !== "string" && !useStdout) {
+        fail(`Missing --output.\n\n${USAGE}`);
         return;
     }
     const inputFile = path_1.default.resolve(args.input);
-    const outputFile = path_1.default.resolve(args.output);
-    console.log(`Reading grammar from ${inputFile} and outputting table to ${outputFile}\n`);
-    if (!(await checkFileExists(inputFile)))
-        return console.log(`File ${inputFile} does not exist`);
-    const grammar = await promises_1.default.readFile(args.input, "utf8");
-    const productionsResult = (0, index_1.tryBuildProductions)(grammar);
+    const outputFile = typeof args.output === "string" ? path_1.default.resolve(args.output) : undefined;
+    let grammar;
+    try {
+        grammar = await promises_1.default.readFile(inputFile, "utf8");
+    }
+    catch (_a) {
+        fail(`Cannot read grammar file ${inputFile}: it does not exist.`);
+        return;
+    }
+    const onWarning = (warning) => {
+        if (args.quiet === true)
+            return;
+        console.error(`Warning in ${inputFile} at ${warning.token.line}:${warning.token.column}: ${warning.reason}`);
+    };
+    const productionsResult = (0, index_1.tryBuildProductions)(grammar, { onWarning });
     if (productionsResult.success === false) {
-        console.log(productionsResult.reason);
-        console.log((0, errorWindowBuilder_1.buildErrorWindow)(grammar, productionsResult.token));
+        fail(`${productionsResult.reason}\n\n${(0, index_1.buildErrorWindow)(grammar, productionsResult.token)}`);
+        return;
+    }
+    const productions = productionsResult.value;
+    if (check) {
+        if (outputFile == null) {
+            fail(`--check needs --output to know which table to check.\n\n${USAGE}`);
+            return;
+        }
+        let existingTable;
+        try {
+            existingTable = await promises_1.default.readFile(outputFile, "utf8");
+        }
+        catch (_b) {
+            fail(`Cannot check ${outputFile}: it does not exist.`);
+            return;
+        }
+        const statesResult = (0, index_1.tryBuildStates)(existingTable, { productions });
+        if (statesResult.success === false) {
+            fail(`The existing table is not a valid parsing table:\n${statesResult.reason}`);
+            return;
+        }
+        const tableCheckResult = (0, index_1.validateTable)(productions, statesResult.value, { source: outputFile });
+        if (tableCheckResult.success === false)
+            fail(`The existing table does not match the grammar:\n${tableCheckResult.reason}`);
+        const expected = (0, index_1.generateStates)(productions, { mode: args.lalr === true ? "lalr1" : "lr1" });
+        if (expected.success === false) {
+            fail(`${expected.reason}\n\n${(0, index_1.buildErrorWindow)(grammar, expected.token)}`);
+            return;
+        }
+        const expectedTable = toTableOrFail(expected.value);
+        if (expectedTable == null)
+            return;
+        const actualTable = existingTable.trim();
+        if (expectedTable !== actualTable) {
+            fail(`${outputFile} is out of date with respect to ${inputFile}.\n` +
+                `Re-run without --check to regenerate it${args.lalr === true ? " (with --lalr)" : ""}.`);
+            return;
+        }
+        console.log(`${outputFile} is up to date with respect to ${inputFile} (${expectedTable.split("\n").length} states).`);
         return;
     }
-    const generatorResult = (0, generator_1.generateStates)(productionsResult.value, { mode: args.lalr === true ? "lalr1" : "lr1" });
+    const generatorResult = (0, index_1.generateStates)(productions, { mode: args.lalr === true ? "lalr1" : "lr1" });
     if (generatorResult.success === false) {
-        console.log(generatorResult.reason);
-        console.log((0, errorWindowBuilder_1.buildErrorWindow)(grammar, generatorResult.token));
+        fail(`${generatorResult.reason}\n\n${(0, index_1.buildErrorWindow)(grammar, generatorResult.token)}`);
         return;
     }
-    await promises_1.default.writeFile(outputFile, generatorResult.value.toTable(), { encoding: "utf-8" });
+    const table = toTableOrFail(generatorResult.value);
+    if (table == null)
+        return;
+    if (useStdout) {
+        console.log(table);
+        return;
+    }
+    if (outputFile == null)
+        return;
+    await promises_1.default.mkdir(path_1.default.dirname(outputFile), { recursive: true });
+    await promises_1.default.writeFile(outputFile, table, { encoding: "utf-8" });
+    const stateCount = table === "" ? 0 : table.split("\n").length;
+    console.log(`Wrote ${stateCount} ${args.lalr === true ? "LALR(1)" : "LR(1)"} states from ${inputFile} to ${outputFile}.`);
 }
 main();
 //# sourceMappingURL=cli.js.map