@sormy/lemon-js 1.0.1

package/LICENSE

@@ -0,0 +1,12 @@
+lemon-js is in the public domain.
+
+The lemon parser generator and its parser template are taken from the SQLite
+distribution, whose author disclaims copyright to them. The translation of the
+template to JavaScript, the changes to the generator and everything else here
+are released on the same terms.
+
+In place of a legal notice, here is a blessing:
+
+   May you do good and not evil.
+   May you find forgiveness for yourself and forgive others.
+   May you share freely, never taking more than you give.

package/README.md

@@ -0,0 +1,194 @@
+# LEMON.JS - LALR(1) Parser Generator for JavaScript
+
+[![CI](https://github.com/sormy/lemon-js/actions/workflows/ci.yml/badge.svg)](https://github.com/sormy/lemon-js/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/@sormy/lemon-js)](https://www.npmjs.com/package/@sormy/lemon-js)
+
+Lemon.JS is an LALR(1) parser generator for JavaScript based on Lemon parser generator for C included in SQLite package distribution.
+
+Looking for a lexer to go with it? [FLEX.JS](https://github.com/sormy/flex-js).
+
+## Parser Code Base
+
+Files `lemon.c`, `lempar.c`, `lemon.html` are extracted from SQLite v3.17.0. Original parser generator code is slightly fixed to produce JavaScript compatible statements. Parser template translated from C to JavaScript. Source comments mostly not touched to keep it easy diff against original file.
+
+Both original C version and patched JS version are included for side by side comparison for reference.
+
+## Installation
+
+From npm, which carries a prebuilt generator for macOS, Linux and Windows on
+both x86_64 and arm64:
+
+```bash
+npm install --save-dev @sormy/lemon-js
+npx lemon-js grammar.y
+```
+
+The package is `@sormy/lemon-js` and the command it installs is `lemon-js`.
+
+Or from source, which is the way on any platform the package does not carry:
+
+```bash
+cc -o lemon-js -O2 lemon-js.c
+./lemon-js grammar.y
+```
+
+Built this way the generator looks for `lempar.js` beside its own executable,
+so keep the two together, or name the template with `-T`.
+
+## Usage
+
+```bash
+lemon-js <filename>.y
+```
+
+The parser is written beside the grammar as `<filename>.js`, along with a
+TypeScript declaration, `<filename>.d.ts`, and a report of the grammar and the
+states it produced, `<filename>.out`. Pass `-q` if the report is not wanted.
+
+- `-q` - do not write the report file.
+- `-l` - leave out the `// line ...` comments pointing back at the grammar.
+- `-T<file>` - read another parser template. Installed from npm this already
+  points at the bundled `lempar.js`, so it is only needed for a template of
+  your own.
+- `-s` - print statistics about the parser tables.
+- `-g` - print the grammar without actions.
+- `-b` - report only the basis of each state, rather than every configuration.
+- `-c` - do not compress the action table.
+- `-p` - show the conflicts that precedence rules resolved.
+- `-r` - do not sort or renumber the states.
+- `-x` - print the version.
+
+A grammar the generator will not accept is reported and exits non-zero.
+
+See `lemon.html`, included here, or https://sqlite.org/lemon.html for more details.
+
+## Special Directives
+
+- %name - Set parser class name (default is "Parse")
+- %include - Include code in the beginning of file (useful for imports)
+- %code - Include code in the end of file (useful for exports or main code)
+- %token_destructor - Define code which will be executed on token destruction.
+- %default_destructor - Destructor for the non-terminals that have none of their own.
+- %token_prefix - Define token name prefix.
+- %syntax_error - Define custom error handler for syntax errors.
+- %parse_accept - Code to run when the parser accepts its input, meaning every token was processed without error.
+- %parse_failure - Code to run once error recovery has failed and the parse cannot go on.
+- %stack_overflow - Define handler for stack overflow.
+- %extra_argument - Declare a name visible in every action, given to the constructor: `%extra_argument { ctx }` in the grammar, `new Parser(ctx)` in the code.
+- %token_type - Type of the value a token carries, used in the generated declaration: `%token_type { number }` makes `parse` take a `number`.
+- %default_type - **NOT SUPPORTED**, nothing to declare while the output is JavaScript
+- %stack_size - Depth of the parser stack, 100 by default.
+- %start_symbol - Symbol the grammar starts at, the left side of the first rule by default.
+- %left - Set left associative tokens.
+- %right - Set right associative tokens.
+- %nonassoc - Set non associative tokens.
+- %destructor - Destructor for one non-terminal symbol, as %token_destructor is for terminals.
+- %type - **NOT SUPPORTED**, nothing to declare while the output is JavaScript
+- %fallback - Give tokens an alternative meaning, tried when the original would be a syntax error.
+- %wildcard - Name the token that matches any input token.
+- %token_class - Define a class of tokens usable as one symbol in rules, `%token_class number INTEGER|FLOAT.`
+
+Notes:
+
+- a regular expression holding `/*`, such as `/\/*/`, ends a `%code` or
+  `%include` section early: the generator reads `/*` as the start of a comment
+  and swallows the rest of the file. It says so and stops rather than producing
+  anything, and writing the star as a character class, `/\/[*]/`, avoids it.
+- the best place to put something like `module.exports = ParserName;` or `export default ParserName;` is in `%code` section.
+
+## Generated Parser
+
+The generator writes a `.js` beside the grammar holding one class, named by
+`%name`. Nothing is exported unless you say so, which is what the `%code`
+section is for:
+
+```javascript
+%code { module.exports = Parser; }
+```
+
+The class carries a constant for every terminal, named with `%token_prefix`,
+and these methods:
+
+- `new Parser(argument)` - the argument is the one `%extra_argument` names, and
+  is left out by a grammar that declares none.
+- `parse(major, minor)` - hand over one token: its constant, and the value the
+  actions receive. Call `parse()` with nothing to say the input has ended.
+- `setTraceCallback(callback, prompt)` - report every shift, reduce and accept.
+  The callback is given the text, and the prompt goes in front of each line.
+- `trace(message)` - write one line through that callback.
+- `getStackPeak()` - how deep the parser stack has been, to compare against
+  `%stack_size`.
+- `init()` - return the parser to its starting state, ready for another input.
+  The constructor calls it, so it is only needed to reuse a parser.
+- `finalize()` - run the destructors over whatever the stack still holds, for a
+  parse given up partway. It is worth calling only for a grammar that declares
+  destructors: they may hold something JavaScript will not reclaim on its own,
+  where the stack itself it will. The parser needs `init()` again afterwards,
+  so this is not a cleanup to reach for out of habit.
+
+A declaration is written beside the parser, so TypeScript callers get the token
+constants as literal types and are told when one does not exist. It assumes the
+parser is exported as `export =`, which is what `module.exports = Parser` in a
+`%code` section comes to.
+
+```javascript
+var parser = new Parser(context);
+
+parser.parse(parser.TOKEN_INT, 42);
+parser.parse(parser.TOKEN_PLUS);
+parser.parse(parser.TOKEN_INT, 8);
+parser.parse();
+```
+
+## Lexer
+
+A primitive single-state lexer is bundled in `lexer/`. It has no dependencies,
+takes the first rule that matches, and is enough for the calculator example or
+as a starting point for one of your own.
+
+Past that, [FLEX.JS](https://github.com/sormy/flex-js) is the lexer this project
+is meant to be used with: FLEX semantics, longest match whatever order the rules
+were added in, start conditions, and actions that run while scanning. Lemon.JS is
+the parser generator that goes with it.
+
+## Development
+
+```bash
+npm test          # against a generator compiled from this tree
+./build-dev.sh    # both generators, and the C example, to compare them
+./build-dist.sh   # the binaries the package ships
+npm run test:dist # the same tests, against those binaries
+```
+
+`npm test` compiles `lemon-js.c` first, so it never runs against a stale
+binary. Publishing builds the shipped binaries and then runs the tests against
+them, reaching them the way an install does.
+
+The original `lemon.c` and `lempar.c` are kept beside the ported ones so the
+changes stay easy to read as a diff. Nothing should reformat them, which is
+why trailing whitespace is left alone in C files.
+
+## Alternative Lexers
+
+Maintained:
+
+- [moo](https://github.com/no-context/moo) - all rules compiled into one sticky expression, first match wins
+- [chevrotain](https://chevrotain.io/) - lexer and parser toolkit, token modes, first match wins
+
+Unmaintained:
+
+- [lexed.js](https://github.com/tantaman/lexed.js)
+- [lexer](https://github.com/aaditmshah/lexer)
+- [jslex](https://github.com/YuhangGe/jslex)
+
+## Alternative Parsers
+
+- [jison](https://github.com/zaach/jison) - parser generator, `jison-lex` accepts FLEX-style lex files
+- [peggy](https://peggyjs.org/) - PEG parser generator, successor to PEG.js, no separate scanner
+- [jscc](http://jscc.brobston.com) - LALR(1) parser generator
+
+## License
+
+Public domain. The generator and its template come from SQLite, whose author
+disclaims copyright to them, and the translation to JavaScript and the changes
+to the generator are released on the same terms. See [LICENSE](LICENSE).

package/bin/lemon-js.js

@@ -0,0 +1,50 @@
+#!/usr/bin/env node
+
+/*
+** Runs the prebuilt generator for the platform in use.
+**
+** The generator looks for lempar.js beside its own executable, which is not
+** where it lives here, so the template is named explicitly unless the caller
+** already named one.
+*/
+
+'use strict';
+
+var path = require('path');
+var childProcess = require('child_process');
+
+var BINARIES = {
+  'darwin-x64': 'lemon-js-darwin-universal',
+  'darwin-arm64': 'lemon-js-darwin-universal',
+  'linux-x64': 'lemon-js-linux-x64',
+  'linux-arm64': 'lemon-js-linux-arm64',
+  'win32-x64': 'lemon-js-win32-x64.exe',
+  'win32-arm64': 'lemon-js-win32-arm64.exe'
+};
+
+var platform = process.platform + '-' + process.arch;
+var binary = BINARIES[platform];
+
+if (!binary) {
+  process.stderr.write(
+    '@sormy/lemon-js ships no generator for ' + platform + '.\n' +
+    'It can be built from source: https://github.com/sormy/lemon-js#installation\n'
+  );
+  process.exit(1);
+}
+
+var args = process.argv.slice(2);
+var named = args.some(function (arg) { return arg.indexOf('-T') === 0; });
+
+if (!named) {
+  args.unshift('-T' + path.join(__dirname, '..', 'lempar.js'));
+}
+
+var run = childProcess.spawnSync(path.join(__dirname, '..', 'dist', binary), args, { stdio: 'inherit' });
+
+if (run.error) {
+  process.stderr.write(run.error.message + '\n');
+  process.exit(1);
+}
+
+process.exit(run.status === null ? 1 : run.status);

package/examples/calculator-js.y

@@ -0,0 +1,74 @@
+%name Parser
+
+%token_prefix TOKEN_
+
+%left PLUS MINUS.
+%left DIVIDE TIMES.
+
+%include {
+  // include something
+}
+
+%code {
+  var Lexer = require('../lexer/lexer');
+
+  var parser = new Parser();
+
+  parser.setTraceCallback(function (value) {
+    process.stdout.write(value);
+  }, '> ');
+
+  var lexer = new Lexer();
+
+  lexer.addRule(/\d+/, function (value) {
+    return { major: parser.TOKEN_INTEGER, minor: parseInt(value, 10) };
+  });
+  lexer.addRule('+', function (value) {
+    return { major: parser.TOKEN_PLUS, minor: null };
+  });
+  lexer.addRule('-', function (value) {
+    return { major: parser.TOKEN_MINUS, minor: null };
+  });
+  lexer.addRule('/', function (value) {
+    return { major: parser.TOKEN_DIVIDE, minor: null };
+  });
+  lexer.addRule('*', function (value) {
+    return { major: parser.TOKEN_TIMES, minor: null };
+  });
+  lexer.addRule(/\s+/, function () {});
+
+  var data = '';
+
+  process.stdin.on('data', function (chunk) {
+    data += chunk;
+  });
+
+  process.stdin.on('end', function () {
+    var token;
+
+    lexer.setInput(data);
+
+    while (token = lexer.lex()) {
+      parser.parse(token.major, token.minor);
+    }
+
+    parser.parse();
+  });
+}
+
+%syntax_error {
+  console.log("Syntax error");
+}
+
+program ::= expr(A). { console.log("Result=" + A); }
+expr(A) ::= expr(B) MINUS expr(C). { A = B - C; }
+expr(A) ::= expr(B) PLUS expr(C). { A = B + C; }
+expr(A) ::= expr(B) TIMES expr(C). { A = B * C; }
+expr(A) ::= expr(B) DIVIDE expr(C). {
+  if (C != 0) {
+    A = B / C;
+  } else {
+    throw new Error("Divide by zero");
+  }
+}
+expr(A) ::= INTEGER(B). { A = B; }