@willbooster/tree-sitter-dart 0.0.0-stage → 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kasper Overgård Nielsen (kasper@byolimit.com)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,143 @@
-# Temporary Holding Version
+# @willbooster/tree-sitter-dart
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![npm version](https://img.shields.io/npm/v/@willbooster/tree-sitter-dart.svg)](https://www.npmjs.com/package/@willbooster/tree-sitter-dart)
+[![license](https://img.shields.io/npm/l/@willbooster/tree-sitter-dart.svg)](https://www.npmjs.com/package/@willbooster/tree-sitter-dart)
+[![Test](https://github.com/WillBooster/tree-sitter-dart/actions/workflows/test.yml/badge.svg)](https://github.com/WillBooster/tree-sitter-dart/actions/workflows/test.yml)
+[![Test rust](https://github.com/WillBooster/tree-sitter-dart/actions/workflows/test-rust.yml/badge.svg)](https://github.com/WillBooster/tree-sitter-dart/actions/workflows/test-rust.yml)
+[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg)](https://github.com/semantic-release/semantic-release)
+[![wbfy](https://img.shields.io/badge/wbfy-20.36.11-1e90ff.svg)](https://github.com/WillBooster/shared/tree/main/packages/wbfy)
+[![crates.io](https://img.shields.io/crates/v/willbooster-tree-sitter-dart.svg)](https://crates.io/crates/willbooster-tree-sitter-dart)
+
+Dart grammar for [tree-sitter](https://github.com/tree-sitter/tree-sitter), forked from
+[nielsenko/tree-sitter-dart](https://github.com/nielsenko/tree-sitter-dart). We are grateful
+to its authors and contributors. This is not an official release of that project.
+
+This fork keeps the grammar and the external scanner of upstream's version 0.2.0 and generates the parser with the
+[WillBooster/tree-sitter](https://github.com/WillBooster/tree-sitter) generator. The parser carries that fork's own ABI number (`0x57420001`), so it loads only with the fork's runtime,
+version 2.0.0 or later; upstream's runtime rejects it, and the fork's runtime rejects parsers generated by upstream's CLI.
+
+## Differences from upstream's parser
+
+Valid Dart parses to the same trees as with upstream's 0.2.0 parser. Two differences are known:
+
+- Error recovery on invalid input can differ, because the WillBooster/tree-sitter generator extracts keywords per
+  lexer state. Of 13,595 real-world files (flutter/flutter, dart-lang/core, rrousselGit/riverpod, and part of the Dart
+  SDK), 3 files that are invalid on purpose recover into a different tree, as do 2 of 10,197 truncated files; the error
+  flag is the same in every case. Their minimal inputs are `void main() {\n  print(\n        "foo" !==\n      null);`,
+  `test_nullAssert_insideCast(x) {\n  switch (x) {\n    case _! as num?:`, and
+  `    final v3 = (buffer.(p) {\n      return p == buffer;`.
+- A NUL character in a line comment or between tokens parses as ordinary text, where upstream's lexer reports an error.
+
+## Usage
+
+The npm package ships `tree-sitter-dart.wasm` for
+[@willbooster/web-tree-sitter](https://www.npmjs.com/package/@willbooster/web-tree-sitter) version 2.0.0 or later, which runs in Node.js, Bun, browsers, and Cloudflare Workers. In Node.js and Bun, load the grammar from its path:
+
+```js
+import { fileURLToPath } from 'node:url';
+import { Language, Parser } from '@willbooster/web-tree-sitter';
+
+await Parser.init();
+const parser = new Parser();
+const wasmPath = fileURLToPath(import.meta.resolve('@willbooster/tree-sitter-dart/tree-sitter-dart.wasm'));
+parser.setLanguage(await Language.load(wasmPath));
+const tree = parser.parse('void main() {}\n');
+```
+
+In browsers, serve both `.wasm` files and load them by URL (the example uses Vite's `?url` imports):
+
+```js
+import { Language, Parser } from '@willbooster/web-tree-sitter';
+import runtimeUrl from '@willbooster/web-tree-sitter/web-tree-sitter.wasm?url';
+import grammarUrl from '@willbooster/tree-sitter-dart/tree-sitter-dart.wasm?url';
+
+await Parser.init({ locateFile: () => runtimeUrl });
+const parser = new Parser();
+parser.setLanguage(await Language.load(grammarUrl));
+```
+
+Cloudflare Workers do not allow compiling Wasm at run time, so import both `.wasm` files as modules, with or without
+Node.js compatibility:
+
+```js
+import { Language, Parser } from '@willbooster/web-tree-sitter';
+import runtime from '@willbooster/web-tree-sitter/web-tree-sitter.wasm';
+import grammar from '@willbooster/tree-sitter-dart/tree-sitter-dart.wasm';
+
+await Parser.init({ wasmModule: runtime });
+const parser = new Parser();
+parser.setLanguage(await Language.load(grammar));
+```
+
+The package also ships `grammar.js`, the queries in `queries/`, and the node types in `src/node-types.json`.
+
+In Rust, depend on the [crate](https://crates.io/crates/willbooster-tree-sitter-dart) and on
+[willbooster-tree-sitter](https://crates.io/crates/willbooster-tree-sitter), the runtime this package is tested and
+fuzzed with:
+
+```toml
+[dependencies]
+tree-sitter = { package = "willbooster-tree-sitter", version = "2.0.0" }
+tree-sitter-dart = { package = "willbooster-tree-sitter-dart", version = "1" }
+```
+
+```rust
+let mut parser = tree_sitter::Parser::new();
+parser.set_language(&tree_sitter_dart::LANGUAGE.into())?;
+```
+
+## Development
+
+```sh
+mise install
+bun install --frozen-lockfile
+bun run test/ci-setup # installs Chromium for the browser test
+bun run build/ci
+bun run test
+script/parse-examples
+cargo test
+```
+
+The scripts and tests generate, build, test, and parse with `script/tree-sitter`, the tree-sitter CLI of the
+WillBooster/tree-sitter runtime version locked in `Cargo.lock`, since the generator and the runtime of upstream's CLI are
+not the ones this package ships with. Its first run downloads that CLI from the runtime's GitHub Release, or builds it
+with `cargo` (whose build runs the CMake that `mise.toml` pins) when the download fails or the release has no binary that runs here. Run other CLI commands through it as
+well (e.g. `script/tree-sitter parse --grammar-path . file.dart`).
+
+`bun run generate` records a fresh generation profile from the applicable `test/corpus` cases, then generates
+compact parser tables. It would also profile files tracked in `examples/`, but this repository tracks none: the
+directory holds only the clones `script/parse-examples` makes. The parser also embeds metadata from `tree-sitter.json`.
+After changing a grammar, `tree-sitter.json`, or a corpus case, regenerate and commit `src/`.
+Profiles in `.tmp/generation-profiles/` are temporary and must not be committed. `bun run build-wasm`, `bun run build/ci`,
+and the release build regenerate the parsers before compiling them.
+
+`bun run test` runs:
+
+- the corpus in `test/corpus`, with the native build and with the Wasm build (the first run downloads the WASI SDK);
+- an incremental-parsing check (`test/unit/incremental.test.ts`): `script/fuzz-corpus` runs `tree-sitter fuzz`, which
+  edits each corpus case at random, reparses it, undoes the edits, and reparses again. `TREE_SITTER_SEED`,
+  `TREE_SITTER_ITERATIONS`, and `TREE_SITTER_EDITS` run other or more edits;
+- a check that the real-world Dart files in `examples/` fail to parse exactly as listed in
+  `script/known-failures.txt`, which is empty now. The first run clones the repositories. The example repositories are pinned to commits in
+  `script/parse-examples`. After a grammar change or a moved pin alters that list, `script/parse-examples` rewrites it;
+  review its diff before committing;
+- a performance check (`test/unit/performance.test.ts`) that recovering from an error on each line takes linear
+  time, since consumers parse files while they are being edited. It loads the Wasm build through
+  @willbooster/web-tree-sitter, which `bun run build/ci` rebuilds after regenerating the parser;
+- a check (`test/unit/runtimeVersion.test.ts`) that `@willbooster/web-tree-sitter` in `package.json` and
+  `willbooster-tree-sitter` in `Cargo.lock` are the same version, since the Wasm tests run on the former and the Rust
+  tests, the fuzzer, and the CLI on the latter;
+- tests that load the Wasm build with @willbooster/web-tree-sitter in Chromium (`test/e2e/browser.test.ts`) and in
+  Cloudflare Workers with and without Node.js compatibility (`test/e2e/workers.test.ts`).
+
+The tests and `script/parse-examples` compile the parser into `.tmp/tree-sitter-lib` rather than the CLI's cache shared
+by every checkout; `script/fuzz-corpus` builds a parser of its own in `.tmp/fuzz` for each run and deletes it
+afterwards.
+
+CI also runs these tests on Linux arm64 and macOS, where the CLI compiles the parser and the scanner with the platform's
+C compiler as the crate does for its users, and fuzzes the parser with libFuzzer and sanitizers
+(`.github/workflows/robustness.yml`).
+
+### References
+
+- [Dart language specification](https://dart.dev/guides/language/spec)