@willbooster/tree-sitter-ruby 0.0.0-stage → 1.0.0

package/LICENSE

@@ -0,0 +1,23 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Rob Rix
+Copyright (c) 2026 WillBooster Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

package/README.md

@@ -1,3 +1,129 @@
-# Temporary Holding Version
+# @willbooster/tree-sitter-ruby
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![Test](https://github.com/WillBooster/tree-sitter-ruby/actions/workflows/test.yml/badge.svg)](https://github.com/WillBooster/tree-sitter-ruby/actions/workflows/test.yml)
+[![Test rust](https://github.com/WillBooster/tree-sitter-ruby/actions/workflows/test-rust.yml/badge.svg)](https://github.com/WillBooster/tree-sitter-ruby/actions/workflows/test-rust.yml)
+[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg)](https://github.com/semantic-release/semantic-release)
+[![wbfy](https://img.shields.io/badge/wbfy-20.36.6-1e90ff.svg)](https://github.com/WillBooster/shared/tree/main/packages/wbfy)
+[![crates.io](https://img.shields.io/crates/v/willbooster-tree-sitter-ruby.svg)](https://crates.io/crates/willbooster-tree-sitter-ruby)
+
+Ruby grammar for [tree-sitter](https://github.com/tree-sitter/tree-sitter), forked from
+[tree-sitter/tree-sitter-ruby](https://github.com/tree-sitter/tree-sitter-ruby). We are grateful
+to its authors and contributors. This is not an official release of that project.
+
+This fork ships upstream's grammar with a parser generated by
+[WillBooster/tree-sitter](https://github.com/WillBooster/tree-sitter), whose compact ABI 16 parse tables make the
+parser smaller. The scanner differs from upstream's only where upstream's code aborts the process: when a heredoc word is
+longer than 255 bytes, or when the scanner state with the open heredocs would not fit the runtime's serialization
+buffer, the open heredocs are not recognized and their bodies are parsed as code.
+
+## Usage
+
+The npm package ships `tree-sitter-ruby.wasm` for
+[@willbooster/web-tree-sitter](https://www.npmjs.com/package/@willbooster/web-tree-sitter), which runs in Node.js, Bun,
+browsers, and Cloudflare Workers. The compact ABI 16 parser requires runtime 1.3.0 or later.
+
+In Node.js and Bun, load the grammar from its path:
+
+```js
+import { fileURLToPath } from 'node:url';
+import { Language, Parser } from '@willbooster/web-tree-sitter';
+
+await Parser.init();
+const parser = new Parser();
+const wasmPath = fileURLToPath(import.meta.resolve('@willbooster/tree-sitter-ruby/tree-sitter-ruby.wasm'));
+parser.setLanguage(await Language.load(wasmPath));
+const tree = parser.parse('puts "Hello"\n');
+```
+
+In browsers, load the grammar from its URL. With Vite:
+
+```js
+import { Language, Parser } from '@willbooster/web-tree-sitter';
+import runtimeUrl from '@willbooster/web-tree-sitter/web-tree-sitter.wasm?url';
+import rubyUrl from '@willbooster/tree-sitter-ruby/tree-sitter-ruby.wasm?url';
+
+await Parser.init({ locateFile: () => runtimeUrl });
+const parser = new Parser();
+parser.setLanguage(await Language.load(rubyUrl));
+```
+
+In Cloudflare Workers, which do not allow compiling Wasm at run time, import both `.wasm` files as modules:
+
+```js
+import { Language, Parser } from '@willbooster/web-tree-sitter';
+import runtime from '@willbooster/web-tree-sitter/web-tree-sitter.wasm';
+import ruby from '@willbooster/tree-sitter-ruby/tree-sitter-ruby.wasm';
+
+await Parser.init({ wasmModule: runtime });
+const parser = new Parser();
+parser.setLanguage(await Language.load(ruby));
+```
+
+The package also ships the queries in `queries/` and the node types in `src/node-types.json`.
+
+In Rust, depend on the [crate](https://crates.io/crates/willbooster-tree-sitter-ruby) and on
+[willbooster-tree-sitter](https://crates.io/crates/willbooster-tree-sitter), the runtime this package is tested and
+fuzzed with. The compact ABI 16 parser requires runtime 1.3.0 or later:
+
+```toml
+[dependencies]
+tree-sitter = { package = "willbooster-tree-sitter", version = "1.3.0" }
+tree-sitter-ruby = { package = "willbooster-tree-sitter-ruby", version = "1" }
+```
+
+```rust
+let mut parser = tree_sitter::Parser::new();
+parser.set_language(&tree_sitter_ruby::LANGUAGE.into())?;
+```
+
+## Development
+
+```sh
+mise install
+bun install --frozen-lockfile
+bun run build/ci
+bun run test
+script/parse-examples
+cargo test
+```
+
+The scripts and tests generate, build, test, and parse with `script/tree-sitter`, the tree-sitter CLI of the
+WillBooster/tree-sitter runtime version locked in `Cargo.lock`, since the generator and the runtime of upstream's CLI are
+not the ones this package ships with. Its first run downloads that CLI from the runtime's GitHub Release, or builds it
+with `cargo` (whose build runs the CMake that `mise.toml` pins) when the download fails or the release has no binary
+that runs here. Run other CLI commands through it as well (e.g. `script/tree-sitter parse file.rb`).
+
+`bun run generate` records a fresh ABI 16 generation profile from the applicable `test/corpus` cases and Git-tracked
+files in `examples/`, then generates compact parser tables. The parser also embeds metadata from `tree-sitter.json`.
+After changing the grammar, `tree-sitter.json`, a corpus case, or a tracked example, regenerate and commit `src/`. Stage
+added or removed examples with `git add -A examples` before generation so the profile uses the intended file list.
+Profiles in `.tmp/generation-profiles/` are temporary and must not be committed. `bun run build-wasm`,
+`bun run build/ci`, and the release build regenerate the parser before compiling it.
+
+`bun run test` runs:
+
+- the corpus in `test/corpus`, with the native build and with the Wasm build (the first run downloads the WASI SDK);
+- an incremental-parsing check (`test/unit/incremental.test.ts`): `script/fuzz-corpus` runs `tree-sitter fuzz`, which
+  edits each corpus case at random, reparses it, undoes the edits, and reparses again. `TREE_SITTER_SEED`,
+  `TREE_SITTER_ITERATIONS`, and `TREE_SITTER_EDITS` run other or more edits;
+- a check that the real-world Ruby files of [ruby/spec](https://github.com/ruby/spec) cloned into `examples/` fail to
+  parse exactly as listed in `script/known-failures.txt`. The first run clones the repository, which is pinned to a
+  commit in `script/parse-examples`. After a grammar change or a moved pin alters that list, `script/parse-examples`
+  rewrites it; review its diff before committing;
+- a performance check (`test/unit/performance.test.ts`) that recovering from an error on each line takes linear time
+  (ten times the lines take about ten times the CPU time, under a ceiling), since consumers parse files while they are
+  being edited. It loads the Wasm build through @willbooster/web-tree-sitter, which `bun run build/ci` rebuilds after
+  regenerating the parser;
+- a check that `package.json` and `Cargo.lock` test the same runtime version (`test/unit/runtimeVersion.test.ts`);
+- checks that the Wasm build parses in Chromium (`test/unit/browser/`) and in Cloudflare Workers with and without
+  Node.js compatibility (`test/unit/workerd.test.ts`). Run `bun run test/ci-setup` once to install Chromium.
+
+The tests and `script/parse-examples` compile the parser into `.tmp/tree-sitter-lib` rather than the CLI's cache shared
+by every checkout; `script/fuzz-corpus` builds a per-run parser in `.tmp/fuzz` and deletes it afterwards.
+
+CI also runs these tests on Linux arm64 and macOS, where the Rust binding compiles the parser natively, and fuzzes the
+parser with libFuzzer and sanitizers (`.github/workflows/robustness.yml`).
+
+### References
+
+- [AST Format of the Whitequark parser](https://github.com/whitequark/parser/blob/master/doc/AST_FORMAT.md)