@sshadows/tree-sitter-al 2.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Torben Leth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,127 @@
+# tree-sitter-al
+
+A [tree-sitter](https://tree-sitter.github.io/tree-sitter/) parser for the AL programming language used in Microsoft Dynamics 365 Business Central.
+
+[![PyPI](https://img.shields.io/pypi/v/tree-sitter-al)](https://pypi.org/project/tree-sitter-al/)
+
+## Parser Status
+
+Validated against **15,358 production AL files** from the Business Central codebase:
+
+| Metric | Value |
+|--------|-------|
+| **Success rate** | **99.95%** (15,351 / 15,358 files) |
+| Tests | 1,404 |
+| parser.c size | 10.6 MB |
+| grammar.js | ~3,000 lines |
+| Named keywords | 80 (queryable via highlights/tags) |
+| Query files | 5 (highlights, locals, tags, indents, folds) |
+
+## Installation
+
+### Python (tree-sitter 0.24+)
+
+```bash
+pip install tree-sitter-al
+```
+
+```python
+import tree_sitter
+import tree_sitter_al
+
+lang = tree_sitter.Language(tree_sitter_al.language())
+parser = tree_sitter.Parser(lang)
+tree = parser.parse(b'codeunit 50100 MyCodeunit { }')
+print(tree.root_node.sexp())
+```
+
+### Node.js
+
+```bash
+npm install tree-sitter-al
+```
+
+### Pre-built binaries
+
+Download from [GitHub Releases](https://github.com/SShadowS/tree-sitter-al/releases):
+
+| File | Platform | Use case |
+|------|----------|----------|
+| `tree-sitter-al.wasm` | All | web-tree-sitter |
+| `tree-sitter-al.so` | Linux x86_64 | ast-grep, native bindings |
+| `tree-sitter-al.dll` | Windows x86_64 | ast-grep, native bindings |
+| `tree-sitter-al.dylib` | macOS ARM64 | ast-grep, native bindings |
+
+## V2 Architecture
+
+The grammar was rewritten from scratch in March 2026, achieving a **10x reduction in parser size** while improving correctness.
+
+### Before / After
+
+| Metric | V1 | V2 |
+|--------|-----|-----|
+| parser.c | 106 MB (can't push to GitHub) | **10.6 MB** |
+| Errors | 14 | **7** |
+| Success rate | 99.91% | **99.95%** |
+| Symbols | 2,249 | **724** |
+| States | 29,126 | **5,179** |
+| grammar.js | 8,500 lines | **~3,000 lines** |
+| Tests | 1,225 | **1,404** |
+| Keywords | invisible in queries | **80 named nodes** |
+| Query files | 3 (partial) | **5 (comprehensive)** |
+
+### Key design decisions
+
+- **Scanner-based property disambiguation** — One `PROPERTY_NAME` scanner token replaces 291 individual property rules. The scanner checks if `identifier` is followed by `=` (not `:=`) with a single-character lookahead.
+- **Parse structure, don't validate** — Accept any `Name = Value ;` as a property. Semantic validation belongs in linters/LSP servers, not the parser.
+- **Generic preprocessor** — One `preproc_conditional` rule + ~12 dedicated rules for genuinely complex split constructs, replacing 63 specialized rules.
+- **80 named keyword nodes** — All keywords except `begin`/`end` are named nodes, enabling proper syntax highlighting and code navigation queries.
+
+See [docs/v2-blog-post-notes.md](docs/v2-blog-post-notes.md) for the full rewrite narrative.
+
+## Development
+
+### Prerequisites
+
+- Node.js (v16+)
+- tree-sitter CLI: `npm install -g tree-sitter-cli`
+
+### Building
+
+```bash
+tree-sitter generate    # Generate parser from grammar.js
+tree-sitter test        # Run test suite
+```
+
+### Validation
+
+```bash
+./validate-grammar.sh        # Quick: generation, tests, orphan/duplicate detection
+./validate-grammar.sh --full # Full: includes production AL file parsing
+```
+
+### Parsing AL files
+
+```bash
+tree-sitter parse path/to/file.al
+tree-sitter parse path/to/file.al -d    # Debug output
+tree-sitter parse path/to/file.al -q    # Quiet (errors only)
+```
+
+### Key files
+
+| File | Purpose |
+|------|---------|
+| `grammar.js` | Main grammar definition |
+| `src/scanner.c` | External scanner (property disambiguation, preprocessor) |
+| `test/corpus/` | Test suite (1,404 tests) |
+| `queries/` | Syntax highlighting, code navigation, folding, indentation |
+
+## Contributing
+
+See [CLAUDE.md](CLAUDE.md) for detailed development guidelines including architecture, debugging, and conventions.
+
+---
+
+**Author**: Torben Leth (sshadows@sshadows.dk)
+**License**: MIT (see [LICENSE](LICENSE))

package/binding.gyp

@@ -0,0 +1,30 @@
+{
+  "targets": [
+    {
+      "target_name": "tree_sitter_al_binding",
+      "dependencies": [
+        "<!(node -p \"require('node-addon-api').targets\"):node_addon_api_except",
+      ],
+      "include_dirs": [
+        "src",
+      ],
+      "sources": [
+        "bindings/node/binding.cc",
+        "src/parser.c",
+        "src/scanner.c"
+      ],
+      "conditions": [
+        ["OS!='win'", {
+          "cflags_c": [
+            "-std=c11",
+          ],
+        }, { # OS == "win"
+          "cflags_c": [
+            "/std:c11",
+            "/utf-8",
+          ],
+        }],
+      ],
+    }
+  ]
+}

package/bindings/node/binding.cc

@@ -0,0 +1,20 @@
+#include <napi.h>
+
+typedef struct TSLanguage TSLanguage;
+
+extern "C" TSLanguage *tree_sitter_al();
+
+// "tree-sitter", "language" hashed with BLAKE2
+const napi_type_tag LANGUAGE_TYPE_TAG = {
+  0x8AF2E5212AD58ABF, 0xD5006CAD83ABBA16
+};
+
+Napi::Object Init(Napi::Env env, Napi::Object exports) {
+    exports["name"] = Napi::String::New(env, "al");
+    auto language = Napi::External<TSLanguage>::New(env, tree_sitter_al());
+    language.TypeTag(&LANGUAGE_TYPE_TAG);
+    exports["language"] = language;
+    return exports;
+}
+
+NODE_API_MODULE(tree_sitter_al_binding, Init)

package/bindings/node/index.d.ts

@@ -0,0 +1,28 @@
+type BaseNode = {
+  type: string;
+  named: boolean;
+};
+
+type ChildNode = {
+  multiple: boolean;
+  required: boolean;
+  types: BaseNode[];
+};
+
+type NodeInfo =
+  | (BaseNode & {
+      subtypes: BaseNode[];
+    })
+  | (BaseNode & {
+      fields: { [name: string]: ChildNode };
+      children: ChildNode[];
+    });
+
+type Language = {
+  name: string;
+  language: unknown;
+  nodeTypeInfo: NodeInfo[];
+};
+
+declare const language: Language;
+export = language;

package/bindings/node/index.js

@@ -0,0 +1,7 @@
+const root = require("path").join(__dirname, "..", "..");
+
+module.exports = require("node-gyp-build")(root);
+
+try {
+  module.exports.nodeTypeInfo = require("../../src/node-types.json");
+} catch (_) {}