tree-sitter-al 0.0.0__tar.gz

tree_sitter_al-0.0.0/.claude/rules/attributes.md

@@ -0,0 +1,45 @@
+# Attribute Handling
+
+## Architecture
+
+Attributes are first-class statements using the Rust/C# pattern.
+
+**Key points:**
+- Attributes appear as `attribute_item` nodes in the parse tree
+- They are siblings to declarations, not nested within them
+- Preprocessor directives can appear between attributes and declarations
+- Semantic analysis (post-parse) associates attributes with following declarations
+
+## Parse Tree Structure
+
+```
+(attribute_item
+  attribute: (attribute_content ...))
+(procedure ...)
+```
+
+Attributes are separate nodes at the same level as declarations.
+
+## Valid Patterns
+
+```al
+[Scope('OnPrem')]
+[IntegrationEvent(false, false)]
+procedure MyEvent() begin end;
+
+[Obsolete('Use NewVersion', '24.0')]
+#if not CLEAN24
+procedure OldVersion() begin end;
+#endif
+
+// Attributes on parameters
+procedure Test([Mandatory] Param: Integer) begin end;
+
+// Attributes on enum values
+[Caption('Active')]
+value(0; Active) { }
+
+// Attributes on variables (in var sections)
+[NonDebuggable]
+MyVar: Integer;
+```

tree_sitter_al-0.0.0/.claude/rules/contextual-keywords.md

@@ -0,0 +1,37 @@
+# Contextual Keywords
+
+## V2 Architecture
+
+In V2, most property/variable disambiguation is handled by the `PROPERTY_NAME` scanner token, not by keyword-level tricks. The scanner distinguishes `identifier =` (property) from `identifier :` (variable).
+
+## Keyword-as-Identifier
+
+Some AL keywords are used as identifiers in certain contexts (e.g., `field`, `key`, `value`, `filter`, `action`, `type`, `version`). These are handled by the `keyword_as_identifier` rule:
+
+```javascript
+keyword_as_identifier: $ => choice(
+  'field', 'key', 'value', 'separator', 'dataset', 'type', 'version', 'action'
+),
+```
+
+When adding new keywords that can also be identifiers, add them to this choice list.
+
+## Named Keywords
+
+80 keywords are named rules for query matching. Use `kw()` (regex) for simple keywords:
+
+```javascript
+if_keyword: $ => kw('if'),
+```
+
+For CamelCase compound keywords, use explicit case variants:
+
+```javascript
+controladdin_keyword: $ => prec(10, choice(
+  'controladdin', 'CONTROLADDIN', 'Controladdin', 'ControlAddIn'
+)),
+```
+
+## begin/end Limitation
+
+`begin`/`end` MUST stay as anonymous `kw()` tokens. Any naming mechanism (named rules, alias named, alias anonymous) changes the token type and breaks GLR backtracking in preprocessor-split constructs. This is a fundamental tree-sitter limitation.

tree_sitter_al-0.0.0/.claude/rules/debugging.md

@@ -0,0 +1,50 @@
+# Parse Failure Debugging
+
+## Quick Workflow
+
+```bash
+# 1. Parse with debug output
+tree-sitter parse file.al -d > debug.log 2>&1
+
+# 2. Analyze with bug finder
+python parse_bug_finder.py file.al debug.log
+```
+
+## Error Research Agent
+
+For systematic analysis of parse failures, use the `error-research` agent:
+- Analyzes failing files after validation reveals errors
+- Categorizes error patterns across multiple files
+- Identifies root causes and prioritizes fixes by impact
+
+## Batch Error Analysis
+
+```bash
+# Run full production parse
+./parse-al-parallel.sh ./BC.History/ .
+
+# Check error list
+cat ./BC.History/errors.txt
+
+# Parse specific error files
+head -10 ./BC.History/errors.txt | while IFS= read -r f; do
+  echo "=== $(basename "$f") ==="
+  tree-sitter parse "$f" 2>&1 | grep -E "ERROR|MISSING" | head -3
+done
+```
+
+## Debugging Process
+
+1. Isolate failing construct, test in minimal context
+2. Check if rule exists in grammar.js
+3. Test with precedence if parser state issues
+4. Use `parse_bug_finder.py` to analyze debug output
+5. Update expectations (`-u`) only if no ERROR/MISSING nodes
+
+## Philosophy: No Known Limitations
+
+**Never give up on a failing pattern:**
+- Don't disable tests or mark issues as "known limitations"
+- Research how other parsers handle similar constructs in `other-languages/`
+- Use `error-research` agent for systematic failure analysis
+- Every "impossible" pattern has been solved somewhere — find it and adapt it

tree_sitter_al-0.0.0/.claude/rules/scanner.md

@@ -0,0 +1,46 @@
+# External Scanner Rules
+
+The external scanner (`src/scanner.c`) handles patterns that can't be expressed in JavaScript grammar alone.
+
+## Scanner Tokens
+
+| Token | Purpose | Lookahead |
+|-------|---------|-----------|
+| `PROPERTY_NAME` | `identifier` followed by `=` (not `:=`) — disambiguates properties from variables | Skip whitespace after identifier, check next char |
+| `CONTINUE_AS_IDENTIFIER` | `continue` followed by `:=` — used as variable name | Check for `:=` after `continue` |
+
+## PROPERTY_NAME Token
+
+This is the V2 grammar's key architectural innovation. When the parser state allows both properties and variables, the scanner checks what follows the identifier:
+
+1. Match identifier regex (Unicode-aware)
+2. Skip whitespace
+3. If next char is `=` and NOT part of `:=` or `==`: emit `PROPERTY_NAME`
+4. Otherwise: don't match, let grammar handle as `identifier`
+
+**Critical constraint:** `PROPERTY_NAME` must never be in `valid_symbols` inside `var_section` or statement contexts. The grammar naturally ensures this because properties appear in object/section bodies, not in code blocks.
+
+## Adding Scanner Features
+
+1. Add token to `TokenType` enum in `src/scanner.c`
+2. Add token to `externals` array in `grammar.js`
+3. Implement lookahead logic in `tree_sitter_al_external_scanner_scan`
+4. Create grammar rules using the token
+5. Test with edge cases
+
+## Debugging
+
+Enable debug output:
+```c
+#define SCANNER_DEBUG 1
+```
+
+Trace what the parser is asking for:
+```c
+if (SCANNER_DEBUG) {
+    fprintf(stderr, "SCANNER: valid_symbols PROPERTY_NAME=%d CONTINUE=%d at '%c'\n",
+            valid_symbols[PROPERTY_NAME],
+            valid_symbols[CONTINUE_AS_IDENTIFIER],
+            (char)lexer->lookahead);
+}
+```

tree_sitter_al-0.0.0/.claude/rules/test-failures.md

@@ -0,0 +1,36 @@
+# Test Failure Patterns
+
+## Common Patterns
+
+| Pattern | Symptom | Fix |
+|---------|---------|-----|
+| **Missing construct** | ERROR nodes | Add rule to `_body_element` or relevant choice list |
+| **Case-sensitivity** | Keywords not matching | Use `kw()` or explicit `choice()` with case variants |
+| **Preprocessor splits** | MISSING tokens in #if contexts | Add dedicated `preproc_split_*` rule |
+| **Complex property syntax** | Property value fails to parse | Add dedicated complex property rule |
+| **Keyword as identifier** | Variable name conflicts with keyword | Add to `keyword_as_identifier` choice list |
+| **Structural mismatch** | Different node structure in tests | Update tests with `-u` if parsing is correct (no ERRORs) |
+
+## When to Update vs Fix
+
+| Action | Condition |
+|--------|-----------|
+| **Update tests** (`-u`) | Grammar correct, tree structure evolved, no ERROR/MISSING nodes |
+| **Fix grammar** | ERROR or MISSING nodes present in parse output |
+
+## Test Guidelines
+
+- **Never delete or disable test files** — fix the underlying issue
+- Use `tree-sitter test -u` only if no ERROR/MISSING nodes exist
+- **BC.History (15,358 files) is the real validation gate** — tests are a development aid
+- When stuck, study similar patterns in `other-languages/` parsers
+
+## Debugging
+
+```bash
+# Parse with debug output
+tree-sitter parse file.al -d > debug.log 2>&1
+
+# Analyze with bug finder
+python parse_bug_finder.py file.al debug.log
+```

tree_sitter_al-0.0.0/.editorconfig

@@ -0,0 +1,43 @@
+root = true
+
+[*]
+charset = utf-8
+
+[*.{json,toml,yml,gyp}]
+indent_style = space
+indent_size = 2
+
+[*.js]
+indent_style = space
+indent_size = 2
+
+[*.scm]
+indent_style = space
+indent_size = 2
+
+[*.{c,cc,h}]
+indent_style = space
+indent_size = 4
+
+[*.rs]
+indent_style = space
+indent_size = 4
+
+[*.{py,pyi}]
+indent_style = space
+indent_size = 4
+
+[*.swift]
+indent_style = space
+indent_size = 4
+
+[*.go]
+indent_style = tab
+indent_size = 8
+
+[Makefile]
+indent_style = tab
+indent_size = 8
+
+[parser.c]
+indent_size = 2

tree_sitter_al-0.0.0/.gitattributes

@@ -0,0 +1,13 @@
+* text=auto eol=lf
+
+src/*.json linguist-generated
+src/parser.c linguist-generated
+src/tree_sitter/* linguist-generated
+
+bindings/** linguist-generated
+binding.gyp linguist-generated
+setup.py linguist-generated
+Makefile linguist-generated
+CMakeLists.txt linguist-generated
+Package.swift linguist-generated
+go.mod linguist-generated

tree_sitter_al-0.0.0/.github/workflows/build-wasm.yml

@@ -0,0 +1,113 @@
+name: Build and release artifacts
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  build-shared-lib:
+    name: Build shared library (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        include:
+          - os: ubuntu-latest
+            artifact: tree-sitter-al.so
+            lib-glob: "*.so"
+          - os: windows-latest
+            artifact: tree-sitter-al.dll
+            lib-glob: "*.dll"
+          - os: macos-latest
+            artifact: tree-sitter-al.dylib
+            lib-glob: "*.dylib"
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up tree-sitter CLI
+        uses: tree-sitter/setup-action/cli@v2
+
+      - name: Add swap space (Linux only)
+        if: runner.os == 'Linux'
+        run: |
+          sudo swapoff /swapfile || true
+          sudo rm -f /swapfile
+          sudo fallocate -l 12G /swapfile
+          sudo chmod 600 /swapfile
+          sudo mkswap /swapfile
+          sudo swapon /swapfile
+
+      - name: Build shared library
+        run: tree-sitter build .
+
+      - name: Find and rename output
+        shell: bash
+        run: |
+          LIB=$(find . -maxdepth 1 -name '${{ matrix.lib-glob }}' | head -1)
+          echo "Found: $LIB"
+          mv "$LIB" "${{ matrix.artifact }}"
+
+      - name: Upload shared library artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.artifact }}
+          path: ${{ matrix.artifact }}
+
+  release:
+    name: Create GitHub Releases
+    needs: [build-shared-lib]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Download all artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: artifacts
+          merge-multiple: true
+
+      - name: Copy prebuilt WASM
+        run: cp tree-sitter-al.wasm artifacts/
+
+      - name: List artifacts
+        run: ls -la artifacts/
+
+      - name: Get version from package.json
+        id: version
+        run: |
+          VERSION=$(node -p "require('./package.json').version")
+          echo "version=$VERSION" >> "$GITHUB_OUTPUT"
+          echo "tag=v$VERSION" >> "$GITHUB_OUTPUT"
+          echo "Version: $VERSION (tag: v$VERSION)"
+
+      - name: Check if versioned release already exists
+        id: check
+        run: |
+          if gh release view "v${{ steps.version.outputs.version }}" >/dev/null 2>&1; then
+            echo "exists=true" >> "$GITHUB_OUTPUT"
+            echo "Versioned release v${{ steps.version.outputs.version }} already exists, skipping"
+          else
+            echo "exists=false" >> "$GITHUB_OUTPUT"
+          fi
+        env:
+          GH_TOKEN: ${{ github.token }}
+
+      - name: Create versioned release
+        if: steps.check.outputs.exists == 'false'
+        uses: softprops/action-gh-release@v2
+        with:
+          tag_name: ${{ steps.version.outputs.tag }}
+          name: ${{ steps.version.outputs.tag }}
+          body: |
+            ## tree-sitter-al ${{ steps.version.outputs.tag }}
+
+            | File | Platform | Use case |
+            |------|----------|----------|
+            | `tree-sitter-al.wasm` | All | web-tree-sitter |
+            | `tree-sitter-al.so` | Linux x86_64 | ast-grep, native bindings |
+            | `tree-sitter-al.dll` | Windows x86_64 | ast-grep, native bindings |
+            | `tree-sitter-al.dylib` | macOS ARM64 | ast-grep, native bindings |
+          files: artifacts/*
+          make_latest: true
+          prerelease: false

tree_sitter_al-0.0.0/.github/workflows/claude-code-review.yml

@@ -0,0 +1,44 @@
+name: Claude Code Review
+
+on:
+  pull_request:
+    types: [opened, synchronize, ready_for_review, reopened]
+    # Optional: Only run on specific file changes
+    # paths:
+    #   - "src/**/*.ts"
+    #   - "src/**/*.tsx"
+    #   - "src/**/*.js"
+    #   - "src/**/*.jsx"
+
+jobs:
+  claude-review:
+    # Optional: Filter by PR author
+    # if: |
+    #   github.event.pull_request.user.login == 'external-contributor' ||
+    #   github.event.pull_request.user.login == 'new-developer' ||
+    #   github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
+
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Run Claude Code Review
+        id: claude-review
+        uses: anthropics/claude-code-action@v1
+        with:
+          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
+          plugin_marketplaces: 'https://github.com/anthropics/claude-code.git'
+          plugins: 'code-review@claude-code-plugins'
+          prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }}'
+          # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
+          # or https://code.claude.com/docs/en/cli-reference for available options
+

tree_sitter_al-0.0.0/.github/workflows/claude.yml

@@ -0,0 +1,50 @@
+name: Claude Code
+
+on:
+  issue_comment:
+    types: [created]
+  pull_request_review_comment:
+    types: [created]
+  issues:
+    types: [opened, assigned]
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  claude:
+    if: |
+      (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
+      (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
+      (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
+      (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+      actions: read # Required for Claude to read CI results on PRs
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Run Claude Code
+        id: claude
+        uses: anthropics/claude-code-action@v1
+        with:
+          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
+
+          # This is an optional setting that allows Claude to read CI results on PRs
+          additional_permissions: |
+            actions: read
+
+          # Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
+          # prompt: 'Update the pull request description to include a summary of changes.'
+
+          # Optional: Add claude_args to customize behavior and configuration
+          # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
+          # or https://code.claude.com/docs/en/cli-reference for available options
+          # claude_args: '--allowed-tools Bash(gh pr:*)'
+

tree_sitter_al-0.0.0/.github/workflows/publish-pypi.yml

@@ -0,0 +1,58 @@
+name: Publish Python Package
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+jobs:
+  build-wheels:
+    name: Build wheels (${{ matrix.os }})
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install cibuildwheel
+      - run: cibuildwheel --output-dir dist
+      - uses: actions/upload-artifact@v4
+        with:
+          name: wheels-${{ matrix.os }}
+          path: dist/*.whl
+
+  build-sdist:
+    name: Build sdist
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install build
+      - run: python -m build --sdist
+      - uses: actions/upload-artifact@v4
+        with:
+          name: sdist
+          path: dist/*.tar.gz
+
+  publish:
+    name: Publish to PyPI
+    needs: [build-wheels, build-sdist]
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          path: dist
+          merge-multiple: true
+      - uses: pypa/gh-action-pypi-publish@release/v1

tree_sitter_al-0.0.0/.gitignore

@@ -0,0 +1,110 @@
+# Rust artifacts
+target/
+
+# Node artifacts
+build/
+prebuilds/
+node_modules/
+
+# Swift artifacts
+.build/
+
+# Go artifacts
+_obj/
+
+# Python artifacts
+.venv/
+dist/
+*.egg-info
+*.whl
+
+# C artifacts
+*.a
+*.so
+*.so.*
+*.dylib
+*.dll
+*.pc
+
+# Example dirs
+/examples/*/
+
+# Grammar volatiles
+*.wasm
+!tree-sitter-al.wasm
+*.obj
+*.o
+
+# Archives
+*.tar.gz
+*.tgz
+*.zip
+.aider*
+.env
+/env
+/developer
+/.vscode
+*.al
+*.ps1
+*.log
+log.html
+grammarPascal.js
+
+/memory-bank/*
+
+DC/*
+
+# Analysis and utility scripts
+*.py
+# But allow keyword-sync tools
+!tools/keyword-sync/*.py
+!tools/keyword-sync/output/
+tools/keyword-sync/__pycache__/
+current_orphans.txt
+orphan_report.txt
+run_query*.sh
+# Ignore queries/ except standard tree-sitter query files
+queries/*
+!queries/tags.scm
+!queries/highlights.scm
+!queries/locals.scm
+!queries/indents.scm
+!queries/folds.scm
+FIXES.md
+
+# AI assistant files
+.claude/
+.clinerules/
+.goosehints
+CLAUDE.md
+settings.local.json
+
+# Development and analysis files
+*PLAN*.md
+*SUMMARY*.md
+*MIGRATION*.md
+*MERGE*.md
+*SOLUTION*.md
+*CONVENTION*.md
+*MISSING*.md
+*QUERIES*.md
+*RULES*.md
+*GUIDE*.md
+property_*.md
+property_*.txt
+all_properties.txt
+context_mapping.txt
+migration_strategy.md
+validation_report.md
+orphan_*.txt
+
+# Build artifacts
+*.pdb
+*.exp
+*.lib
+al_testfiles/parsed.txt
+
+# Backup files
+bak/
+
+BC.History/