tree-sitter-ucode 0.1.2 → 0.3.0

package/README.md

@@ -7,7 +7,12 @@ Two grammars are provided:
 | Grammar | Scope | File types |
 |---------|-------|------------|
 | `ucode` | `source.uc` | `.uc` |
-| `ucode_tmpl` | `source.uc.tmpl` | `.uc.tmpl`, `.utpl` |
+| `ucode_tmpl` | `source.uc.tmpl` | `.uc` (template files — detected by content) |
+
+Both grammars use the `.uc` extension. Template files are distinguished from plain
+code files by content: any `.uc` file whose first tag opener (`{%`, `{{`, or `{#`)
+appears at the start of a line is automatically parsed by `ucode_tmpl`. Plain code
+files fall back to `ucode`. See [File-type detection](#file-type-detection) below.
 
 ## Ucode vs JavaScript
 
@@ -17,13 +22,12 @@ Ucode is an ECMAScript subset with OpenWrt-specific extensions. Key differences:
 |---------|-------|------------|
 | Alternative block syntax | `if/elif/else/endif`, `for/endfor`, `while/endwhile`, `function/endfunction` | Not supported |
 | Two-variable for-in | `for (k, v in obj)` | Single variable only |
-| Forward declaration | `function foo;` | Not supported |
-| Removed keywords | `var`, `new`, `typeof`, `void`, `class`, `instanceof`, `do`, `async`, `await`, `yield` | All supported |
-| Removed features | Destructuring, `for...of`, `do-while`, generators | All supported |
+| Removed keywords | `var`, `new`, `throw`, `typeof`, `void`, `class`, `instanceof`, `do`, `async`, `await`, `yield` | All supported |
+| Removed features | Destructuring, `for...of`, `do-while`, generators, forward declarations, dynamic `import()` | All supported |
 | Added number literals | `0177` (C octal), `0x1.8` (hex float), `0B`/`0O` prefixes | Standard only |
 | Added escape sequences | `\e` (ESC), `\a` (BEL), octal `\177` | Standard only |
 | Regex flags | `g`, `i`, `s` only | Full set |
-| Module system | Static `import`/`export`, dynamic `import(path)`; no `from` on re-exports | Full ES modules |
+| Module system | Static `import`/`export` only; no `from` on re-exports | Full ES modules |
 
 ## Requirements
 
@@ -56,10 +60,18 @@ npm test             # runs tree-sitter test for ucode and ucode_tmpl
 To filter by corpus file name:
 
 ```sh
-npx tree-sitter test --file-name control_flow
+npx tree-sitter test --lib-path ucode.so --lang-name ucode --file-name control_flow
 npx tree-sitter test -p tmpl --file-name template
 ```
 
+## File-type detection
+
+Both grammars claim the `.uc` extension. Tools that respect `content-regex` in
+`tree-sitter.json` (including the tree-sitter CLI ≥ 0.24) automatically route
+template files to `ucode_tmpl` when a tag opener appears at the start of a line.
+Editors that manage their own filetype dispatch (Neovim, Helix) need an explicit
+rule — see the editor sections below.
+
 ## Use in Neovim (nvim-treesitter)
 
 Add to your nvim-treesitter config (e.g. `~/.config/nvim/lua/plugins/treesitter.lua`):
@@ -86,15 +98,26 @@ parser_config.ucode_tmpl = {
 }
 ```
 
-Associate `.uc` and `.uc.tmpl` files with the right filetypes:
+Associate `.uc` files with the right filetype. Since nvim-treesitter does not
+use `content-regex`, the mapping must be done with a `BufRead` autocmd or a
+filetype function that inspects the file content:
 
 ```lua
 vim.filetype.add({
   extension = {
-    uc   = "ucode",
-    utpl = "ucode_tmpl",
+    uc = function(path)
+      -- Files whose first tag opener is at the start of a line are templates
+      local f = io.open(path, "r")
+      if f then
+        local content = f:read("*a")
+        f:close()
+        if content:find("^%s*{[%%{#]", 1, false) or content:find("\n%s*{[%%{#]") then
+          return "ucode_tmpl"
+        end
+      end
+      return "ucode"
+    end,
   },
-  pattern = { [".*%.uc%.tmpl"] = "ucode_tmpl" },
 })
 ```
 
@@ -104,32 +127,38 @@ Add to `~/.config/helix/languages.toml`:
 
 ```toml
 [[language]]
-name        = "ucode"
-scope       = "source.uc"
-file-types  = ["uc"]
+name          = "ucode"
+scope         = "source.uc"
+file-types    = [{ glob = "*.uc" }]
 comment-token = "//"
-indent      = { tab-width = 2, unit = "  " }
-grammar     = "ucode"
+indent        = { tab-width = 2, unit = "  " }
+grammar       = "ucode"
 
 [[language]]
-name        = "ucode-tmpl"
-scope       = "source.uc.tmpl"
-file-types  = ["uc.tmpl", "utpl"]
-grammar     = "ucode_tmpl"
+name          = "ucode-tmpl"
+scope         = "source.uc.tmpl"
+comment-token = "{#"
+indent        = { tab-width = 2, unit = "  " }
+grammar       = "ucode_tmpl"
 
 [[grammar]]
 name   = "ucode"
-source = { git = "https://github.com/m00qek/tree-sitter-ucode", rev = "main" }
+source = { git = "https://github.com/m00qek/tree-sitter-ucode", rev = "v0.3.0" }
 
 [[grammar]]
 name   = "ucode_tmpl"
-source = { git = "https://github.com/m00qek/tree-sitter-ucode", rev = "main", subpath = "tmpl" }
+source = { git = "https://github.com/m00qek/tree-sitter-ucode", rev = "v0.3.0", subpath = "tmpl" }
 ```
 
-## Template files (.uc.tmpl / .utpl)
+Helix does not support content-based filetype detection. To open a `.uc`
+template file as `ucode-tmpl`, use `:set-language ucode-tmpl` in command mode,
+or configure a file-specific override via a `.helix/languages.toml` in your
+project.
+
+## Template files
 
-Template files mix raw text with code tags. The `ucode_tmpl` grammar produces
-a document tree; editors use language injection to apply ucode highlighting
+Template files mix raw text with code tags. The `ucode_tmpl` grammar produces a
+`document` tree; editors use language injection to apply ucode highlighting
 inside the code and expression tags.
 
 | Tag | Purpose |
@@ -142,7 +171,17 @@ inside the code and expression tags.
 | `{%+ ... %}` | Statement block — suppress `lstrip_blocks` stripping |
 | `{#- ... -#}` | Comment — strip whitespace on both sides |
 
-Opener and closer markers are independent: any opener variant may be combined with any closer variant. `{%-` / `{{-` / `{#-` strip the preceding raw text; `-%}` / `-}}` / `-#}` strip the following raw text. `{%+` suppresses `lstrip_blocks` stripping and may be combined with `-%}` (e.g. `{%+ ... -%}`).
+Opener and closer markers are independent: any opener variant may be combined
+with any closer variant. `{%-` / `{{-` / `{#-` strip the preceding raw text;
+`-%}` / `-}}` / `-#}` strip the following raw text. `{%+` suppresses
+`lstrip_blocks` stripping and may be combined with `-%}`.
+
+**EOF-implicit close**: a `{%` statement tag that reaches end-of-file without
+an explicit `%}` is treated as implicitly closed. This supports the common
+OpenWrt pattern of a file that opens one `{%` block at the top and contains
+only ucode code, with no closing `%}`. The parse tree shows an `(eof_close)`
+node in the `close` field. Expression (`{{`) and comment (`{#`) tags still
+require explicit closers.
 
 Example:
 

package/grammar.js

@@ -46,6 +46,7 @@ module.exports = grammar({
       'if',
       'import',
       'in',
+      'let',
       'null',
       'return',
       'switch',
@@ -99,7 +100,6 @@ module.exports = grammar({
     ['member', 'call', $.expression],
     ['declaration', 'literal'],
     [$.primary_expression, $.statement_block, 'object'],
-    [$.import_statement, $.import],
     [$.export_statement, $.primary_expression],
     [$.lexical_declaration, $.primary_expression],
   ],
@@ -143,7 +143,7 @@ module.exports = grammar({
             'default',
             seq(
               field('value', $.expression),
-              $._semicolon,
+              ';',
             ),
           ),
         ),
@@ -173,18 +173,14 @@ module.exports = grammar({
 
     declaration: $ => choice(
       $.function_declaration,
-      $.function_forward_declaration,
       $.lexical_declaration,
     ),
 
     //
     // Import declarations
     // Ucode keeps full ES module static import syntax.
-    // Also adds dynamic `import(path)` as a unary expression (see import_expression).
     //
 
-    import: _ => token('import'),
-
     import_statement: $ => seq(
       'import',
       choice(
@@ -461,7 +457,6 @@ module.exports = grammar({
       $.binary_expression,
       $.ternary_expression,
       $.update_expression,
-      $.import_expression,
     ),
 
     primary_expression: $ => choice(
@@ -485,14 +480,6 @@ module.exports = grammar({
       $.call_expression,
     ),
 
-    // Dynamic import expression: `let m = import("./mod.uc")`
-    import_expression: $ => prec('call', seq(
-      $.import,
-      '(',
-      field('source', $.expression),
-      ')',
-    )),
-
     object: $ => prec('object', seq(
       '{',
       commaSep(optional(choice(
@@ -662,13 +649,6 @@ module.exports = grammar({
       optional($._automatic_semicolon),
     )),
 
-    // Forward declaration: `function name;`
-    function_forward_declaration: $ => seq(
-      'function',
-      field('name', $.identifier),
-      ';',
-    ),
-
     arrow_function: $ => seq(
       choice(
         field('parameter', choice(
@@ -857,7 +837,6 @@ module.exports = grammar({
     _reserved_identifier: _ => choice(
       'get',
       'set',
-      'let',
     ),
 
     _semicolon: $ => choice($._automatic_semicolon, ';'),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tree-sitter-ucode",
-  "version": "0.1.2",
+  "version": "0.3.0",
   "description": "Ucode grammar for tree-sitter",
   "repository": {
     "type": "git",
@@ -35,7 +35,7 @@
   },
   "devDependencies": {
     "prebuildify": "^6.0.1",
-    "tree-sitter-cli": "^0.26.0"
+    "tree-sitter-cli": "0.26.9"
   },
   "peerDependencies": {
     "tree-sitter": "^0.22.0"
@@ -48,7 +48,7 @@
   "scripts": {
     "install": "node-gyp-build",
     "build": "tree-sitter generate && tree-sitter generate tmpl/grammar.js --output tmpl/src && node-gyp-build && tree-sitter build --output ucode.so . && tree-sitter build --output ucode_tmpl.so ./tmpl",
-    "test": "tree-sitter test && tree-sitter test -p tmpl",
+    "test": "node scripts/run-tests.js",
     "prestart": "tree-sitter build --wasm",
     "start": "tree-sitter playground"
   }

package/queries/folds.scm

@@ -0,0 +1,38 @@
+; Fold queries for ucode.
+; Each captured node becomes a collapsible fold region.
+
+; -------------------------------------------------------------------------
+; Brace-delimited blocks
+;
+; statement_block covers brace-body function_declaration, function_expression,
+; and arrow_function bodies — the signature line stays visible because the
+; fold starts at '{', not at the 'function' keyword.
+; -------------------------------------------------------------------------
+
+[
+  (statement_block)
+  (switch_body)
+  (object)
+  (array)
+] @fold
+
+; -------------------------------------------------------------------------
+; Alternative (colon/endif) syntax — fold the entire construct
+; -------------------------------------------------------------------------
+
+[
+  (if_alt_statement)
+  (for_alt_statement)
+  (for_in_alt_statement)
+  (while_alt_statement)
+] @fold
+
+; -------------------------------------------------------------------------
+; Colon-body function declaration (function f(): ... endfunction)
+;
+; Brace-body function declarations are folded via (statement_block) above.
+; The colon form has no statement_block, so we fold the whole declaration.
+; We filter on "endfunction" so this pattern only matches the colon form.
+; -------------------------------------------------------------------------
+
+(function_declaration "endfunction") @fold

package/queries/highlights.scm

@@ -53,9 +53,6 @@
 (function_declaration
   name: (identifier) @function)
 
-(function_forward_declaration
-  name: (identifier) @function)
-
 (function_expression
   name: (identifier) @function)
 
@@ -121,8 +118,12 @@
 
 ["import" "export"] @keyword.import
 ["as" "from"] @keyword.import
-; Dynamic import() uses a named node, not an anonymous string
-(import_expression (import) @keyword.import)
+
+; 'default' in export context is a module keyword, not a conditional.
+; These patterns are more specific than the generic ["switch" "case" "default"]
+; below, so they take priority for 'default' tokens inside export constructs.
+(export_statement "default" @keyword.import)
+(export_specifier "default" @keyword.import)
 
 ; -------------------------------------------------------------------------
 ; Keywords — storage / operators

package/queries/indents.scm

@@ -0,0 +1,63 @@
+; Indentation queries for ucode.
+; Capture names follow the nvim-treesitter standard.
+
+; -------------------------------------------------------------------------
+; Brace-delimited constructs — increase indent inside
+; -------------------------------------------------------------------------
+
+[
+  (statement_block)
+  (switch_body)
+  (object)
+  (array)
+  (arguments)
+  (formal_parameters)
+] @indent.begin
+
+; Closing delimiters — decrease indent
+[
+  "}"
+  "]"
+  ")"
+] @indent.end
+
+; -------------------------------------------------------------------------
+; Alternative (colon/endif) syntax
+; -------------------------------------------------------------------------
+
+; The ':' after the condition opens the body
+(if_alt_statement ":" @indent.begin)
+(elif_clause ":" @indent.begin)
+(for_alt_statement ":" @indent.begin)
+(for_in_alt_statement ":" @indent.begin)
+(while_alt_statement ":" @indent.begin)
+(function_declaration ":" @indent.begin)
+
+; Closing keywords close the body
+[
+  "endif"
+  "endfor"
+  "endwhile"
+  "endfunction"
+] @indent.end
+
+; -------------------------------------------------------------------------
+; Branch keywords (else/elif) — dedent keyword, indent following body
+; -------------------------------------------------------------------------
+
+(else_clause "else") @indent.branch
+(else_alt_clause "else") @indent.branch
+(elif_clause "elif") @indent.branch
+
+; else_alt_clause has no ':' delimiter (grammar: 'else' repeat(statement)),
+; so @indent.branch alone only dedents the 'else' keyword — it does not
+; open an indent scope for the body. Add @indent.begin on the whole node
+; so the body statements are indented one level relative to 'else'.
+(else_alt_clause) @indent.begin
+
+; -------------------------------------------------------------------------
+; Switch
+; -------------------------------------------------------------------------
+
+(switch_case "case") @indent.branch
+(switch_default "default") @indent.branch

package/queries/locals.scm

@@ -6,7 +6,6 @@
 ; -------------------------------------------------------------------------
 
 (function_declaration) @local.scope
-; function_forward_declaration has no body — it is a definition, not a scope
 (function_expression) @local.scope
 (arrow_function) @local.scope
 ; statement_block provides block scoping for let/const
@@ -23,16 +22,12 @@
 ; Definitions — functions
 ; -------------------------------------------------------------------------
 
-; Declaration and forward-declaration names belong to the *enclosing* scope
-; so that callers outside the function body can resolve them.
+; Function declaration names belong to the enclosing scope so that
+; callers outside the function body can resolve them.
 (function_declaration
   name: (identifier) @local.definition.function
   (#set! definition.function.scope parent))
 
-(function_forward_declaration
-  name: (identifier) @local.definition.function
-  (#set! definition.function.scope parent))
-
 ; Named function expressions (let f = function foo() {}) keep their name
 ; *inside* the expression scope — foo is only visible for self-recursion,
 ; not to the surrounding block.  No (#set! parent) here.

package/queries/tags.scm

@@ -14,19 +14,6 @@
   (#select-adjacent! @doc @definition.function)
 )
 
-; -------------------------------------------------------------------------
-; Function definitions — forward declarations: `function name;`
-; -------------------------------------------------------------------------
-
-(
-  (comment)* @doc
-  .
-  (function_forward_declaration
-    name: (identifier) @name) @definition.function
-  (#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
-  (#select-adjacent! @doc @definition.function)
-)
-
 ; -------------------------------------------------------------------------
 ; Function definitions — named function expressions
 ; -------------------------------------------------------------------------

package/queries/textobjects.scm

@@ -0,0 +1,84 @@
+; Text-object queries for nvim-treesitter-textobjects.
+; Provides @function, @parameter, @conditional, @loop, @call, and @block
+; text objects for ucode source files.
+
+; -------------------------------------------------------------------------
+; Functions
+; -------------------------------------------------------------------------
+
+; Brace-body function declaration
+(function_declaration
+  body: (statement_block) @function.inner) @function.outer
+
+; Colon-body function declaration — outer only; no block node to target for inner
+(function_declaration "endfunction") @function.outer
+
+(function_expression
+  body: (statement_block) @function.inner) @function.outer
+
+(arrow_function
+  body: (statement_block) @function.inner) @function.outer
+
+; Arrow function with expression body (no braces).
+; Uses the 'expression' supertype so all expression forms are matched
+; without an explicit enumeration — including update_expression (x => x++),
+; assignment_expression (x => y = 1), and any type added to the grammar later.
+(arrow_function
+  body: (expression) @function.inner) @function.outer
+
+; -------------------------------------------------------------------------
+; Parameters
+;
+; @parameter.inner — the parameter node itself
+; @parameter.outer — same node; nvim-treesitter-textobjects extends the
+;   selection to absorb adjacent ',' delimiters automatically when the
+;   capture name ends in .outer
+; -------------------------------------------------------------------------
+
+(formal_parameters (_) @parameter.inner)
+(formal_parameters (_) @parameter.outer)
+
+; -------------------------------------------------------------------------
+; Conditionals
+; -------------------------------------------------------------------------
+
+(if_statement
+  consequence: (statement_block) @conditional.inner) @conditional.outer
+
+(if_alt_statement) @conditional.outer
+
+; -------------------------------------------------------------------------
+; Loops
+; -------------------------------------------------------------------------
+
+(for_statement
+  body: (statement_block) @loop.inner) @loop.outer
+
+(for_in_statement
+  body: (statement_block) @loop.inner) @loop.outer
+
+(while_statement
+  body: (statement_block) @loop.inner) @loop.outer
+
+(for_alt_statement) @loop.outer
+(for_in_alt_statement) @loop.outer
+(while_alt_statement) @loop.outer
+
+; -------------------------------------------------------------------------
+; Calls
+; -------------------------------------------------------------------------
+
+(call_expression
+  arguments: (arguments) @call.inner) @call.outer
+
+; -------------------------------------------------------------------------
+; Blocks
+;
+; @block.outer — the full statement_block including braces
+; @block.inner — named children only; nvim-treesitter-textobjects selects
+;   from the first to the last captured child, covering the block interior
+;   without needing the deprecated #make-range! predicate
+; -------------------------------------------------------------------------
+
+(statement_block) @block.outer
+(statement_block (_) @block.inner)

package/src/grammar.json

@@ -88,8 +88,8 @@
                           }
                         },
                         {
-                          "type": "SYMBOL",
-                          "name": "_semicolon"
+                          "type": "STRING",
+                          "value": ";"
                         }
                       ]
                     }
@@ -221,23 +221,12 @@
           "type": "SYMBOL",
           "name": "function_declaration"
         },
-        {
-          "type": "SYMBOL",
-          "name": "function_forward_declaration"
-        },
         {
           "type": "SYMBOL",
           "name": "lexical_declaration"
         }
       ]
     },
-    "import": {
-      "type": "TOKEN",
-      "content": {
-        "type": "STRING",
-        "value": "import"
-      }
-    },
     "import_statement": {
       "type": "SEQ",
       "members": [
@@ -1690,10 +1679,6 @@
         {
           "type": "SYMBOL",
           "name": "update_expression"
-        },
-        {
-          "type": "SYMBOL",
-          "name": "import_expression"
         }
       ]
     },
@@ -1779,35 +1764,6 @@
         }
       ]
     },
-    "import_expression": {
-      "type": "PREC",
-      "value": "call",
-      "content": {
-        "type": "SEQ",
-        "members": [
-          {
-            "type": "SYMBOL",
-            "name": "import"
-          },
-          {
-            "type": "STRING",
-            "value": "("
-          },
-          {
-            "type": "FIELD",
-            "name": "source",
-            "content": {
-              "type": "SYMBOL",
-              "name": "expression"
-            }
-          },
-          {
-            "type": "STRING",
-            "value": ")"
-          }
-        ]
-      }
-    },
     "object": {
       "type": "PREC",
       "value": "object",
@@ -3472,27 +3428,6 @@
         ]
       }
     },
-    "function_forward_declaration": {
-      "type": "SEQ",
-      "members": [
-        {
-          "type": "STRING",
-          "value": "function"
-        },
-        {
-          "type": "FIELD",
-          "name": "name",
-          "content": {
-            "type": "SYMBOL",
-            "name": "identifier"
-          }
-        },
-        {
-          "type": "STRING",
-          "value": ";"
-        }
-      ]
-    },
     "arrow_function": {
       "type": "SEQ",
       "members": [
@@ -4666,10 +4601,6 @@
         {
           "type": "STRING",
           "value": "set"
-        },
-        {
-          "type": "STRING",
-          "value": "let"
         }
       ]
     },
@@ -4842,16 +4773,6 @@
         "value": "object"
       }
     ],
-    [
-      {
-        "type": "SYMBOL",
-        "name": "import_statement"
-      },
-      {
-        "type": "SYMBOL",
-        "name": "import"
-      }
-    ],
     [
       {
         "type": "SYMBOL",
@@ -4984,6 +4905,10 @@
         "type": "STRING",
         "value": "in"
       },
+      {
+        "type": "STRING",
+        "value": "let"
+      },
       {
         "type": "STRING",
         "value": "null"