papagaio 0.38.2 → 0.41.1

package/README.md

@@ -36,7 +36,6 @@ Patterns are composed of whitespace-separated tokens. The engine uses a "flex-ma
 - **Literal**: Matches exact text.
 - **Variable**: `$name` (captures a sequence up to the next pattern match).
 - **Optional**: `$name?` or `literal?` (marker is configurable, e.g., `MAYBE`, via `$changesymbols`).
-- **Escaping**: Use `$$` to match a literal `$`.
 
 ### Modifiers
 Modifiers specify the data type or constraints of a match:
@@ -147,6 +146,56 @@ This changes the sigil to `@`, delimiters to `< >`, and the optional marker to `
 
 ---
 
+## Dynamic Built-in Operators
+
+Papagaio provides several built-in operators that give you access to the engine's internal syntax configuration and allow you to precisely inject unrepresentable characters (such as whitespace and binary ASCII) directly into patterns or output text. 
+
+> [!NOTE]
+> All built-in operators support the **trailing sigil** syntax (e.g., `$sigil$`). This allows the operator to immediately consume all following whitespace or safely concatenate with adjacent alphanumeric text, exactly like standard variables.
+
+### Reserved Symbol Emitters
+When generating macros or writing complex rules, you may need to emit a literal sigil without it being evaluated. Instead of complex double-escaping, you can use:
+- **`$sigil`**: Emits the current sigil (e.g., `$`)
+- **`$open`**: Emits the current open delimiter (e.g., `{`)
+- **`$close`**: Emits the current close delimiter (e.g., `}`)
+- **`$marker`**: Emits the current optional marker (e.g., `?`)
+
+**Solving the Infinite Interpretation Problem:**
+```text
+$pattern {hello} {world $sigil$A$sigil$from{X}}
+hello
+```
+*Output: `world $A$from{X}`* — Since `$sigil` is processed during the single initial pass, it emits a literal `$` that safely bypasses any subsequent evaluation.
+
+### Formatting and Binary Control
+For precise layout control, especially inside flex-matched `$pattern` rules that normally skip extra whitespace:
+- **`$space`**: Emits a literal space character (`' '`)
+- **`$newline`**: Emits a literal newline (`\n`)
+- **`$tab`**: Emits a literal tab character (`\t`)
+
+To generate specific binary characters (such as null bytes) or handle complex ASCII injection:
+- **`$ascii$code`** (Inline): E.g., `$ascii$65` outputs `A`
+- **`$ascii{code}`** (Block): E.g., `$ascii{0}` outputs a binary null byte (`\0`)
+
+### Mathematical Evaluation
+The **`$math{...}`** operator allows native numerical processing and comparison directly on the preprocessor pipeline. Powered internally by `tinyexpr`, it supports floating point arithmetic, trigonometric functions, exponents, and logical evaluations. The mathematical string is processed recursively by Papagaio before being evaluated, meaning you can easily inject your workflow variables.
+
+```text
+$X$from{5.5}
+$Y$from{4.5}
+$math{sqrt($X^2 + $Y^2 - 0.5)}
+```
+*Output: `7.07107`*
+
+**Comparisons and Logic Gates**
+Since comparison operators (`<`, `>`, `==`, `!=`) evaluate to `1` (true) or `0` (false), `$math` is perfect for chained conditionals when paired with Papagaio's flow controllers:
+```text
+$math{10 > 5}$compare{1}$then{ Math confirms 10 is greater than 5! }
+```
+*(Note: If the expression contains syntax errors like `$math{5 + *}`, the operator fails silently and emits an empty string to maintain engine stability).*
+
+---
+
 ## Recursive Priority System
 
 Papagaio allows you to control the order of execution and side-effects (such as pattern definitions) using the **`$priority$N`** directive.
@@ -178,6 +227,14 @@ The **`$from`** operator allows you to capture processed content and assign it t
 2. **Immediate Registration**: The variable is registered as an exact-match rule as soon as it is parsed. This allows for **chained assignments**.
 3. **Output Suppression**: The entire `$from` directive is removed from the output text.
 
+### Lexical Scopes and Sandboxing
+
+Every block operator or preprocessing field evaluated dynamically (such as inside `$from`, `$then`, `$else`, `$while`, `$repeat`, and `$until`) is executed in its own **local nested scope**.
+
+- **Sandboxing**: Any variable declared inside a local scope (e.g. `$B$from{local_val}` inside `$A$from{...}`) that does *not* exist in a parent scope is treated as a local variable. It will be completely destroyed and freed when the block finishes evaluating (sandboxed).
+- **Shadowing & Upward Updates**: If a variable updated inside a local scope already exists in a parent/ancestor scope, Papagaio avoids shadowing it. Instead, it propagates the update upwards, modifying the existing variable in the parent scope.
+- **Recursive Nesting**: Local scopes can be nested arbitrarily. Each child scope has full read/write access to variables in parent scopes, but parent or sibling scopes do not have access to variables declared exclusively in child scopes.
+
 ### Examples
 
 #### Chained Assignments

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "papagaio",
-  "version": "0.38.2",
+  "version": "0.41.1",
   "description": "easy yet powerful preprocessor",
   "main": "dist/wasm/papagaio.js",
   "bin": {
@@ -26,7 +26,6 @@
     "eval",
     "parser",
     "codegen",
-    "regex",
     "syntax",
     "easy"
   ],