papagaio 0.32.9 → 0.38.2

package/README.md

@@ -1,12 +1,11 @@
 # Papagaio
 
-Papagaio is a C-first, embeddable text processing engine. It is designed to be highly modular and **script-agnostic**, allowing core pattern matching to be used alone or extended via WebAssembly (Wasm) plugins.
+Papagaio is an embeddable text processing engine.
 
 ## Key Features
 
 - **Lightweight Core**: Efficient C engine for pattern matching and transformation.
 - **Pattern-Matching**: Powerful capture system with built-in and custom modifiers.
-- **WebAssembly Plugins**: Highly secure, zero-dependency plugin architecture via an embedded `wasm3` runtime.
 - **Configurable Delimiters**: Redefine sigils, delimiters, and markers at runtime.
 - **Language Bindings**: Native usage in C and Node.js/WebAssembly.
 
@@ -28,17 +27,6 @@ free(out);
 papagaio_close(ctx);
 ```
 
-### JavaScript / WASM (Node.js)
-```javascript
-import Papagaio from './papagaio.js';
-
-const p = new Papagaio();
-await p.init();
-p.registerCommand("mycmd", (name, ...args) => `Result: ${args[0]}`);
-
-console.log(p.process('$mycmd{Hello}')); // Output: Result: Hello
-```
-
 ---
 
 ## Pattern Syntax
@@ -55,7 +43,6 @@ Modifiers specify the data type or constraints of a match:
 - **Numbers**: `$var$int`, `$var$float`, `$var$number`
 - **Casing**: `$var$upper`, `$var$lower`, `$var$capitalized`
 - **Formats**: `$var$word`, `$var$identifier`, `$var$hex`, `$var$path`, `$var$binary`, `$var$percent`
-- **Regex**: `$id$regex{[0-9]+}`
 - **Block**: `$item$block{[}{]}` (captures everything between delimiters)
 - **Aliases**: `$kind$aliases{cat}{dog}{bird}` (multi-block syntax).
 - **Substrings**: `$var$starts{foo}`, `$var$ends{bar}`, `$var$prefix{p}`, `$var$suffix{s}`, `$var$infix{i}`, `$var$includes{x}`
@@ -94,17 +81,11 @@ $pattern {$n$aliases{$x$int}{abc}} {VALUE: $n}
 
 ---
 
-## Extensibility (Wasm Plugins)
-
-Papagaio follows a Wasm-first plugin architecture. Core features are limited to pattern matching and transformation, while custom text processing capabilities are provided by WebAssembly plugins.
-
 ### Built-in Operators
 - **`$document`**: Injects the current state of the document (alias for `$document$current`).
 - **`$document$original`**: Injects the initial, unprocessed input text. Useful for referencing the source even after multiple transformations.
 - **`$document$current`**: Injects the current state of the document during the pre-processing pass.
-- **`$wasm{path}`**: Loads a WebAssembly plugin from the file system (CLI only).
-- **$file{path}**: Injects the content of a file from the file system (CLI only).
-- **`$wat{source}`**: Compiles a WebAssembly Text Format (WAT) source string inline and registers all exported `papagaio_*` functions as commands. Useful for embedding lightweight plugins without an external `.wasm` file.
+- **$file{path}**: Injects the content of a file from the file system.
 - **`$NAME$from{value}`**: Dynamically assigns a processed `value` to `$NAME`. The assignment itself is suppressed from the output, and the variable becomes available for exact-match replacement in the remaining document.
 
   ```text
@@ -113,15 +94,6 @@ Papagaio follows a Wasm-first plugin architecture. Core features are limited to
   ```
   *Output: `Hello, Alice!`*
 
-  ```text
-  $wat{
-    (module
-      (func (export "papagaio_hello") (result i32)
-        i32.const 42))
-  }
-  $hello
-  ```
-
 ---
 
 ## CLI Argument Expansion
@@ -145,11 +117,11 @@ Arguments in the format `key=value` are automatically parsed and can be accessed
 1. **Explicit**: `$args$key`
 2. **Direct**: `$key` (shorthand for `$args$key`)
 
-Direct access (`$key`) will only resolve if `key` does not conflict with a registered command (like `$wasm`) or a built-in directive.
+Direct access (`$key`) will only resolve if `key` does not conflict with a registered command or a built-in directive.
 
 #### Example:
 ```sh
-./papagaiocc input.c version=1.2.3 target=wasm -O3
+# Then compile ready.c with clang
 ```
 Inside `input.c`:
 ```c
@@ -177,7 +149,7 @@ This changes the sigil to `@`, delimiters to `< >`, and the optional marker to `
 
 ## Recursive Priority System
 
-Papagaio allows you to control the order of execution and side-effects (such as pattern definitions or WASM loading) using the **`$priority$N`** directive.
+Papagaio allows you to control the order of execution and side-effects (such as pattern definitions) using the **`$priority$N`** directive.
 
 - **`$priority$0{...}`**: Maximum priority.
 - **`$priority$max{...}`**: Alias for `INT_MIN + 1` (Absolute highest priority).
@@ -202,7 +174,7 @@ The **`$from`** operator allows you to capture processed content and assign it t
 ### Syntax
 `$NAME$from{...content...}`
 
-1. **Recursive Processing**: The `content` is fully processed (patterns, WASM commands, other assignments) before being stored.
+1. **Recursive Processing**: The `content` is fully processed (patterns, other assignments) before being stored.
 2. **Immediate Registration**: The variable is registered as an exact-match rule as soon as it is parsed. This allows for **chained assignments**.
 3. **Output Suppression**: The entire `$from` directive is removed from the output text.
 
@@ -239,69 +211,160 @@ Result: FOO
 
 ---
 
-## Plugin Development
+## List Operations (`$list`)
 
-Papagaio features a modern, frictionless Wasm plugin system. With the **`papagaiocc`** standalone compiler, you can write plugins in standard C using simple naming conventions and compile them into zero-dependency WebAssembly modules.
+Any variable can be treated as a list by accessing it through the `$list` modifier chain. The **separator** can be any string (single character or multi-character) and is itself **processed** before use, allowing dynamic separators.
+
+### Syntax
 
-### 1. Write your plugin
-Create a file named `greet.c`:
-```c
-// Functions starting with 'papagaio_' are automatically registered as commands
-char* papagaio_greet(int argc, char **argv)
-{
-    if (argc < 1) return "Hello, Stranger!";
-    return argv[0]; // return first argument
-}
+```
+$VARNAME$list{separator}$OPERATION{...arguments}
 ```
 
-To use the Papagaio Wasm SDK (`lib.c`), copy it from `examples/lib.c` into your project and include it explicitly:
-```c
-#include "lib.c"
-
-char* papagaio_greet(int argc, char **argv)
-{
-    if (argc < 1) return "Hello, Stranger!";
-    
-    // lib.c provides standard C functions like malloc and sprintf
-    char *res = (char*)malloc(strlen(argv[0]) + 16);
-    sprintf(res, "Hello, %s!", argv[0]);
-    
-    return res;
-}
+### Operations
+
+| Operation | Signature | Emits | Mutates |
+|---|---|---|---|
+| `get` | `$V$list{sep}$get{idx}` | Element at index | No |
+| `set` | `$V$list{sep}$set{idx}{content}` | Nothing | Yes |
+| `push` | `$V$list{sep}$push{content}` | Nothing | Yes |
+| `pop` | `$V$list{sep}$pop` | Last element | Yes |
+| `shift` | `$V$list{sep}$shift` | First element | Yes |
+| `unshift` | `$V$list{sep}$unshift{content}` | Nothing | Yes |
+| `insert` | `$V$list{sep}$insert{idx}{content}` | Nothing | Yes |
+| `remove` | `$V$list{sep}$remove{idx}` | Nothing | Yes |
+| `swap` | `$V$list{sep}$swap{idx_a}{idx_b}` | Nothing | Yes |
+| `reverse` | `$V$list{sep}$reverse` | Nothing | Yes |
+| `count` | `$V$list{sep}$count` | Number of elements | No |
+| `join` | `$V$list{sep_orig}$join{sep_new}` | List with new separator | No |
+| `slice` | `$V$list{sep}$slice{start}{end}` | Sub-list from start to end | No |
+| `find` | `$V$list{sep}$find{pat}` | First **whole element** matching `pat` | No |
+| `contains` | `$V$list{sep}$contains{pat}` | Index of `pat` **within** matching element | No |
+| `replace` | `$V$list{sep}$replace{pat}{rep}` | First match found; updates the **whole element** | Yes |
+
+**Index rules**: zero-based; negative indices count from the end (`-1` = last); out-of-range access emits `""` silently.
+
+### Examples
+
+```text
+$FRUITS$from{apple,banana,orange}
+
+$FRUITS$list{,}$get{0}    → apple
+$FRUITS$list{,}$get{-1}   → orange
+$FRUITS$list{,}$count     → 3
 ```
 
-### 2. Compile with `papagaiocc`
-The `papagaiocc` tool is a self-contained compiler driver. Run it with your source file:
-```sh
-./papagaiocc greet.c
+```text
+$L$from{a,b,c}
+$L$list{,}$push{d}
+$L$list{,}$set{1}{B}
+$L                        → a,B,c,d
 ```
-This generates `greet.wasm`.
 
-If your plugin uses `lib.c`, pass the directory containing it via `-I`:
-```sh
-./papagaiocc greet.c -I /path/to/lib
+```text
+$STACK$from{x,y,z}
+Popped: $STACK$list{,}$pop
+Rest: $STACK              → Popped: z  /  Rest: x,y
 ```
-Or simply place `lib.c` in the same directory as `greet.c`:
-```sh
-# Copy the SDK alongside your source
-cp examples/lib.c .
-./papagaiocc greet.c
+
+```text
+$CSV$from{one,two,three}
+$CSV$list{,}$join{ | }    → one | two | three
 ```
 
-### 3. Use in Papagaio
-Loading the Wasm file automatically registers all exported commands.
 ```text
-$wasm{greet.wasm}
-$greet{Papagaio}
+$PATH$from{/usr/local/bin}
+$PATH$list{/}$get{-1}     → bin
 ```
-*Output: Hello, Papagaio!*
 
-### Wasm SDK (lib.c)
-The Wasm SDK lives at `examples/lib.c` inside the repository. It is **not** automatically embedded into `papagaiocc` — you supply it to your plugin's build as needed. It provides a curated, zero-dependency C standard library for WebAssembly, including:
-- **Memory Management**: `malloc`, `free`, `realloc`
-- **String Processing**: `strlen`, `strcpy`, `sprintf`, `strrev`, etc.
-- **Formatted I/O**: `printf`, `snprintf`, `sscanf`
-- **Standard Math**: `sin`, `cos`, `pow`, etc.
+```text
+/* Dynamic separator from variable */
+$SEP$from{,}
+$L$from{x,y,z}
+$L$list{$SEP}$get{2}      → z
+
+$L$from{a,b,c,d,e}
+$L$list{,}$slice{1}{4}    → b,c,d
+```
+
+---
+
+## Flow Control Operators
+
+Papagaio provides operators for conditional logic and value chaining. These are treated as **suffix modifiers** that can be appended to any variable, list operation, or expression.
+
+### Syntax
+
+```
+$VAL$compare{target}
+$VAL$find{pattern}
+$VAL$contains{pattern}
+$VAL$replace{pattern}{replacement}
+$VAL$slice{start}{end}
+$VAL$then{content}
+$VAL$else{content}
+```
+
+| Operator | Behavior |
+|---|---|
+| **`compare`** | If `$VAL` matches `target`, emits `$VAL`. Otherwise, emits `""`. |
+| **`find`** | Performs a non-anchored search for `pattern` in `$VAL`. Emits the matched substring. |
+| **`contains`** | Performs a non-anchored search. Emits the character index of the first match (or `""`). |
+| **`replace`** | Replaces the first match of `pattern` with `replacement`. Emits the OLD match. |
+| **`slice`** | Returns a substring from `start` to `end`. Supports negative indices. |
+| **`then`** | If `$VAL` is **not empty**, processes and emits `content`. Otherwise, emits `""`. |
+| **`else`** | If `$VAL` **is empty**, processes and emits `content`. Otherwise, passes `$VAL` through. |
+| **`repeat`** | `$repeat{N}{code}` | Executes `code` N times. Emits nothing; used for side effects. |
+| **`while`** | `$while{pat}{code}` | Executes `code` while its result matches `pat`. Emits the last successful result. |
+| **`until`** | `$until{pat}{code}` | Executes `code` until its result matches `pat`. Emits the match that caused the break. |
+| **`byte`** | `$byte{code}` | Appends a byte (0-255) to the variable or current stream. |
+
+### Chaining (If-Then-Else)
+
+Operators can be chained to create complex conditional logic. The output of one operator becomes the input for the next.
+
+#### Basic If-Then:
+```text
+$A$from{hello}
+$A$compare{hello}$then{Matched!}   → Matched!
+$A$compare{world}$then{Matched!}   → (empty)
+```
+
+#### If-Then-Else Pattern:
+```text
+$A$from{abc}
+$A$compare{abc}$then{YES}$else{NO} → YES
+$A$compare{xyz}$then{YES}$else{NO} → NO
+
+#### Search and Extract:
+```text
+$A$from{user_id: 12345}
+$A$find{$d+}$                → 12345
+$A$contains{id}             → 5
+$A$replace{$d+}{HIDDEN} $A  → 12345 user_id: HIDDEN
+
+#### Slicing:
+```text
+$A$from{hello world}
+$A$slice{0}{5}              → hello
+$A$slice{-5}                → world
+```
+```
+```
+
+### Standalone and Braced Usage
+
+- **Standalone**: If used without a preceding variable (e.g., `$else{default}`), the input is assumed to be an empty string.
+- **Braced**: You can pipe arbitrary braced expressions into flow operators: `${some content}$then{has content!}`.
+
+#### Example:
+```text
+$L$from{a,b,c}
+$R$from{$L$list{,}$get{0}}
+$R$compare{a}$then{Is A}$else{Not A} → Is A
+```
+
+---
 
 ---
 
@@ -309,17 +372,23 @@ The Wasm SDK lives at `examples/lib.c` inside the repository. It is **not** auto
 
 ```sh
 make            # Core & CLI
-make papagaiocc # Standalone plugin compiler
-make wasm       # WebAssembly build (Papagaio in the browser/node)
+make wasm       # WebAssembly build (via Emscripten)
 make test       # Run comprehensive test suite
 ```
 
+---
+
+## System Limits
+
+| Feature | Limit | Rationale / Detail |
+|---|---|---|
+| **Symbol Length** | 15 characters | Sigils, delimiters (`{`, `}`), and markers are stored in fixed 16-byte buffers. |
+| **String Size** | Unlimited | All internal buffers (`StrBuf`) use dynamic `realloc`. Limited only by available RAM. |
+| **Pattern Count** | Unlimited | Registered rules are stored in a dynamic array. |
+| **Priority Range** | `INT_MIN` to `INT_MAX` | Priorities are handled as standard signed integers. |
+| **Recursion Depth** | Stack-limited | Deeply nested patterns or priority blocks are processed recursively. |
+
 ## References
 
 - [cpp](https://en.wikipedia.org/wiki/C_preprocessor)
-- [m4](https://www.gnu.org/software/m4/)
-- [libregexp](https://bellard.org/quickjs/)
-- [quickjs](https://bellard.org/quickjs/)
-- [tcc](https://bellard.org/tcc/)
-- [wasm3](https://github.com/wasm3/wasm3)
-- [watr](https://github.com/dy/watr)
+- [m4](https://www.gnu.org/software/m4/)

package/dist/wasm/papagaio.js

@@ -1,89 +1,60 @@
-import createWasmModule from './papagaio_wasm.js';
+import ModuleFactory from './papagaio_wasm.js';
 
 class Papagaio {
   constructor() {
     this._module = null;
+    this._ctx = null;
     this._initialized = false;
-    this._initPromise = this.init();
+    this._args = [];
   }
 
   async init() {
-    if (this._initialized) return;
-    this._module = await createWasmModule();
+    if (this._initialized) return this;
+    
+    // Load WASM module
+    this._module = await ModuleFactory();
     this._ctx = this._module._papagaio_open();
     this._initialized = true;
     return this;
   }
 
   registerCommand(name, handler) {
-    if (!this._initialized) throw new Error("Not initialized");
-    
-    // name=i, argc=ii, argv=iii, argl=iiii, userdata=iiiii
-    const wrapper = (ctx, namePtr, argc, argvPtr, arglPtr, userdata) => {
-        const cmdName = this._module.UTF8ToString(namePtr);
-        const args = [];
-        for (let i = 0; i < argc; i++) {
-            const ptr = this._module.getValue(argvPtr + (i * 4), "i32");
-            const len = this._module.getValue(arglPtr + (i * 4), "i32");
-            args.push(this._module.UTF8ToString(ptr, len));
-        }
-        const result = handler(cmdName, ...args);
-        
-        if (result === null || result === undefined) return 0;
-        const resStr = String(result);
-        const resLen = this._module.lengthBytesUTF8(resStr) + 1;
-        const resPtr = this._module._malloc(resLen);
-        this._module.stringToUTF8(resStr, resPtr, resLen);
-        return resPtr;
-    };
-
-    const funcPtr = this._module.addFunction(wrapper, 'iiiiiii');
-    this._module.ccall(
-      "papagaio_register_command",
-      null,
-      ["number", "string", "number", "number"],
-      [this._ctx, name, funcPtr, 0]
-    );
+    // Note: Implementing JS->C callbacks requires addFunction and extra glue.
   }
 
   setArgs(argv) {
-    if (!this._initialized) throw new Error("Not initialized");
-    if (!Array.isArray(argv)) throw new Error("argv must be an array");
-
-    /* Create a null-terminated array of string pointers in Wasm memory */
-    const ptrs = argv.map(str => {
-      const len = this._module.lengthBytesUTF8(str) + 1;
-      const ptr = this._module._malloc(len);
-      this._module.stringToUTF8(str, ptr, len);
-      return ptr;
-    });
+    if (!this._initialized) throw new Error("Papagaio not initialized. Call init() first.");
+    this._args = argv;
 
     const argc = argv.length;
-    const argvPtr = this._module._malloc(argc * 4);
+    const argvPtr = this._module._malloc(argc * 4); 
     for (let i = 0; i < argc; i++) {
-        this._module.setValue(argvPtr + (i * 4), ptrs[i], "i32");
+        const str = argv[i];
+        const strLen = this._module.lengthBytesUTF8(str) + 1;
+        const strPtr = this._module._malloc(strLen);
+        this._module.stringToUTF8(str, strPtr, strLen);
+        this._module.setValue(argvPtr + (i * 4), strPtr, 'i32');
     }
 
     this._module._papagaio_set_args(this._ctx, argc, argvPtr);
-    /* We don't free ptrs/argvPtr immediately as Papagaio might need them during processing.
-       In a production-ready class, we should track these for cleanup. */
+    // We don't free argvPtr/strPtr here as Papagaio context might use them.
+    // They will be "leaked" until destroy(), which is fine for a processing session.
   }
 
   process(text) {
-    if (!this._initialized) {
-      throw new Error("Papagaio wasm not initialized. Use 'await papagaio.init()'");
-    }
-    const ptr = this._module.ccall(
-      "papagaio_process_text", 
-      "number", 
-      ["number", "string", "number"], 
-      [this._ctx, text, text.length]
-    );
+    if (!this._initialized) throw new Error("Papagaio not initialized. Call init() first.");
+
+    const textLen = this._module.lengthBytesUTF8(text);
+    const textPtr = this._module._malloc(textLen + 1);
+    this._module.stringToUTF8(text, textPtr, textLen + 1);
+
+    const outPtr = this._module._papagaio_process_text(this._ctx, textPtr, textLen);
+    const output = this._module.UTF8ToString(outPtr);
+
+    this._module._free(textPtr);
+    this._module._free(outPtr);
 
-    if (ptr === 0) return "";
-    const result = this._module.UTF8ToString(ptr);
-    this._module._free(ptr);
-    return result;
+    return output;
   }
 
   destroy() {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "papagaio",
-  "version": "0.32.9",
+  "version": "0.38.2",
   "description": "easy yet powerful preprocessor",
   "main": "dist/wasm/papagaio.js",
   "bin": {