@fairyhunter13/opentui-core 0.1.88 → 0.1.90

package/docs/tree-sitter.md

@@ -0,0 +1,311 @@
+# Tree-Sitter
+
+## Adding Custom Parsers
+
+There are two ways to add custom parsers to your application:
+
+### 1. Global Default Parsers (Recommended)
+
+Use `addDefaultParsers()` to add parsers globally before initializing any clients. This is useful when you want all Tree-Sitter clients in your application to support the same languages.
+
+```typescript
+import { addDefaultParsers, getTreeSitterClient } from "@opentui/core"
+
+// Add Python parser globally
+addDefaultParsers([
+  {
+    filetype: "python",
+    wasm: "https://github.com/tree-sitter/tree-sitter-python/releases/download/v0.23.6/tree-sitter-python.wasm",
+    queries: {
+      highlights: ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-python/master/queries/highlights.scm"],
+    },
+  },
+])
+
+// Now all clients will have Python support
+const client = getTreeSitterClient()
+await client.initialize()
+
+// Highlight Python code
+const pythonCode = 'def hello():\n    print("world")'
+const result = await client.highlightOnce(pythonCode, "python")
+```
+
+### 2. Per-Client Parsers
+
+Use `client.addFiletypeParser()` to add parsers to a specific client instance. This is useful when different parts of your application need different language support.
+
+```typescript
+import { TreeSitterClient } from "@opentui/core"
+
+const client = new TreeSitterClient({ dataPath: "./cache" })
+await client.initialize()
+
+// Add Rust parser to this specific client
+client.addFiletypeParser({
+  filetype: "rust",
+  wasm: "https://github.com/tree-sitter/tree-sitter-rust/releases/download/v0.23.2/tree-sitter-rust.wasm",
+  queries: {
+    highlights: ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-rust/master/queries/highlights.scm"],
+  },
+})
+
+// Highlight Rust code
+const rustCode = 'fn main() {\n    println!("Hello, world!");\n}'
+const result = await client.highlightOnce(rustCode, "rust")
+```
+
+## Parser Configuration Structure
+
+The `FiletypeParserOptions` interface defines how to configure a parser:
+
+```typescript
+interface FiletypeParserOptions {
+  filetype: string // The filetype identifier (e.g., "python", "rust")
+  wasm: string // URL or local file path to the .wasm parser file
+  queries: {
+    highlights: string[] // Array of URLs or local file paths to .scm query files
+  }
+}
+```
+
+## Finding Parsers and Queries
+
+### Official Tree-Sitter Parsers
+
+Most popular languages have official parsers:
+
+```typescript
+// Official parsers follow this pattern:
+const parserUrl =
+  "https://github.com/tree-sitter/tree-sitter-{language}/releases/download/v{version}/tree-sitter-{language}.wasm"
+
+// Examples:
+// Python: https://github.com/tree-sitter/tree-sitter-python/releases/download/v0.23.6/tree-sitter-python.wasm
+// Rust: https://github.com/tree-sitter/tree-sitter-rust/releases/download/v0.23.2/tree-sitter-rust.wasm
+// Go: https://github.com/tree-sitter/tree-sitter-go/releases/download/v0.23.4/tree-sitter-go.wasm
+```
+
+### Finding Highlight Queries
+
+Highlight queries are usually found in the parser repository's `queries/` directory:
+
+```typescript
+// Official queries:
+const queryUrl = "https://raw.githubusercontent.com/tree-sitter/tree-sitter-{language}/master/queries/highlights.scm"
+
+// Or from nvim-treesitter (often more comprehensive):
+const nvimQueryUrl =
+  "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/master/queries/{language}/highlights.scm"
+```
+
+### Combining Multiple Queries
+
+Some languages require multiple query files. For example, TypeScript uses JavaScript queries plus TypeScript-specific queries:
+
+```typescript
+addDefaultParsers([
+  {
+    filetype: "typescript",
+    wasm: "https://github.com/tree-sitter/tree-sitter-typescript/releases/download/v0.23.2/tree-sitter-typescript.wasm",
+    queries: {
+      highlights: [
+        // Base ECMAScript/JavaScript queries
+        "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/master/queries/ecma/highlights.scm",
+        // TypeScript-specific queries
+        "https://raw.githubusercontent.com/nvim-treesitter/nvim-treesitter/master/queries/typescript/highlights.scm",
+      ],
+    },
+  },
+])
+```
+
+## Using Local Files
+
+For better performance and offline support, you can bundle parsers and queries with your application:
+
+```typescript
+// Using Bun's file import
+import pythonWasm from "./parsers/tree-sitter-python.wasm" with { type: "file" }
+import pythonHighlights from "./queries/python/highlights.scm" with { type: "file" }
+
+addDefaultParsers([
+  {
+    filetype: "python",
+    wasm: pythonWasm,
+    queries: {
+      highlights: [pythonHighlights],
+    },
+  },
+])
+```
+
+## Automated Parser Management
+
+You can automate parser downloads and import generation using the `updateAssets` utility. This is especially useful when supporting multiple languages or integrating parser management into your build pipeline.
+
+### Creating a Parser Configuration
+
+Create a `parsers-config.json` file in your project:
+
+```json
+{
+  "parsers": [
+    {
+      "filetype": "python",
+      "wasm": "https://github.com/tree-sitter/tree-sitter-python/releases/download/v0.23.6/tree-sitter-python.wasm",
+      "queries": {
+        "highlights": ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-python/master/queries/highlights.scm"]
+      }
+    },
+    {
+      "filetype": "rust",
+      "wasm": "https://github.com/tree-sitter/tree-sitter-rust/releases/download/v0.23.2/tree-sitter-rust.wasm",
+      "queries": {
+        "highlights": ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-rust/master/queries/highlights.scm"]
+      }
+    }
+  ]
+}
+```
+
+### Integrating into Build Pipeline
+
+#### CLI Usage
+
+Add the update script to your `package.json`:
+
+```json
+{
+  "scripts": {
+    "prebuild": "bun node_modules/@opentui/core/lib/tree-sitter/assets/update.ts --config ./parsers-config.json --assets ./src/parsers --output ./src/parsers.ts",
+    "build": "bun build ./src/index.ts"
+  }
+}
+```
+
+#### Programmatic Usage
+
+Or call it programmatically in your build script:
+
+```typescript
+import { updateAssets } from "@opentui/core"
+
+await updateAssets({
+  configPath: "./parsers-config.json",
+  assetsDir: "./src/parsers",
+  outputPath: "./src/parsers.ts",
+})
+```
+
+### Using Generated Parsers
+
+The script generates a TypeScript file with all parsers pre-configured:
+
+```typescript
+import { addDefaultParsers, getTreeSitterClient } from "@opentui/core"
+import { getParsers } from "./parsers" // Generated file
+
+addDefaultParsers(getParsers())
+
+const client = getTreeSitterClient()
+await client.initialize()
+
+const result = await client.highlightOnce('def hello():\n    print("world")', "python")
+```
+
+## Complete Example: Adding Multiple Languages
+
+```typescript
+import { addDefaultParsers, getTreeSitterClient, SyntaxStyle } from "@opentui/core"
+
+// Add support for multiple languages before initializing
+addDefaultParsers([
+  {
+    filetype: "python",
+    wasm: "https://github.com/tree-sitter/tree-sitter-python/releases/download/v0.23.6/tree-sitter-python.wasm",
+    queries: {
+      highlights: ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-python/master/queries/highlights.scm"],
+    },
+  },
+  {
+    filetype: "rust",
+    wasm: "https://github.com/tree-sitter/tree-sitter-rust/releases/download/v0.23.2/tree-sitter-rust.wasm",
+    queries: {
+      highlights: ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-rust/master/queries/highlights.scm"],
+    },
+  },
+  {
+    filetype: "go",
+    wasm: "https://github.com/tree-sitter/tree-sitter-go/releases/download/v0.23.4/tree-sitter-go.wasm",
+    queries: {
+      highlights: ["https://raw.githubusercontent.com/tree-sitter/tree-sitter-go/master/queries/highlights.scm"],
+    },
+  },
+])
+
+// Initialize the client
+const client = getTreeSitterClient()
+await client.initialize()
+
+// Use with different languages
+const syntaxStyle = new SyntaxStyle()
+
+const pythonResult = await client.highlightOnce('def hello():\n    print("world")', "python")
+
+const rustResult = await client.highlightOnce('fn main() {\n    println!("Hello");\n}', "rust")
+
+const goResult = await client.highlightOnce('func main() {\n    fmt.Println("Hello")\n}', "go")
+```
+
+## Using with CodeRenderable
+
+The `CodeRenderable` component automatically uses the Tree-Sitter client for syntax highlighting:
+
+```typescript
+import { CodeRenderable, getTreeSitterClient } from "@opentui/core"
+
+// Initialize the client with custom parsers
+const client = getTreeSitterClient()
+await client.initialize()
+
+// Create a code renderable
+const codeBlock = new CodeRenderable("code-1", {
+  content: 'def hello():\n    print("world")',
+  filetype: "python",
+  width: 40,
+  height: 10,
+})
+
+// The CodeRenderable will automatically use the Tree-Sitter client
+// to highlight the code
+```
+
+## Caching
+
+Parser and query files are automatically cached in the `dataPath` directory to avoid re-downloading them.
+You can customize the cache location when creating a client:
+
+```typescript
+const client = new TreeSitterClient({
+  dataPath: "./my-custom-cache",
+})
+```
+
+## File Type Resolution
+
+OpenTUI provides utilities to automatically determine filetypes from file paths:
+
+```typescript
+import { pathToFiletype, extToFiletype } from "@opentui/core"
+
+// Get filetype from file path
+const ft1 = pathToFiletype("src/main.rs") // "rust"
+const ft2 = pathToFiletype("app.py") // "python"
+
+// Get filetype from extension
+const ft3 = extToFiletype("ts") // "typescript"
+const ft4 = extToFiletype("js") // "javascript"
+```
+
+Built-in mappings include: `js`, `jsx`, `ts`, `tsx`, `py`, `rb`, `go`, `rs`, `c`, `cpp`, `html`, `css`, and more.

package/package.json

@@ -1,56 +1,32 @@
 {
   "name": "@fairyhunter13/opentui-core",
-  "module": "index.js",
-  "main": "index.js",
-  "types": "index.d.ts",
-  "type": "module",
-  "version": "0.1.88",
   "description": "OpenTUI is a TypeScript library on a native Zig core for building terminal user interfaces (TUIs)",
-  "license": "MIT",
   "repository": {
     "type": "git",
     "url": "https://github.com/fairyhunter13/opentui",
     "directory": "packages/core"
   },
-  "exports": {
-    ".": {
-      "import": "./index.js",
-      "require": "./index.js",
-      "types": "./index.d.ts"
-    },
-    "./3d": {
-      "import": "./3d.js",
-      "require": "./3d.js",
-      "types": "./3d.d.ts"
-    },
-    "./testing": {
-      "import": "./testing.js",
-      "require": "./testing.js",
-      "types": "./testing.d.ts"
-    },
-    "./runtime-plugin": {
-      "import": "./runtime-plugin.js",
-      "require": "./runtime-plugin.js",
-      "types": "./runtime-plugin.d.ts"
-    },
-    "./runtime-plugin-support": {
-      "import": "./runtime-plugin-support.js",
-      "require": "./runtime-plugin-support.js",
-      "types": "./runtime-plugin-support.d.ts"
-    },
-    "./parser.worker": {
-      "import": "./lib/tree-sitter/parser.worker.js",
-      "require": "./lib/tree-sitter/parser.worker.js",
-      "types": "./lib/tree-sitter/parser.worker.d.ts"
-    }
-  },
-  "dependencies": {
-    "bun-ffi-structs": "0.1.2",
-    "diff": "8.0.2",
-    "jimp": "1.6.0",
-    "marked": "17.0.1",
-    "yoga-layout": "3.2.1"
+  "types": "src/index.ts",
+  "module": "src/index.ts",
+  "main": "src/index.ts",
+  "version": "0.1.90",
+  "type": "module",
+  "scripts": {
+    "build": "bun run build:native && bun run build:lib",
+    "build:dev": "bun run build:native:dev && bun run build:lib",
+    "build:lib": "bun scripts/build.ts --lib",
+    "build:native": "bun scripts/build.ts --native",
+    "build:native:dev": "bun scripts/build.ts --native --dev",
+    "build:examples": "bun src/examples/build.ts",
+    "test:native": "cd src/zig && zig build test --summary all",
+    "bench:native": "cd src/zig && zig build bench -Dbench-optimize=ReleaseFast --",
+    "bench:text-table": "bun src/benchmark/text-table-benchmark.ts",
+    "bench:ts": "bun src/benchmark/native-span-feed-benchmark.ts --suite=quick --json=src/benchmark/latest-quick-bench-run.json && bun src/benchmark/native-span-feed-benchmark.ts --suite=default --json=src/benchmark/latest-default-bench-run.json && bun src/benchmark/native-span-feed-benchmark.ts --suite=large --json=src/benchmark/latest-large-bench-run.json && bun src/benchmark/native-span-feed-benchmark.ts --suite=all --json=src/benchmark/latest-all-bench-run.json && bun src/benchmark/native-span-feed-async-benchmark.ts --json=src/benchmark/latest-async-bench-run.json",
+    "publish": "bun scripts/publish.ts",
+    "test:js": "bun test",
+    "test": "bun run test:native && bun run test:js"
   },
+  "license": "MIT",
   "devDependencies": {
     "@types/bun": "latest",
     "@types/node": "^24.0.0",
@@ -59,6 +35,13 @@
     "typescript": "^5",
     "web-tree-sitter": "0.25.10"
   },
+  "dependencies": {
+    "bun-ffi-structs": "0.1.2",
+    "diff": "8.0.2",
+    "jimp": "1.6.0",
+    "marked": "17.0.1",
+    "yoga-layout": "3.2.1"
+  },
   "peerDependencies": {
     "web-tree-sitter": "0.25.10"
   },
@@ -67,14 +50,40 @@
     "bun-webgpu": "0.1.5",
     "planck": "^1.4.2",
     "three": "0.177.0",
-    "@fairyhunter13/opentui-core-darwin-x64": "0.1.88",
-    "@fairyhunter13/opentui-core-darwin-arm64": "0.1.88",
-    "@fairyhunter13/opentui-core-linux-x64": "0.1.88",
-    "@fairyhunter13/opentui-core-linux-arm64": "0.1.88",
-    "@fairyhunter13/opentui-core-win32-x64": "0.1.88",
-    "@fairyhunter13/opentui-core-win32-arm64": "0.1.88"
+    "@opentui/core-darwin-x64": "0.1.87",
+    "@opentui/core-darwin-arm64": "0.1.87",
+    "@opentui/core-linux-x64": "0.1.87",
+    "@opentui/core-linux-arm64": "0.1.87",
+    "@opentui/core-win32-x64": "0.1.87",
+    "@opentui/core-win32-arm64": "0.1.87"
+  },
+  "exports": {
+    ".": {
+      "types": "./src/index.ts",
+      "import": "./src/index.ts"
+    },
+    "./3d": {
+      "types": "./src/3d.ts",
+      "import": "./src/3d.ts"
+    },
+    "./testing": {
+      "types": "./src/testing.ts",
+      "import": "./src/testing.ts"
+    },
+    "./runtime-plugin": {
+      "types": "./src/runtime-plugin.ts",
+      "import": "./src/runtime-plugin.ts"
+    },
+    "./runtime-plugin-support": {
+      "types": "./src/runtime-plugin-support.ts",
+      "import": "./src/runtime-plugin-support.ts"
+    },
+    "./parser.worker": {
+      "types": "./src/lib/tree-sitter/parser.worker.ts",
+      "import": "./src/lib/tree-sitter/parser.worker.ts"
+    }
   },
-  "publishConfig": {
-    "access": "public"
+  "engines": {
+    "bun": ">=1.3.0"
   }
 }