textbringer-tree-sitter 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b7273acdfaffa92b93724cc56ebf35c6e0d645756203b7b32a1dedea38f615fb
-  data.tar.gz: a7b77614d915a8aaa64810e7332cae96342e108fda2cf0e7fc23d02e5c82e6bd
+  metadata.gz: e49bbd134f0163af928c442de2eae8804655ca7acf9a86d13b29e3b4335857bf
+  data.tar.gz: fcb3bdd6fd34d0b199e27e647174c1b432aa5fdd7cbde79319d747e33ef13be4
 SHA512:
-  metadata.gz: b7904ef48cc189a0130d440bd32a2f5f5ee4381300b93ed93127297f8258a3600d4db1e885c204e4e49d415f0535847e3348016795915bdee13fa7b2a118371d
-  data.tar.gz: 0aee8946b437bafc0b5051c6d2000b37a3dd9da5f9c992cf7e40baa0ac91c322f3af725470fea44697f234f36084ac733280f6103e960d1fb3bd211bca0f8607
+  metadata.gz: a4ff22e0d8c167cd96b019815411ac1f91b0f8687d8de5559d652269531316f3bc465a7851c964fc252ec2895aabe45af0e163b732976f767ec3f5618240748d
+  data.tar.gz: 9c29a6c31361c69b29212dfc52731c51889eebea551c6e14a8a30a48a8ff98e6edeca29a68a4b2588879506c8cf7dcd7d9176b90b69aba7a48fca204fbea71a9

data/CI_PARSER_SETUP.md

@@ -0,0 +1,92 @@
+# CI Parser Setup Instructions
+
+## Problem
+
+Tests skip when tree-sitter parsers are not installed, leading to false-green builds in CI.
+
+## Solution
+
+Add a step to download parsers before running tests in your CI workflow.
+
+## Required Changes to `.github/workflows/ci.yml`
+
+Add the following step **before** the "Run tests" step:
+
+```yaml
+- name: Setup parsers for tests
+  run: bundle exec rake parsers:download
+```
+
+### Complete Example
+
+```yaml
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        ruby-version: ['3.3', '3.4', '4.0']
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Ruby ${{ matrix.ruby-version }}
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: ${{ matrix.ruby-version }}
+          bundler-cache: true
+
+      - name: Setup parsers for tests
+        run: bundle exec rake parsers:download
+
+      - name: Run tests
+        run: bundle exec rake test
+```
+
+## Available Rake Tasks
+
+- `rake parsers:download` - Download prebuilt parsers (fastest, recommended for CI)
+- `rake parsers:build` - Build parsers from source (requires compiler)
+- `rake parsers:setup` - Download if possible, fall back to build
+
+## Parsers Required for Tests
+
+- `ruby` - For Ruby syntax highlighting tests
+- `hcl` - For HCL/Terraform tests
+- `markdown` - For Markdown integration tests
+
+## Manual Setup (Development)
+
+To run tests locally with parsers:
+
+```bash
+bundle exec rake parsers:download
+bundle exec rake test
+```
+
+Or build from source:
+
+```bash
+./scripts/build_parsers.sh
+bundle exec rake test
+```
+
+## Troubleshooting
+
+If parser download fails in CI:
+1. Check network connectivity to GitHub
+2. Verify the Faveod/tree-sitter-parsers release version in `scripts/download_parsers.sh`
+3. Fall back to building from source: `bundle exec rake parsers:build`
+
+## References
+
+- Issue: #20
+- Parser source: https://github.com/Faveod/tree-sitter-parsers

data/CLAUDE.md

@@ -6,6 +6,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Textbringer（Ruby製ターミナルエディタ）に Tree-sitter ベースのシンタックスハイライトを提供するプラグイン gem。textbringer-rouge の代替として、より正確な構文解析を実現する（特に Terraform/HCL で Rouge の lexer に問題があるため）。
 
+## Textbringer とは
+
+* Ruby 製の Emacs 風テキストエディタ
+* コマンド名は `txtb`
+
 ## 開発コマンド
 
 ```bash
@@ -14,8 +19,35 @@ bundle exec rake test
 bundle exec ruby -Ilib:test test/test_*.rb  # 単一テスト
 bundle exec rake build
 bundle exec rubocop
+
+# デバッグモード（/tmp/tree_sitter_debug.log に出力）
+TEXTBRINGER_TREE_SITTER_DEBUG=1 textbringer
+```
+
+## CLI ツール
+
+gem に同梱の `textbringer-tree-sitter` コマンドで parser 管理:
+
+```bash
+textbringer-tree-sitter list                # 利用可能な parser 一覧
+textbringer-tree-sitter get <lang>          # parser ダウンロード/ビルド + node_map 自動生成
+textbringer-tree-sitter get <lang> --no-map # node_map 生成をスキップ
+textbringer-tree-sitter get-all             # Faveod prebuilt を一括インストール
+textbringer-tree-sitter generate-map <lang> # 既存 parser から node_map 再生成
+textbringer-tree-sitter path                # parser ディレクトリ表示
 ```
 
+### Parser の配置場所
+
+- `~/.textbringer/parsers/{platform}/` にインストールされる
+- プラットフォーム例: `darwin-arm64`, `linux-x64`
+- カスタムパスは `CONFIG[:tree_sitter_parser_dir]` で指定可能
+
+### Parser の種類
+
+- **Faveod prebuilt**: bash, c, c-sharp, cobol, embedded-template, groovy, haml, html, java, javascript, json, pascal, php, python, ruby, rust
+- **ビルド必須**: hcl, yaml, go, typescript, tsx, sql, markdown (各リポジトリから clone & build)
+
 ## アーキテクチャ
 
 ### 構造
@@ -23,54 +55,102 @@ bundle exec rubocop
 ```
 lib/
 ├── textbringer/
-│   ├── tree_sitter_adapter.rb    # Window モンキーパッチ + アダプター
-│   ├── tree_sitter_config.rb     # Parser ロード、フェイス定義
+│   ├── tree_sitter_adapter.rb    # Window モンキーパッチ + ハイライト実装
+│   ├── tree_sitter_config.rb     # Parser パス解決、Face 定義
 │   └── tree_sitter/
 │       ├── version.rb
-│       └── node_maps/            # 言語ごとのノードマッピング
-│           ├── ruby.rb
-│           └── hcl.rb
+│       ├── node_maps.rb          # NodeMap レジストリ
+│       └── node_maps/            # デフォルト NodeMap 定義
+│           ├── ruby.rb, hcl.rb, bash.rb, ...
 └── textbringer_plugin.rb         # エントリポイント
-parsers/                          # プリビルド parser (.dylib/.so)
-├── darwin-arm64/
-└── linux-x64/
+exe/
+└── textbringer-tree-sitter       # CLI tool（parser 管理）
 ```
 
 ### 主要コンポーネント
 
-- **TreeSitterAdapter**: Window クラスをモンキーパッチして `custom_highlight` を実装
-- **TreeSitterConfig**: プラットフォーム判定と parser ロード
-- **NodeMaps**: 言語固有のノードタイプ → Face マッピング
+- **TreeSitterAdapter**:
+  - Window#highlight をモンキーパッチして `custom_highlight` に差し替え
+  - Mode クラスに `use_tree_sitter(:lang)` を提供（prepend で注入）
+  - Emacs 風 4 段階ハイライトレベル制御（HIGHLIGHT_LEVELS）
+
+- **TreeSitterConfig**:
+  - プラットフォーム検出（darwin-arm64, linux-x64 等）
+  - Parser パス解決（CONFIG → ~/.textbringer/parsers → gem内 の優先順）
+  - デフォルト Face 定義（comment, string, keyword, ...）
+
+- **NodeMaps**:
+  - 言語ごとの `node_type → face` マッピング辞書
+  - `register(:lang, mapping)` でレジストリに登録
+  - ユーザーカスタム NodeMap は `~/.textbringer/tree_sitter/node_maps/` に配置
 
 ### 依存関係
 
-- `ruby_tree_sitter` (~> 2.0) - Faveod/ruby-tree-sitter
+- `ruby_tree_sitter` (~> 2.0) - LANGUAGE_VERSION 6-14 対応
 - `textbringer` (>= 1.0)
 
-## Tree-sitter Parser
+### カスタマイズ設定（~/.textbringer.rb）
 
-### LANGUAGE_VERSION 互換性
+```ruby
+# ハイライトレベル (1-4, default: 3)
+CONFIG[:tree_sitter_highlight_level] = 4
 
-ruby_tree_sitter 2.0.0 は LANGUAGE_VERSION 6-14 をサポート。parser ビルド時は互換性を確認：
+# 個別 feature 選択（レベルより優先）
+CONFIG[:tree_sitter_enabled_features] = [:comment, :string, :keyword]
 
+# カスタム parser ディレクトリ
+CONFIG[:tree_sitter_parser_dir] = "/path/to/parsers"
+
+# カスタム NodeMap 読み込み
+require "~/.textbringer/tree_sitter/node_maps/mylang.rb"
+```
+
+## NodeMap の追加方法
+
+新しい言語をサポートする手順:
+
+1. **Parser インストール**
+   ```bash
+   textbringer-tree-sitter get <lang>  # 自動で node_map も生成される
+   ```
+
+2. **生成された NodeMap を確認・編集**
+   - `~/.textbringer/tree_sitter/node_maps/<lang>.rb` に生成される
+   - ヒューリスティックで推測されたマッピングをレビュー
+   - コメント化された unmapped nodes を必要に応じて追加
+
+3. **~/.textbringer.rb で読み込み**
+   ```ruby
+   require "~/.textbringer/tree_sitter/node_maps/<lang>.rb"
+   ```
+
+4. **gem にコントリビュート**（オプション）
+   - `lib/textbringer/tree_sitter/node_maps/<lang>.rb` に配置
+   - デフォルト NodeMap として同梱
+
+## Tree-sitter Parser の互換性
+
+### LANGUAGE_VERSION
+
+ruby_tree_sitter 2.0.0 は **LANGUAGE_VERSION 6-14** をサポート。
+
+ビルド前に互換性確認:
 ```bash
 grep LANGUAGE_VERSION src/parser.c | head -1
 ```
 
-### HCL Parser のビルド
+### 手動ビルド例（HCL）
 
-mitchellh/tree-sitter-hcl を使用（LANGUAGE_VERSION 13）：
+CLI tool 使わずビルドする場合:
 
 ```bash
 git clone https://github.com/mitchellh/tree-sitter-hcl.git
 cd tree-sitter-hcl
 c++ -shared -fPIC -O2 -std=c++14 -Isrc src/parser.c src/scanner.cc -o libtree-sitter-hcl.dylib
+cp libtree-sitter-hcl.dylib ~/.textbringer/parsers/darwin-arm64/
 ```
 
-### プリビルド parser の取得
-
-Faveod/tree-sitter-parsers から取得可能：bash, c, javascript, json, python, ruby, rust 等
-
 ## 参考実装
 
 - [textbringer-rouge](https://github.com/yancya/textbringer-rouge) - Window モンキーパッチの実装パターン
+- [Faveod/tree-sitter-parsers](https://github.com/Faveod/tree-sitter-parsers) - prebuilt parsers の配布元

data/DISCLAIMER

@@ -0,0 +1,9 @@
+DISCLAIMER
+
+THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/LICENSE

@@ -0,0 +1,13 @@
+DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
+                   Version 2, December 2004
+
+Copyright (C) 2026 yancya <yancya@upec.jp>
+
+Everyone is permitted to copy and distribute verbatim or modified
+copies of this license document, and changing it is allowed as long
+as the name is changed.
+
+           DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
+  TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. You just DO WHAT THE FUCK YOU WANT TO.

data/README.md

@@ -15,20 +15,49 @@ Tree-sitter based syntax highlighting plugin for Textbringer.
 gem 'textbringer-tree-sitter'
 ```
 
-### Installing Parsers
+### Default Parsers
 
-Parsers are not bundled with the gem. Use the CLI tool to install them:
+The following parsers are **automatically installed** during `gem install`:
+
+- **ruby**
+- **python**
+- **javascript**
+- **json**
+- **bash**
+
+These are downloaded from [Faveod/tree-sitter-parsers](https://github.com/Faveod/tree-sitter-parsers) and placed in `~/.textbringer/parsers/{platform}/`.
+
+### Opt-out of Automatic Downloads
+
+To skip automatic parser downloads (useful in offline or restricted environments), set the environment variable:
 
 ```bash
-# List available parsers
+export TEXTBRINGER_TREE_SITTER_NO_DOWNLOAD=1
+gem install textbringer-tree-sitter
+```
+
+When this variable is set:
+- `gem install` will skip automatic parser downloads
+- CLI commands (`get`, `get-all`) will refuse to download with an error message
+- You can still manually place parsers in `~/.textbringer/parsers/{platform}/`
+
+### Installing Additional Parsers
+
+Use the CLI tool to install additional parsers:
+
+```bash
+# List available parsers and their installation status
 textbringer-tree-sitter list
 
 # Install a specific parser (downloads prebuilt or builds from source)
-textbringer-tree-sitter get ruby
+# Also generates a node_map if one doesn't exist in the gem
 textbringer-tree-sitter get hcl
 textbringer-tree-sitter get markdown
 
-# Install all prebuilt parsers at once
+# Install parser only, skip node_map generation
+textbringer-tree-sitter get markdown --no-map
+
+# Install all Faveod prebuilt parsers at once
 textbringer-tree-sitter get-all
 ```
 
@@ -43,7 +72,16 @@ textbringer-tree-sitter path
 
 ## Usage
 
-Once a parser is installed and a node_map exists for the language, syntax highlighting is automatically enabled for the corresponding Mode.
+### Automatic Highlighting
+
+Once a parser is installed and a node_map exists for the language, syntax highlighting is automatically enabled for the corresponding Mode (e.g., RubyMode, PythonMode, HCLMode).
+
+The `get` command:
+1. Downloads or builds the parser
+2. Automatically generates a node_map if one doesn't exist in the gem
+3. Places the node_map in `~/.textbringer/tree_sitter/node_maps/`
+
+### Custom Modes
 
 For custom Modes, call `use_tree_sitter`:
 
@@ -54,6 +92,59 @@ class MyMode < ProgrammingMode
 end
 ```
 
+## Custom Languages
+
+You can add languages not included in the gem by creating a configuration file.
+
+### 1. Initialize config file
+
+```bash
+textbringer-tree-sitter init
+```
+
+This creates `~/.textbringer/tree_sitter/languages.yml`.
+
+### 2. Edit the configuration file
+
+```yaml
+# Simple format (minimal config)
+elixir:
+  repo: elixir-lang/tree-sitter-elixir
+
+# Detailed format (full control)
+zig:
+  repo: maxxnino/tree-sitter-zig
+  branch: master
+  commit: abc123  # Optional: pin to specific commit
+  subdir: ""      # Optional: subdirectory within repo
+  build_cmd: "cc -shared -fPIC -O2 -I{src}/src {src}/src/parser.c -o {output}"
+
+# Use a fork instead of curated version
+ruby:
+  repo: my-username/tree-sitter-ruby
+  branch: experimental
+
+# Use Faveod prebuilt parser
+groovy:
+  source: faveod
+```
+
+### 3. Install the language
+
+```bash
+textbringer-tree-sitter get elixir
+```
+
+The custom language will override any curated language with the same name.
+
+### 4. List all languages
+
+```bash
+textbringer-tree-sitter list
+```
+
+This shows both curated (built-in) and user-defined languages.
+
 ## Customization
 
 ### Highlight Level (Emacs-style)
@@ -91,26 +182,59 @@ CONFIG[:tree_sitter_parser_dir] = "/path/to/your/parsers"
 
 ## Supported Languages
 
-### Prebuilt Parsers (via Faveod)
-
-bash, c, c-sharp, cobol, embedded-template, groovy, haml, html, java, javascript, json, pascal, php, python, ruby, rust
-
-### Build-required Parsers
-
-| Language | Command |
-|----------|---------|
-| HCL (Terraform) | `textbringer-tree-sitter get hcl` |
-| YAML | `textbringer-tree-sitter get yaml` |
-| Go | `textbringer-tree-sitter get go` |
-| TypeScript | `textbringer-tree-sitter get typescript` |
-| TSX | `textbringer-tree-sitter get tsx` |
-| SQL | `textbringer-tree-sitter get sql` |
-| Markdown | `textbringer-tree-sitter get markdown` |
+### Ready to Use (Prebuilt + Node Map Included)
+
+These parsers are available from Faveod and include node_maps in the gem:
+
+| Language | Auto-installed on `gem install` | Command |
+|----------|--------------------------------|---------|
+| bash | ✅ | `textbringer-tree-sitter get bash` |
+| c | | `textbringer-tree-sitter get c` |
+| c-sharp | | `textbringer-tree-sitter get c-sharp` |
+| cobol | | `textbringer-tree-sitter get cobol` |
+| embedded-template | | `textbringer-tree-sitter get embedded-template` |
+| groovy | | `textbringer-tree-sitter get groovy` |
+| haml | | `textbringer-tree-sitter get haml` |
+| html | | `textbringer-tree-sitter get html` |
+| java | | `textbringer-tree-sitter get java` |
+| javascript | ✅ | `textbringer-tree-sitter get javascript` |
+| json | ✅ | `textbringer-tree-sitter get json` |
+| pascal | | `textbringer-tree-sitter get pascal` |
+| php | | `textbringer-tree-sitter get php` |
+| python | ✅ | `textbringer-tree-sitter get python` |
+| ruby | ✅ | `textbringer-tree-sitter get ruby` |
+| rust | | `textbringer-tree-sitter get rust` |
+
+### Build-required (Node Map Included)
+
+These parsers require building from source but include node_maps:
+
+| Language | Command | Repository |
+|----------|---------|------------|
+| HCL (Terraform) | `textbringer-tree-sitter get hcl` | mitchellh/tree-sitter-hcl |
+| YAML | `textbringer-tree-sitter get yaml` | tree-sitter-grammars/tree-sitter-yaml |
+| SQL | `textbringer-tree-sitter get sql` | m-novikov/tree-sitter-sql |
+
+### Build-required (Node Map Not Included)
+
+These parsers require building from source and node_map generation:
+
+| Language | Command | Note |
+|----------|---------|------|
+| Go | `textbringer-tree-sitter get go` | Generates node_map in `~/.textbringer/tree_sitter/node_maps/` |
+| TypeScript | `textbringer-tree-sitter get typescript` | Generates node_map in `~/.textbringer/tree_sitter/node_maps/` |
+| TSX | `textbringer-tree-sitter get tsx` | Generates node_map in `~/.textbringer/tree_sitter/node_maps/` |
+| Markdown | `textbringer-tree-sitter get markdown` | Generates node_map in `~/.textbringer/tree_sitter/node_maps/` |
+
+To regenerate a node_map manually:
+```bash
+textbringer-tree-sitter generate-map <language>
+```
 
 ## License
 
-WTFPL - See LICENSE.txt for details.
+WTFPL - See [LICENSE](LICENSE) for details.
 
 ## Disclaimer
 
-THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+See [DISCLAIMER](DISCLAIMER) for details.

data/Rakefile

@@ -9,4 +9,21 @@ Rake::TestTask.new(:test) do |t|
   t.test_files = FileList["test/**/test_*.rb"]
 end
 
+namespace :parsers do
+  desc "Download prebuilt parsers for CI (ruby, hcl, markdown)"
+  task :download do
+    sh "bash scripts/download_parsers.sh ruby hcl markdown"
+  end
+
+  desc "Build parsers from source (HCL, Ruby)"
+  task :build do
+    sh "bash scripts/build_parsers.sh"
+  end
+
+  desc "Setup parsers for testing (downloads if available, falls back to build)"
+  task :setup do
+    sh "bash scripts/download_parsers.sh ruby hcl markdown || bash scripts/build_parsers.sh"
+  end
+end
+
 task default: :test