@mars167/git-ai 2.3.0

package/docs/README.md

@@ -0,0 +1,44 @@
+# Documentation Center
+
+This collects all documentation for `git-ai`.
+
+## Overview
+
+`git-ai` is a global CLI:
+- Default behavior acts like `git`: `git-ai status/commit/push/...` proxies to system `git`.
+- AI capabilities are under `git-ai ai ...`: Indexing, Retrieval, Packing, Hooks, MCP Server.
+
+### Core Goals
+- Store structured code repository indexes under `.git-ai/`, shareable via archive `.git-ai/lancedb.tar.gz`.
+- Enable Agents to hit symbols/snippets via MCP tools at low cost, then read files as needed.
+- Persist per-commit semantic change as DSR (immutable, deterministic), and rebuild caches from it.
+
+### Important Directories
+- `.git-ai/meta.json`: Index metadata (locally generated, usually not committed).
+- `.git-ai/lancedb/`: Local vector index directory (usually not committed).
+- `.git-ai/lancedb.tar.gz`: Archived index (can be committed/tracked via git-lfs).
+- `.git-ai/ast-graph.sqlite`: AST graph database (CozoDB).
+- `.git-ai/ast-graph.export.json`: AST graph export snapshot (for non-SQLite backend cross-process reuse).
+- `.git-ai/dsr/<commit_hash>.json`: Per-commit DSR (canonical artifact, immutable).
+- `.git-ai/dsr/dsr-index.sqlite`: DSR query accelerator (rebuildable cache from DSR + Git).
+
+## Directory
+
+### Usage Guides
+- [Installation & Quick Start](./zh-CN/quickstart.md) (Chinese)
+- [Windows Setup Guide](./windows-setup.md)
+- [CLI Usage](./zh-CN/cli.md) (Chinese)
+- [Hooks Workflow](./zh-CN/hooks.md) (Chinese)
+- [MCP Server Integration](./zh-CN/mcp.md) (Chinese)
+- [Manifest Workspace Support](./zh-CN/manifests.md) (Chinese)
+- [Troubleshooting](./zh-CN/troubleshooting.md) (Chinese)
+- [DSR (Deterministic Semantic Record)](./zh-CN/dsr.md) (Chinese)
+
+### Advanced & Principles
+- [Advanced: Index Archiving & LFS](./zh-CN/advanced.md) (Chinese)
+- [Architecture Design](./zh-CN/design.md) (Chinese)
+- [Development Rules](./zh-CN/rules.md) (Chinese)
+- [Cross-Encoder Reranking](./cross-encoder.md) (English)
+
+## Agent Integration
+- [MCP Skill & Rule Templates](./zh-CN/mcp.md#agent-skills--rules) (Chinese)

package/docs/cross-encoder.md

@@ -0,0 +1,157 @@
+# Cross-Encoder Reranking & ONNX Runtime
+
+## Overview
+
+git-ai v2.2+ includes an optional **Cross-Encoder Reranking** feature that uses ONNX Runtime for high-quality result re-ranking. This is an optional enhancement that improves search result quality when a model is available.
+
+## Architecture
+
+```
+Query → [Vector Search] → [Graph Search] → [DSR Search] → [Cross-Encoder Rerank] → Results
+```
+
+The cross-encoder takes query-candidate pairs and scores their relevance, providing higher quality re-ranking than simple score fusion.
+
+## Configuration
+
+### Model Path
+
+The cross-encoder uses a configurable model path. By default, it looks for:
+1. `<modelName>` (as absolute or relative path)
+2. `<modelName>/model.onnx`
+3. `<modelName>/onnx/model.onnx`
+
+The default model name is `non-existent-model.onnx`, which means the system will use hash-based fallback by default.
+
+```typescript
+// Reranker configuration
+interface RerankerConfig {
+  modelName: string;      // Path to ONNX model
+  device: 'cpu' | 'gpu';  // Execution device
+  batchSize: number;      // Batch processing size
+  topK: number;           // Max candidates to re-rank
+  scoreWeights: {
+    original: number;      // Weight for original retrieval score
+    crossEncoder: number;  // Weight for cross-encoder score
+  };
+}
+```
+
+### Default Behavior
+
+When no model is found, the system automatically falls back to **hash-based scoring**:
+- Uses `hashEmbedding` to create query-content vectors
+- Computes similarity via sigmoid(sum)
+- No external dependencies required
+
+This ensures the system works even without ONNX models.
+
+## Installing ONNX Models
+
+To enable cross-encoder reranking, download a compatible model (e.g., MiniLM, CodeBERT) and configure the path:
+
+```bash
+# Example: Download a cross-encoder model
+mkdir -p models/cross-encoder
+cd models/cross-encoder
+# Download your ONNX model (e.g., from HuggingFace, ONNX Model Zoo)
+# Place model.onnx in this directory
+```
+
+## Performance Considerations
+
+### Memory
+- ONNX Runtime loads models into memory
+- GPU memory required for GPU inference
+- CPU inference works on any modern CPU
+
+### Batch Processing
+- Configure `batchSize` based on available memory
+- Larger batches = better throughput but more memory
+
+### Supported Backends
+- **CPU**: All platforms, no additional setup
+- **GPU**: CUDA-enabled systems (optional CUDA execution provider)
+
+## API Usage
+
+### CLI (Not yet exposed)
+
+Cross-encoder is currently used internally by the retrieval pipeline.
+
+### Programmatic
+
+```typescript
+import { CrossEncoderReranker } from 'git-ai';
+
+const reranker = new CrossEncoderReranker({
+  modelName: './models/cross-encoder',
+  device: 'cpu',
+  batchSize: 32,
+  topK: 100,
+  scoreWeights: {
+    original: 0.3,
+    crossEncoder: 0.7,
+  },
+});
+
+const results = await reranker.rerank('authentication logic', candidates);
+```
+
+## Fallback Mechanism
+
+The system handles missing models gracefully:
+
+1. **Model file missing** → Log `cross_encoder_model_missing` and use hash fallback
+2. **ONNX load failed** → Log `cross_encoder_fallback` and use hash fallback
+3. **Inference error** → Log error and continue with fallback
+
+No crashes or service interruption when model is unavailable.
+
+## Comparison: Hash vs ONNX
+
+| Aspect | Hash Fallback | ONNX Cross-Encoder |
+|--------|---------------|-------------------|
+| Quality | Good for exact matches | Excellent for semantic matching |
+| Speed | <1ms | 10-100ms (depending on model) |
+| Dependencies | None | onnxruntime-node |
+| Memory | <1MB | 50-500MB (model size) |
+| GPU Required | No | Optional |
+
+## Troubleshooting
+
+### Model Load Failed
+
+```
+{"level":"warn","msg":"cross_encoder_fallback","err":"..."}
+```
+
+Causes:
+- Model file doesn't exist
+- Corrupted model file
+- Incompatible ONNX opset version
+
+Solution:
+1. Verify model path is correct
+2. Check model file is valid ONNX
+3. Ensure onnxruntime-node is installed
+
+### Out of Memory
+
+Reduce `batchSize` in configuration or use CPU backend.
+
+### Slow Inference
+
+- Use smaller models (MiniLM instead of large BERT)
+- Enable batching for multiple queries
+- Consider GPU for large-scale usage
+
+## Dependencies
+
+```json
+{
+  "onnxruntime-node": "^1.19.2"
+}
+```
+
+Required for cross-encoder functionality. Optional - system works without it.

package/docs/embedding.md

@@ -0,0 +1,158 @@
+# Embedding Models
+
+git-ai uses ONNX-compatible embedding models for semantic code search. This document covers model configuration, available options, and setup instructions.
+
+## Overview
+
+The embedding system converts code snippets into vector representations for similarity search. git-ai supports:
+
+- **Semantic Embedding**: Neural network-based code representation (CodeBERT, MiniLM)
+- **Structural Embedding**: AST-based structural features (WL kernel hashing)
+- **Symbolic Embedding**: Identifier and symbol relationships
+
+## Configuration
+
+### Environment Variable
+
+Set `GIT_AI_EMBEDDING_MODEL` to override the default embedding model:
+
+```bash
+export GIT_AI_EMBEDDING_MODEL="$HOME/.cache/git-ai/models/minilm/model.onnx"
+```
+
+Add to your shell profile for permanent use:
+
+```bash
+# ~/.zshrc or ~/.bashrc
+export GIT_AI_EMBEDDING_MODEL="$HOME/.cache/git-ai/models/minilm/model.onnx"
+```
+
+### Default Paths
+
+| Model | Default Path |
+|-------|-------------|
+| CodeBERT | `~/.cache/git-ai/models/codebert/model.onnx` |
+| MiniLM | `~/.cache/git-ai/models/minilm/model.onnx` |
+
+The system automatically detects the model type and sets the appropriate embedding dimension:
+- CodeBERT: 768 dimensions
+- MiniLM-L6: 384 dimensions
+
+## Available Models
+
+### MiniLM-L6 (Recommended)
+
+Lightweight, fast model ideal for local development.
+
+- **Size**: ~86MB
+- **Dimensions**: 384
+- **Speed**: Fast (<100ms per query)
+- **Download**:
+
+```python
+from huggingface_hub import hf_hub_download
+
+hf_hub_download(
+    repo_id="Xenova/all-MiniLM-L6-v2",
+    filename="onnx/model.onnx",
+    local_dir="$HOME/.cache/git-ai/models/minilm"
+)
+```
+
+### CodeBERT
+
+Microsoft CodeBERT for code understanding.
+
+- **Size**: ~500MB
+- **Dimensions**: 768
+- **Quality**: Higher semantic understanding
+- **Download**:
+
+```bash
+huggingface-cli download onnx-community/codebert-javascript-ONNX \
+  --local-dir "$HOME/.cache/git-ai/models/codebert"
+```
+
+## Model Directory Structure
+
+```
+~/.cache/git-ai/models/
+├── codebert/
+│   ├── model.onnx          # ONNX model file
+│   └── config.json         # Model configuration
+└── minilm/
+    ├── model.onnx -> onnx/model.onnx  # Symlink to ONNX model
+    ├── onnx/
+    │   └── model.onnx
+    └── config.json
+```
+
+## Fallback Behavior
+
+If no model is found, git-ai automatically falls back to hash-based embedding:
+
+- **Quality**: Good for exact matches
+- **Speed**: <1ms
+- **Memory**: <1MB
+- **Dependencies**: None
+
+No crashes or service interruption when model is unavailable.
+
+## Performance Considerations
+
+| Model | Memory | CPU Inference | GPU Recommended |
+|-------|--------|---------------|-----------------|
+| MiniLM | ~200MB | Excellent | Optional |
+| CodeBERT | ~800MB | Good | Yes |
+
+### Batch Processing
+
+Configure batch size in environment:
+
+```bash
+export GIT_AI_EMBEDDING_BATCH_SIZE=8
+```
+
+## Troubleshooting
+
+### Model Load Failed
+
+```
+{"level":"warn","msg":"semantic_embed_fallback","err":"..."}
+```
+
+Causes:
+- Model file doesn't exist
+- Corrupted model file
+- Incompatible ONNX opset version
+
+Solution:
+1. Verify model path is correct
+2. Check model file is valid ONNX
+3. Ensure onnxruntime-node is installed
+
+### Dimension Mismatch
+
+If you see dimension errors, verify the model path matches the expected dimension:
+- MiniLM: 384 dimensions
+- CodeBERT: 768 dimensions
+
+## Comparison
+
+| Aspect | MiniLM | CodeBERT | Hash Fallback |
+|--------|--------|----------|---------------|
+| Size | 86MB | 500MB | <1MB |
+| Dimensions | 384 | 768 | N/A |
+| Speed | <100ms | 100-500ms | <1ms |
+| Quality | Good | Excellent | Exact matches |
+| Memory | Low | High | Minimal |
+
+## Dependencies
+
+```json
+{
+  "onnxruntime-node": "^1.19.2"
+}
+```
+
+Required for embedding functionality. Optional - the system works with hash fallback without it.

package/docs/windows-setup.md

@@ -0,0 +1,67 @@
+# Windows Development and Installation Guide
+
+[简体中文](./zh-CN/windows-setup.md) | **English**
+
+This guide describes how to set up the development environment for `git-ai` on Windows, specifically for the multi-language support (C, Go, Python, Rust).
+
+## Prerequisites
+
+1.  **Node.js**: Install Node.js (LTS version recommended) from [nodejs.org](https://nodejs.org/).
+2.  **Git**: Install Git for Windows from [git-scm.com](https://git-scm.com/).
+
+## Build Tools for Native Dependencies
+
+`git-ai` relies on libraries with native bindings:
+*   `tree-sitter`: For code parsing (C++)
+*   `cozo-node`: Graph database engine (Rust/C++)
+
+While these libraries typically provide prebuilt binaries, you may need to build from source in certain environments (e.g., mismatched Node versions or specific architectures). Therefore, setting up a build environment is recommended.
+
+### Option 1: Install via Admin PowerShell (Recommended)
+
+Open PowerShell as Administrator and run:
+
+```powershell
+npm install --global --production windows-build-tools
+```
+
+*Note: This package is sometimes deprecated or problematic. If it hangs or fails, use Option 2.*
+
+### Option 2: Manual Installation
+
+1.  **Python**: Install Python 3 from [python.org](https://www.python.org/) or the Microsoft Store.
+2.  **Visual Studio Build Tools**:
+    *   Download [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/).
+    *   Run the installer and select the **"Desktop development with C++"** workload.
+    *   Ensure "MSVC ... C++ x64/x86 build tools" and "Windows 10/11 SDK" are selected.
+
+## Installation
+
+Once prerequisites are met:
+
+```bash
+git clone https://github.com/mars167/git-ai-cli.git
+cd git-ai-cli-v2
+npm install
+npm run build
+```
+
+## Running Examples
+
+To verify support for different languages, you can run the parsing test:
+
+```bash
+npx ts-node test/verify_parsing.ts
+```
+
+To fully develop with the polyglot examples, you may need to install the respective language runtimes:
+
+*   **C**: Install MinGW or use MSVC (cl.exe).
+*   **Go**: Install from [go.dev](https://go.dev/dl/).
+*   **Python**: [python.org](https://www.python.org/).
+*   **Rust**: Install via [rustup.rs](https://rustup.rs/).
+
+## Troubleshooting
+
+*   **node-gyp errors**: Ensure Python and Visual Studio Build Tools are correctly installed and in PATH. You can configure npm to use a specific python version: `npm config set python python3`.
+*   **Path issues**: Ensure `git-ai` binary or `npm bin` is in your PATH if running globally.

package/docs/zh-CN/DESIGN.md

@@ -0,0 +1,102 @@
+# git-ai Design（LanceDB + SQ8 + 去重）
+
+## 1. 目标
+- 使用 LanceDB 作为本地索引存储（列式 + 可扩展）。
+- 引入 SQ8（8-bit 标量量化）降低向量存储体积。
+- 引入内容哈希去重：相同内容只存一份向量，多处引用仅存引用关系。
+- 索引口径：仅针对当前 checkout 的 HEAD 工作区；历史版本由 Git 负责管理（通过 checkout 获得对应索引快照）。
+- 提供按提交的 DSR（Deterministic Semantic Record）作为语义工件：按提交、不可变、确定性；数据库仅作可重建缓存。
+
+## 2. 存储布局
+索引产物放在仓库根目录：
+- `.git-ai/`：索引目录
+  - `lancedb/`：LanceDB 数据目录
+  - `lancedb.tar.gz`：打包后的 LanceDB（用于 Git LFS 追踪与传输）
+  - `ast-graph.sqlite`：AST 关系图数据库（CozoDB，优先 SQLite 引擎）
+  - `ast-graph.export.json`：AST 图导出快照（仅在非 SQLite 后端时用于跨进程复用）
+  - `meta.json`：索引元信息（维度、编码、构建时间等）
+
+DSR 产物（按提交，规范工件）：
+- `.git-ai/dsr/<commit_hash>.json`：单提交 DSR（不可变、确定性）
+- `.git-ai/dsr/dsr-index.sqlite`：DSR 查询加速索引（可删缓存，可由 DSR + Git 重建）
+
+## 3. 数据模型（两张表）
+
+### 3.1 chunks（去重后的内容向量表）
+- 一行代表一个“去重后的内容块”（例如某个符号的骨架/签名文本）。
+- 主键：`content_hash`（sha256）
+- 字段：
+  - `content_hash: string`
+  - `text: string`（骨架/签名等用于可解释）
+  - `dim: int32`
+  - `scale: float32`（SQ8 反量化比例）
+  - `qvec_b64: string`（SQ8 量化向量，Int8Array 的 base64 编码）
+
+### 3.2 refs（引用表）
+- 一行代表一次出现位置（文件/行号/符号等），指向 chunks 的 `content_hash`。
+- 字段：
+  - `ref_id: string`（sha256(file + symbol + range + content_hash)）
+  - `content_hash: string`
+  - `file: string`
+  - `symbol: string`
+  - `kind: string`
+  - `signature: string`
+  - `start_line: int32`
+  - `end_line: int32`
+
+## 4. 向量生成与 SQ8
+- v2 默认不依赖外部 embedding API：使用确定性的本地 hash embedding（维度固定）生成 float 向量。
+- SQ8（对称量化）：
+  - `scale = max(|v|)/127`
+  - `q[i] = clamp(round(v[i]/scale), -127..127)`
+  - 反量化：`v'[i] = q[i] * scale`
+
+## 5. 去重策略
+- `content_hash = sha256(text)`，同一 text 只写入 chunks 一次。
+- refs 始终写入，形成多对一关系。
+
+## 6. 查询能力
+- `search_symbols(query)`：在 refs 表过滤 `symbol LIKE %query%` 返回文件 + 行号 + signature。
+- `semantic_search(text, k)`：
+  - 计算 query embedding → SQ8；
+  - 扫描 chunks（或按过滤条件缩小）反量化计算 cosine 相似度；
+  - 取 TopK 后关联 refs 输出定位结果。
+
+## 6.1 AST 图查询（CozoDB）
+
+索引时会把符号及其关系写入 CozoDB，用于表达“包含关系”和“继承关系”等更适合图/递归查询的数据：
+
+### 关系（relations）
+- `ast_file(file_id => file)`：文件节点（file_id 为 `sha256("file:" + file)`）
+- `ast_symbol(ref_id => file, name, kind, signature, start_line, end_line)`：符号节点（ref_id 与 refs 表一致）
+- `ast_contains(parent_id, child_id)`：包含关系边（parent_id 可能是 file_id 或 ref_id）
+- `ast_extends_name(sub_id, super_name)`：继承关系（按名字记录，便于后续 join/解析）
+- `ast_implements_name(sub_id, iface_name)`：实现关系（按名字记录）
+
+### CLI / MCP
+- CLI：`git-ai ai graph ...`
+- MCP：`ast_graph_query({query, params})`
+
+## 7. Git hooks 集成
+- `pre-commit`：自动重建索引（index --overwrite）并打包（pack），把 `.git-ai/lancedb.tar.gz` 添加到暂存区；若安装了 git-lfs 会自动执行 lfs track。
+- `pre-push`：再次打包并校验归档未发生变化；若变化则阻止 push，提示先提交归档文件。
+- `post-checkout` / `post-merge`：若存在 `.git-ai/lancedb.tar.gz`，自动解包到 `.git-ai/lancedb/`。
+- 安装方式：在仓库中执行 `git-ai ai hooks install`（写入 .githooks/* 并设置 core.hooksPath=.githooks）。
+
+## 8. DSR（Deterministic Semantic Record）
+
+DSR 用于固化“每个提交的语义变化”，并严格遵守：
+
+- Git DAG 是历史/分支的唯一权威来源（DSR 只 enrich 节点，不定义边）
+- 一次提交 → 一份 DSR 文件（`.git-ai/dsr/<commit_hash>.json`）
+- DSR 一旦生成不可覆盖；若已存在且内容不同应停止并报错
+- 任何数据库/索引必须可由 DSR + Git 重建（缓存可删）
+
+CLI 入口：
+
+- `git-ai ai dsr context`
+- `git-ai ai dsr generate <commit>`
+- `git-ai ai dsr rebuild-index`
+- `git-ai ai dsr query symbol-evolution <symbol>`
+
+更详细说明见：[DSR 文档](./dsr.md)

package/docs/zh-CN/README.md

@@ -0,0 +1,46 @@
+# 文档中心
+
+[**English**](../README.md) | 简体中文
+
+这里汇集了 `git-ai` 的所有文档。
+
+## 概览
+
+`git-ai` 是一个全局 CLI：
+- 默认行为像 `git`：`git-ai status/commit/push/...` 会代理到系统 `git`
+- AI 能力放在 `git-ai ai ...`：索引、检索、归档、hooks、MCP Server
+
+### 核心目标
+- 把代码仓的结构化索引放在 `.git-ai/` 下，并可通过归档文件 `.git-ai/lancedb.tar.gz` 分享
+- 让 Agent 通过 MCP tools 低成本命中符号/片段，再按需读取文件
+- 把每个提交的语义变化固化为 DSR（按提交、不可变、确定性），并可据此重建缓存
+
+### 重要目录
+- `.git-ai/meta.json`：索引元数据（本地生成，通常不提交）
+- `.git-ai/lancedb/`：本地向量索引目录（通常不提交）
+- `.git-ai/lancedb.tar.gz`：归档后的索引（可提交/可用 git-lfs 追踪）
+- `.git-ai/ast-graph.sqlite`：AST 图数据库（CozoDB）
+- `.git-ai/ast-graph.export.json`：AST 图导出快照（用于非 SQLite 后端跨进程复用）
+- `.git-ai/dsr/<commit_hash>.json`：单提交 DSR（规范工件，按提交、不可变）
+- `.git-ai/dsr/dsr-index.sqlite`：DSR 查询加速索引（可删缓存，可由 DSR + Git 重建）
+
+## 目录
+
+### 使用指引
+- [安装与快速开始](./quickstart.md)
+- [Windows 开发与安装指引](./windows-setup.md)
+- [命令行使用](./cli.md)
+- [Hooks 工作流](./hooks.md)
+- [MCP Server 接入](./mcp.md)
+- [Manifest Workspace 支持](./manifests.md)
+- [排障](./troubleshooting.md)
+- [DSR（Deterministic Semantic Record）](./dsr.md)
+
+### 进阶与原理
+- [技术细节详解](./technical-details.md)
+- [进阶：索引归档与 LFS](./advanced.md)
+- [架构设计](./design.md)
+- [开发规则](./rules.md)
+
+## Agent 集成
+- [MCP Skill & Rule 模版](./mcp.md#agent-skills--rules)

package/docs/zh-CN/advanced.md

@@ -0,0 +1,26 @@
+# 索引归档与 LFS
+
+## pack/unpack
+- `pack`：把 `.git-ai/lancedb/` 打包为 `.git-ai/lancedb.tar.gz`
+- `unpack`：把 `.git-ai/lancedb.tar.gz` 解包为 `.git-ai/lancedb/`
+
+```bash
+git-ai ai pack
+git-ai ai unpack
+```
+
+## Git LFS（可选）
+如果仓库安装了 git-lfs，推荐对 `.git-ai/lancedb.tar.gz` 使用 LFS：
+
+```bash
+git lfs install
+git lfs track ".git-ai/lancedb.tar.gz"
+git add .gitattributes
+git commit -m "track lancedb archive via lfs"
+```
+
+也可以运行一次：
+
+```bash
+git-ai ai pack --lfs
+```