@zvec/zvec-grep 0.1.5 → 0.1.6

package/docs/04-pipeline.md

@@ -0,0 +1,218 @@
+# Retrieval pipeline
+
+[Documentation](./README.md) · [Agents](./01-agents.md) ·
+[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
+[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
+[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
+
+zg provides one search layer across semantic discovery, ranked lexical search,
+and exhaustive ripgrep. The normal flow is:
+
+```text
+workspace → file discovery → local index → query routes → compact results
+```
+
+The index combines lexical and vector retrieval. Managed ripgrep follows the
+same workspace and output conventions but can run without an index.
+
+## 1. Choose the workspace scope
+
+Run indexing from the repository root, or pass it explicitly:
+
+```bash
+cd your-repository
+zg index --embedding local/potion-code-16m-v2
+
+# Equivalent with an explicit root
+zg index /absolute/path/to/your-repository \
+  --embedding local/potion-code-16m-v2
+```
+
+The workspace index is stored under `<root>/.zvec-grep/`. `.git` and
+`.zvec-grep` are always excluded. Common dependency, build, generated, cache,
+and log directories are excluded by default, as are files ignored by the
+repository's ignore rules.
+
+The main workspace files are `manifest.json`, `files.zvec`, and `index.zvec`.
+The manifest stores index metadata and the workspace Embedding runtime settings,
+including an API key when one was explicitly persisted for that workspace.
+
+Scope large repositories early:
+
+```bash
+zg index \
+  --embedding local/potion-code-16m-v2 \
+  -g "src/**" \
+  -g "docs/**" \
+  -g "!dist/**" \
+  -t ts
+```
+
+Useful discovery controls include:
+
+| Option | Effect |
+| --- | --- |
+| `-g, --glob <glob>` | Add an ordered include or `!` exclude rule |
+| `--iglob <glob>` | Add a case-insensitive glob rule |
+| `-t, --type <type>` | Include a ripgrep file type |
+| `-T, --type-not <type>` | Exclude a ripgrep file type |
+| `--hidden` | Include hidden paths except `.git` and `.zvec-grep` |
+| `--no-ignore` | Stop applying ignore files |
+| `--ignore-file <path>` | Add an ignore file |
+| `--max-depth <n>` | Limit recursive depth |
+| `--max-filesize <size>` | Limit file size, for example `500K` or `2M` |
+| `-L, --follow` | Follow symbolic links safely |
+
+File-type filters narrow the result after glob rules. For example,
+`-g "docs/**" -t ts` selects TypeScript files inside `docs`, not every file in
+that directory.
+
+### Supported formats and extraction
+
+The scanner assigns each admitted file to one extraction path. Structure-aware
+extractors preserve useful code symbols or Markdown sections. When structure is
+not available, zg falls back to plain-text chunks so the file can still
+participate in indexed search.
+
+| Files | Formats | Extractor | Indexed representation |
+| --- | --- | --- | --- |
+| Structure-aware code | C/C++ (`.c`, `.cc`, `.cpp`, `.cxx`, `.h`, `.hpp`), Go, Java, JavaScript/JSX, TypeScript/TSX, Python, Rust | `CodeExtractor` | Symbols, signatures, breadcrumbs, and surrounding source |
+| Component scripts | `.vue`, `.svelte` | `CodeExtractor` | JavaScript or TypeScript `<script>` blocks; plain-text fallback when no structure is found |
+| Other recognized code | Ruby, PHP, Swift, Kotlin, C#, Scala, shell, SQL, CSS/SCSS/Less, `Dockerfile`, `Makefile` | `CodeExtractor` | Plain-text chunks until a structural grammar is available |
+| Markdown | `.md`, `.mdx` | `MarkdownExtractor` | Heading sections and breadcrumbs; plain-text fallback for documents without headings |
+| Text documents | `.txt`, `.rst`, `.html`, `.htm`, `.xml` | `TextExtractor` | Plain-text chunks |
+| Text data | `.csv`, `.json`, `.jsonc`, `.toml`, `.yaml`, `.yml` | `TextExtractor` | Plain-text chunks |
+| Other non-binary files | Unrecognized extensions that pass binary detection | `TextExtractor` | Plain-text chunks |
+| Raster images | `.gif`, `.jpeg`, `.jpg`, `.png`, `.webp` | `ImageExtractor` | Image content when explicitly included and the selected Embedding model accepts images |
+
+Raster images are excluded by the default discovery rules and must be selected
+explicitly. A text-only Embedding model cannot add image fragments to its
+vector index.
+
+The following binary formats are currently skipped before extraction:
+
+- documents: `.pdf`, `.doc`, `.docx`, `.ppt`, `.pptx`, `.xls`, `.xlsx`;
+- archives: `.zip`, `.tar`, `.gz`, `.bz2`, `.xz`, `.7z`, `.rar`;
+- executables and compiled artifacts: `.exe`, `.dll`, `.dylib`, `.so`, `.a`,
+  `.o`, `.obj`, `.wasm`, `.class`, `.jar`;
+- media and databases: `.mp3`, `.mp4`, `.mov`, `.avi`, `.mkv`, `.db`,
+  `.sqlite`.
+
+Empty files, files above the configured size limit, and files detected as
+binary are also skipped. Skipped files do not become extraction failures and
+are not included in the current `filesScanned` count, so the index summary does
+not list each skipped path or reason.
+
+<a id="indexing"></a>
+
+## 2. Build and maintain the index
+
+A new index resolves its model from explicit `--embedding`,
+`ZVEC_GREP_EMBEDDING`, then the configured default. Existing indexes reuse their
+stored model and file-selection settings:
+
+```bash
+# First build
+zg index --embedding local/potion-code-16m-v2
+
+# Incremental update with the stored schema
+zg index
+```
+
+Use `zg status` to see the root, selected model, file counts, failures,
+truncation, and the suggested next action:
+
+```bash
+zg status
+zg status --check-ready
+```
+
+Changing the Embedding model or an incompatible endpoint requires an explicit
+rebuild:
+
+```bash
+zg index --rebuild --embedding local/jina-embeddings-v2-base-code
+```
+
+Use `--reset-paths` when the existing file-selection settings should be
+replaced rather than reused. Deleting an index is explicit and destructive:
+
+```bash
+zg index --drop --yes
+```
+
+See [Embedding models](./07-embedding.md) before choosing or changing a model.
+
+<a id="querying"></a>
+
+## 3. Query through one search layer
+
+The shortest query uses hybrid ranked retrieval:
+
+```bash
+zg query "where theme preferences are restored"
+```
+
+Choose an explicit route only when you need more control:
+
+| Route | Use it for | Coverage |
+| --- | --- | --- |
+| Positional query or `--hybrid` | Intent plus useful lexical anchors | Ranked sample |
+| `--fts` | Exact terms ranked through the index | Ranked sample |
+| `--vector` | Conceptual similarity without lexical ranking | Ranked sample |
+| `--rg` | Exhaustive literal or regex matching | Exhaustive unless explicitly bounded |
+
+Examples:
+
+```bash
+# Ranked lexical search
+zg query --fts "AuthService"
+
+# Explicit semantic search
+zg query --vector "where credentials are validated"
+
+# Combine and fuse several query groups
+zg query \
+  --hybrid "authentication flow" \
+  --fts "ForbiddenError" \
+  --fuse \
+  --limit 10
+
+# No index required
+zg query --rg -n -F "AuthService" -g "*.ts" src
+```
+
+Multiple positional queries remain separate groups unless `--fuse` is set.
+Use `-g`, `--iglob`, `-t`, and `-T` on indexed queries to narrow results. Managed
+rg also accepts common ripgrep matching, context, engine, encoding, discovery,
+glob, and type options.
+
+zg owns the managed-rg result format, so output-changing options such as
+`--json`, `--count`, `--files`, `-l`, `-o`, `--replace`, and `--vimgrep` are
+rejected. Use `-A`, `-B`, or `-C` to add context.
+
+## Freshness
+
+Indexed results report `fresh` or `possibly_stale`. A possibly stale result can
+remain immediately useful while an update is pending. See
+[Server and execution modes](./06-server.md#refresh-behavior) for the interaction
+between `auto`, `server`, `direct`, and `--refresh`.
+
+## Output for agents and people
+
+Default CLI output is compact and grouped by file. Indexed source previews are
+omitted unless requested, reducing context passed to an agent:
+
+```bash
+zg query "plugin lifecycle" --preview short --limit 5
+```
+
+For terminal reading, `--human` enables richer presentation and defaults to a
+full preview:
+
+```bash
+zg query --human "plugin lifecycle" --limit 5
+```
+
+Use `--debug` for query diagnostics and `--trace` for per-hit indexed search
+trace information.

package/docs/05-architecture.md

@@ -0,0 +1,86 @@
+# Architecture
+
+[Documentation](./README.md) · [Agents](./01-agents.md) ·
+[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
+[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
+[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
+
+zg presents one search layer to people and agents while keeping execution,
+retrieval, and storage local by default.
+
+## System at a glance
+
+```mermaid
+flowchart LR
+  Human["Human or script"] --> CLI["zg CLI"]
+  Agent["Agent"] --> MCP["MCP client"]
+
+  CLI --> Router{"auto / server / direct"}
+  MCP --> Server["Local Server"]
+  Router --> Server
+  Router --> Direct["Direct runtime"]
+
+  Server --> Engine["zvec-grep engine"]
+  Direct --> Engine
+
+  Engine --> Indexed["Indexed search<br/>BM25 + vector + RRF"]
+  Engine --> RG["Managed ripgrep<br/>exact text + regex"]
+  Engine --> Indexing["Indexing<br/>scan + extract + embed"]
+
+  Files[("Workspace files")] --> RG
+  Files --> Indexing
+  Indexing --> Index[("Workspace index<br/>.zvec-grep/")]
+  Index --> Indexed
+
+  Indexed --> Results["Compact results"]
+  RG --> Results
+```
+
+## Entry and execution
+
+People and scripts enter through the CLI. Agents normally enter through the
+local Streamable HTTP MCP endpoint configured by `zg install`.
+
+The CLI routes indexed operations through `auto`, `server`, or `direct` mode.
+Both Server and Direct modes call the same engine; the difference is process
+lifetime and coordination, not search behavior. MCP requests always arrive
+through the Server. Managed `zg query --rg` can run directly without a Server
+or index.
+
+See [Agent integrations](./01-agents.md), [CLI](./02-cli.md),
+[MCP](./03-mcp.md), and [Server and execution modes](./06-server.md) for the
+individual interfaces.
+
+## Retrieval paths
+
+The engine exposes two complementary paths behind the same product boundary:
+
+| Path | Best for | Data source |
+| --- | --- | --- |
+| Indexed retrieval | Intent, related concepts, and ranked keywords | BM25/FTS and vector data in the workspace index |
+| Managed ripgrep | Known text, symbols, paths, and regular expressions | Workspace files scanned directly |
+
+Indexed search can combine lexical and vector candidates and fuse their ranks
+with reciprocal rank fusion (RRF). Managed ripgrep is exhaustive by default and
+does not require an Embedding model. Both paths apply workspace-aware filtering
+and return file-oriented results suitable for terminal reading or agent context.
+
+The [Retrieval pipeline](./04-pipeline.md) covers indexing, freshness, filters,
+and route selection in detail.
+
+## State and trust boundary
+
+The normal repository index lives under `<workspace>/.zvec-grep/`. Global
+configuration and daemon state live under `~/.zvec-grep/`. Workspace scanning,
+managed ripgrep, index storage, and local Embedding models remain on the local
+machine, and the Server listens on loopback only.
+
+Selecting a remote Embedding provider is the one path that can send query text
+or workspace content outside the machine. zg requests explicit once-only or
+workspace authorization before that transfer. MCP Bearer authentication protects
+the local endpoint; it does not authorize remote Embedding.
+
+See [Embedding models](./07-embedding.md#remote-embedding-and-authorization)
+for provider authorization and
+[Server authentication](./06-server.md#bearer-authentication) for the local
+endpoint boundary.

package/docs/06-server.md

@@ -0,0 +1,210 @@
+# Server and execution modes
+
+[Documentation](./README.md) · [Agents](./01-agents.md) ·
+[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
+[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
+[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
+
+The zvec-grep Server is a local daemon shared by agents and terminal commands.
+It keeps the MCP endpoint available, coordinates active Workspace runtimes,
+supports background index refresh, and can reuse loaded Embedding models across
+requests.
+
+You do not need the Server for every use of zg. Exact managed ripgrep and
+indexed operations can also run directly in the current process.
+
+## Choose an execution mode
+
+Indexed CLI commands accept `--mode auto|server|direct`.
+
+| Mode | Use it when | Behavior |
+| --- | --- | --- |
+| `auto` | Almost all terminal use | Use a ready Server; otherwise run Direct before submitting the operation |
+| `server` | A script requires the daemon, shared state, or background refresh | Require the Server and fail if it is unavailable |
+| `direct` | One-off use, CI, foreground debugging, or no daemon is desired | Run entirely in the current process |
+
+`auto` is the default and the recommended mode for people. It does not start a
+missing Server; it simply chooses the Server when one is already ready.
+
+Use Server mode when:
+
+- an Agent connects through MCP;
+- indexed searches happen repeatedly across sessions;
+- background refresh should keep an active Workspace current;
+- multiple requests can benefit from a shared loaded model;
+- operational tooling needs an explicit ready/not-ready contract.
+
+Use Direct mode when:
+
+- a command is an isolated one-off operation;
+- a CI job should not leave a daemon running;
+- you want foreground failures and resource lifetime tied to one process.
+
+Managed `zg query --rg` does not need an index or loaded Embedding model. It
+runs locally regardless of whether a Server is available.
+
+## Agent setup
+
+`zg install` configures the selected Agent and starts the Server when possible:
+
+```bash
+zg install
+```
+
+Most Agent users therefore never need to run `zg server on` manually. Restart
+the Agent or open a new session after installation so it discovers the MCP
+endpoint.
+
+Set `ZVEC_GREP_INSTALL_SKIP_SERVER=1` only when another process manager will
+start the Server separately.
+
+See [Agent integrations](./01-agents.md) for managed configuration and
+[MCP](./03-mcp.md) for the exposed tools.
+
+## Server lifecycle
+
+Start the background daemon:
+
+```bash
+zg server on
+```
+
+Inspect process readiness, endpoint, PID, and MCP toolset:
+
+```bash
+zg server status
+zg server status --check-ready
+```
+
+`--check-ready` preserves normal output and exits non-zero unless the Server is
+ready, making it suitable for scripts and health checks.
+
+Stop the daemon gracefully:
+
+```bash
+zg server off
+```
+
+Run it in the foreground for logs or process supervision:
+
+```bash
+zg server run
+```
+
+Only one Server instance can own a given zvec-grep home. If a running Server
+uses the wrong MCP toolset, stop it before restarting with the new profile:
+
+```bash
+zg server off
+zg server on --mcp-toolset full
+```
+
+## Configure the mode
+
+Choose a mode for one command:
+
+```bash
+zg query --mode direct "root-local index discovery"
+zg status --mode server --check-ready
+```
+
+Set an environment default:
+
+```bash
+export ZVEC_GREP_MODE=auto
+```
+
+Or set `client.mode` in `~/.zvec-grep/config.json`:
+
+```json
+{
+  "version": 1,
+  "client": {
+    "mode": "auto"
+  }
+}
+```
+
+An explicit `--mode` wins over the environment and global configuration.
+
+## Refresh behavior
+
+The execution mode changes the default indexed-search refresh policy:
+
+| Policy | Server | Direct |
+| --- | --- | --- |
+| Default | `background` | `off` |
+| `--refresh background` | Return current results and schedule an update | Warn and behave as `off` |
+| `--refresh wait` | Wait for a fresh index | Update and wait in the current process |
+| `--refresh off` | Search without updating | Search without updating |
+
+Server searches may return useful results with `freshness: possibly_stale`
+while an update continues. Use the result immediately when it answers the
+question; use `--refresh wait` only when the latest file state is required.
+
+## Endpoint and toolset
+
+The default endpoint is:
+
+```text
+http://127.0.0.1:7999/mcp
+```
+
+The Server only accepts loopback listen addresses. Change the loopback address
+or port with:
+
+```bash
+zg server on --listen 127.0.0.1:8999
+```
+
+Set `ZVEC_GREP_SERVER_URL` when a client should use a non-default configured
+endpoint.
+
+The default `agent` MCP toolset exposes only `zvec_grep_search`. Use
+`--mcp-toolset full` or `ZVEC_GREP_MCP_TOOLSET=full` to expose optional managed
+rg together with the index and status tools. See [MCP](./03-mcp.md) for the tool
+contract.
+
+## Bearer authentication
+
+Authentication is disabled by default because the Server is loopback-only. To
+require a token, provide at least 32 characters through the environment or a
+file:
+
+```bash
+export ZVEC_GREP_SERVER_TOKEN="replace-with-a-long-random-token"
+zg server on
+```
+
+```bash
+zg server on --token-file /secure/path/zvec-grep.token
+```
+
+Clients can use `ZVEC_GREP_SERVER_TOKEN` or `ZVEC_GREP_SERVER_TOKEN_FILE`.
+Supported Agent integrations can reference an environment-backed token:
+
+```bash
+zg install \
+  --target codex \
+  --mcp-token-env ZVEC_GREP_SERVER_TOKEN \
+  --yes
+```
+
+The MCP Bearer token protects the local Server. It does not configure an
+Embedding provider or authorize remote data transfer.
+
+## Logs and state
+
+The default daemon directory is `~/.zvec-grep/daemon/`. Its structured JSON
+Lines log is written to:
+
+```text
+~/.zvec-grep/daemon/logs/server.log
+```
+
+Credential, authorization, token, API-key, and query fields are filtered from
+daemon log records. Repository identities are logged opaquely rather than as
+raw paths where identity is sufficient.
+
+Use `ZVEC_GREP_HOME` to relocate Server state. Check `zg server status` before
+reading logs; routine searches do not need a status preflight.

package/docs/07-embedding.md

@@ -0,0 +1,179 @@
+# Embedding models
+
+[Documentation](./README.md) · [Agents](./01-agents.md) ·
+[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
+[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
+[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
+
+The Embedding model determines the vector representation used by indexed
+search. It affects language coverage, memory use, index size, input length, and
+indexing speed. A new index needs an explicit model, an environment default, or
+a configured default:
+
+```bash
+zg index --embedding local/potion-code-16m-v2
+```
+
+Local models keep workspace content and query text on the machine. Their files
+are downloaded on first use and cached under `~/.zvec-grep/models` by default.
+Remote models avoid local inference but send disclosed query or workspace
+content to the configured provider after authorization.
+
+## Quick selection
+
+| Need | Start with | Why |
+| --- | --- | --- |
+| A fast first index for a code repository | `local/potion-code-16m-v2` | Small static Model2Vec model with an 8,192-token input limit |
+| A Transformer model specialized for code | `local/jina-embeddings-v2-base-code` | Code-oriented, multilingual, and long-context |
+| General multilingual code and documents | `local/embeddinggemma-300m` | Broad language coverage in a local GGUF model |
+| A smaller multilingual model | `local/multilingual-e5-small` | Compact 384-dimensional Transformer model |
+| A lightweight English model | `local/all-minilm-l6-v2` | Small local model for short English text |
+| Long English documents | `local/gte-modernbert-base` or `local/nomic-embed-text-v1.5` | 8,192-token local context |
+| No local model runtime | `qwen/qwen3.7-text-embedding` | Managed text Embedding API |
+| Text and image retrieval | `qwen/qwen3-vl-embedding` | Managed multimodal Embedding API |
+
+The best model still depends on the repository and its real queries. Start with
+the smallest model that covers the required languages and input length, then
+compare representative results before committing to a larger model.
+
+## Supported models
+
+The input limit applies to each extracted entity or fragment, not the entire
+file.
+
+| Model | Runtime | Max input tokens | Dimensions |
+| --- | --- | ---: | ---: |
+| `local/potion-code-16m-v2` | Model2Vec FP16 | 8,192 | 256 |
+| `local/potion-base-8m` | Model2Vec FP32 | 512 | 256 |
+| `local/all-minilm-l6-v2` | ONNX Q4 | 256 | 384 |
+| `local/bge-small-en-v1.5` | ONNX Q4 | 512 | 384 |
+| `local/multilingual-e5-small` | ONNX Q8 | 512 | 384 |
+| `local/jina-embeddings-v2-base-code` | ONNX Q8 | 8,192 | 768 |
+| `local/gte-modernbert-base` | ONNX Q4 | 8,192 | 768 |
+| `local/nomic-embed-text-v1.5` | ONNX Q4 | 8,192 | 768 |
+| `local/embeddinggemma-300m` | GGUF Q8_0 | 2,048 | 768 |
+| `local/qwen3-embedding-0.6b` | GGUF Q8_0 | 8,192 | 1,024 |
+| `qwen/text-embedding-v4` | Remote text | 8,192 | 1,024 |
+| `qwen/qwen3.7-text-embedding` | Remote text | 128,000 | 1,024 |
+| `qwen/qwen3-vl-embedding` | Remote multimodal | 32,000 | 2,560 |
+
+All catalog entries currently use cosine similarity. Exact model revisions are
+pinned by zvec-grep so the same reference resolves consistently for a given
+release.
+
+## Configure a default model
+
+Set a default for new indexes:
+
+```bash
+zg config model set local/potion-code-16m-v2 --default
+zg index
+```
+
+`ZVEC_GREP_EMBEDDING` provides a process-level default that takes priority over
+the configured global default for new indexes:
+
+```bash
+export ZVEC_GREP_EMBEDDING=local/potion-code-16m-v2
+zg index
+```
+
+An existing index always reuses its stored provider, model, dimensions, and
+metric unless `--embedding` and `--rebuild` explicitly change them. `zg index`
+forwards the current CLI environment default in server and auto modes; direct
+MCP calls use the environment inherited by the daemon.
+
+## Local runtime and device
+
+Select a device for local Transformer and GGUF models:
+
+```bash
+zg index \
+  --embedding local/jina-embeddings-v2-base-code \
+  --device auto
+```
+
+Supported values are `auto`, `cpu`, `metal`, `vulkan`, and `cuda`. Save a model
+preference globally with:
+
+```bash
+zg config model set local/jina-embeddings-v2-base-code --device metal
+```
+
+The equivalent environment override is `ZVEC_GREP_DEVICE`. Model2Vec models
+such as Potion use static vector lookup, so selecting a GPU does not improve
+their runtime.
+
+Override the download cache with `--model-cache` or `ZVEC_GREP_MODEL_CACHE`:
+
+```bash
+zg index \
+  --embedding local/potion-code-16m-v2 \
+  --model-cache /path/to/model-cache
+```
+
+## Remote Embedding and authorization
+
+Configure the Qwen provider credential and, optionally, a model endpoint:
+
+```bash
+zg config provider set qwen --api-key "$DASHSCOPE_API_KEY"
+zg config model set qwen/text-embedding-v4 --default
+```
+
+One-off values can be passed directly or through `ZVEC_GREP_API_KEY` and
+`ZVEC_GREP_ENDPOINT`:
+
+```bash
+zg index \
+  --embedding qwen/text-embedding-v4 \
+  --api-key "$DASHSCOPE_API_KEY" \
+  --allow-remote
+```
+
+Credentials configure access to a provider; they do not authorize data
+transfer. `--allow-remote` authorizes Remote Embedding only for the current
+command. To create a signed Workspace grant shared by the CLI and MCP server:
+
+```bash
+zg auth grant \
+  --capability embedding \
+  --scope workspace \
+  --embedding qwen/text-embedding-v4
+
+zg auth status
+zg auth revoke
+```
+
+Before granting access, confirm that the workspace content is permitted to be
+sent to the selected provider and endpoint. MCP tool approval is separate from
+this data authorization.
+
+## Input length and truncation
+
+zvec-grep uses the selected model's input limit when it creates fragments, but
+characters and tokens do not map one-to-one across languages. If an extracted
+input still exceeds the model limit, the provider truncates it and the index
+records the count.
+
+Inspect the index after changing models or file scope:
+
+```bash
+zg status
+```
+
+Prefer a model with a larger input limit or narrow the indexed content when
+`truncated_fragments` is unexpectedly high.
+
+## Change models
+
+Vector spaces from different models are incompatible, even when their
+dimensions match. Rebuild explicitly when changing models:
+
+```bash
+zg index --rebuild --embedding local/jina-embeddings-v2-base-code
+```
+
+Changing a remote endpoint also requires a rebuild because the endpoint is part
+of the stored index schema. API key and local device changes affect runtime and
+do not require rebuilding the vectors.