@zvec/zvec-grep 0.2.1 → 0.3.0-rc.1

package/docs/06-server.md

@@ -1,220 +0,0 @@
-# Server and execution modes
-
-[Documentation](./README.md) · [Agents](./01-agents.md) ·
-[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
-[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
-[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
-
-The zvec-grep Server is a local daemon shared by agents and terminal commands.
-It keeps the MCP endpoint available, coordinates active Workspace runtimes,
-supports background index refresh, and can reuse loaded Embedding models across
-requests.
-
-You do not need the Server for every use of zg. Exact managed ripgrep and
-indexed operations can also run directly in the current process.
-
-## Choose an execution mode
-
-Indexed CLI commands accept `--mode auto|server|direct`.
-
-| Mode | Use it when | Behavior |
-| --- | --- | --- |
-| `auto` | Almost all terminal use | Use a ready Server; otherwise run Direct before submitting the operation |
-| `server` | A script requires the daemon, shared state, or background refresh | Require the Server and fail if it is unavailable |
-| `direct` | One-off use, CI, foreground debugging, or no daemon is desired | Run entirely in the current process |
-
-`auto` is the default and the recommended mode for people. It does not start a
-missing Server; it simply chooses the Server when one is already ready.
-
-Use Server mode when:
-
-- an Agent connects through MCP;
-- indexed searches happen repeatedly across sessions;
-- background refresh should keep an active Workspace current;
-- multiple requests can benefit from a shared loaded model;
-- operational tooling needs an explicit ready/not-ready contract.
-
-Use Direct mode when:
-
-- a command is an isolated one-off operation;
-- a CI job should not leave a daemon running;
-- you want foreground failures and resource lifetime tied to one process.
-
-Managed `zg query --rg` does not need an index or loaded Embedding model. It
-runs locally regardless of whether a Server is available.
-
-## Agent setup
-
-`zg install` configures the selected Agent and starts the Server when possible:
-
-```bash
-zg install
-```
-
-Most Agent users therefore never need to run `zg server on` manually. Restart
-the Agent or open a new session after installation so it discovers the MCP
-endpoint.
-
-Set `ZVEC_GREP_INSTALL_SKIP_SERVER=1` only when another process manager will
-start the Server separately.
-
-See [Agent integrations](./01-agents.md) for managed configuration and
-[MCP](./03-mcp.md) for the exposed tools.
-
-## Server lifecycle
-
-Start the background daemon:
-
-```bash
-zg server on
-```
-
-Inspect process readiness, endpoint, PID, and MCP toolset:
-
-```bash
-zg server status
-zg server status --check-ready
-```
-
-`--check-ready` preserves normal output and exits non-zero unless the Server is
-ready, making it suitable for scripts and health checks.
-
-Stop the daemon gracefully:
-
-```bash
-zg server off
-```
-
-Run it in the foreground for logs or process supervision:
-
-```bash
-zg server run
-```
-
-Only one Server instance can own a given zvec-grep home. If a running Server
-uses the wrong MCP toolset, stop it before restarting with the new profile:
-
-```bash
-zg server off
-zg server on --mcp-toolset full
-```
-
-## Configure the mode
-
-Choose a mode for one command:
-
-```bash
-zg query --mode direct "root-local index discovery"
-zg status --mode server --check-ready
-```
-
-Set an environment default:
-
-```bash
-export ZVEC_GREP_MODE=auto
-```
-
-Or set `client.mode` in `~/.zvec-grep/config.json`:
-
-```json
-{
-  "version": 1,
-  "client": {
-    "mode": "auto"
-  }
-}
-```
-
-An explicit `--mode` wins over the environment and global configuration.
-
-## Refresh behavior
-
-The execution mode changes the default indexed-search refresh policy:
-
-| Policy | Server | Direct |
-| --- | --- | --- |
-| Default | `background` | `off` |
-| `--refresh background` | Return current results and schedule an update | Warn and behave as `off` |
-| `--refresh wait` | Wait for a fresh index | Update and wait in the current process |
-| `--refresh off` | Search without updating | Search without updating |
-
-Server searches return `freshness: possibly_stale` only with evidence of index
-drift. The first refresh after activation may use this conservative status; an
-hourly reconciliation remains `fresh` until its probe finds a mismatch. Use
-`--refresh wait` only when the latest file state is required.
-
-Watcher-reported path updates skip workspace-wide status scans. Because
-file-system watchers can silently miss events, the Server schedules an hourly
-full reconciliation probe; the next search uses it to scan the Workspace and
-repair index drift.
-
-Large bursts of exact watcher events are compacted into directory-scoped
-updates. A full reconciliation is reserved for watcher errors, missing event
-paths, resume drift, and other cases where events may have been lost.
-
-## Endpoint and toolset
-
-The default endpoint is:
-
-```text
-http://127.0.0.1:7999/mcp
-```
-
-The Server only accepts loopback listen addresses. Change the loopback address
-or port with:
-
-```bash
-zg server on --listen 127.0.0.1:8999
-```
-
-Set `ZVEC_GREP_SERVER_URL` when a client should use a non-default configured
-endpoint.
-
-The default `agent` MCP toolset exposes only `zvec_grep_search`. Use
-`--mcp-toolset full` or `ZVEC_GREP_MCP_TOOLSET=full` to expose optional managed
-rg together with the index and status tools. See [MCP](./03-mcp.md) for the tool
-contract.
-
-## Bearer authentication
-
-Authentication is disabled by default because the Server is loopback-only. To
-require a token, provide at least 32 characters through the environment or a
-file:
-
-```bash
-export ZVEC_GREP_SERVER_TOKEN="replace-with-a-long-random-token"
-zg server on
-```
-
-```bash
-zg server on --token-file /secure/path/zvec-grep.token
-```
-
-Clients can use `ZVEC_GREP_SERVER_TOKEN` or `ZVEC_GREP_SERVER_TOKEN_FILE`.
-Supported Agent integrations can reference an environment-backed token:
-
-```bash
-zg install \
-  --target codex \
-  --mcp-token-env ZVEC_GREP_SERVER_TOKEN \
-  --yes
-```
-
-The MCP Bearer token protects the local Server. It does not configure an
-Embedding provider or authorize remote data transfer.
-
-## Logs and state
-
-The default daemon directory is `~/.zvec-grep/daemon/`. Its structured JSON
-Lines log is written to:
-
-```text
-~/.zvec-grep/daemon/logs/server.log
-```
-
-Credential, authorization, token, API-key, and query fields are filtered from
-daemon log records. Repository identities are logged opaquely rather than as
-raw paths where identity is sufficient.
-
-Use `ZVEC_GREP_HOME` to relocate Server state. Check `zg server status` before
-reading logs; routine searches do not need a status preflight.

package/docs/07-embedding.md

@@ -1,203 +0,0 @@
-# Embedding models
-
-[Documentation](./README.md) · [Agents](./01-agents.md) ·
-[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
-[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
-[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
-
-The Embedding model determines the vector representation used by indexed
-search. It affects language coverage, memory use, index size, input length, and
-indexing speed. A new index needs an explicit model, an environment default, or
-a configured default:
-
-```bash
-zg index --embedding local/potion-code-16m-v2
-```
-
-Local models keep workspace content and query text on the machine. Their files
-are downloaded on first use and cached under `~/.zvec-grep/models` by default.
-Remote models avoid local inference but send disclosed query or workspace
-content to the configured provider after authorization.
-
-## Quick selection
-
-| Need | Start with | Why |
-| --- | --- | --- |
-| A fast first index for a code repository | `local/potion-code-16m-v2` | Small static Model2Vec model with a 1,024-token input limit |
-| Fast English document retrieval | `local/potion-retrieval-32m` | Retrieval-tuned static model with 512-dimensional vectors |
-| Fast multilingual document retrieval | `local/potion-multilingual-128m` | Static model trained for 101 languages with compact 256-dimensional vectors |
-| A Transformer model specialized for code | `local/jina-embeddings-v2-base-code` | Code-oriented, multilingual, and long-context |
-| General multilingual code and documents | `local/embeddinggemma-300m` | Broad language coverage in a local GGUF model |
-| A smaller multilingual model | `local/multilingual-e5-small` | Compact 384-dimensional Transformer model |
-| A lightweight English model | `local/all-minilm-l6-v2` | Small local model for short English text |
-| Long English documents | `local/gte-modernbert-base` or `local/nomic-embed-text-v1.5` | 8,192-token local context |
-| No local model runtime | `qwen/qwen3.7-text-embedding` | Managed text Embedding API |
-| Text and image retrieval | `qwen/qwen3-vl-embedding` | Managed multimodal Embedding API |
-
-The best model still depends on the repository and its real queries. Start with
-the smallest model that covers the required languages and input length, then
-compare representative results before committing to a larger model.
-
-## Supported models
-
-The input limit applies to each extracted entity or fragment, not the entire
-file.
-
-| Model | Runtime | Max input tokens | Dimensions |
-| --- | --- | ---: | ---: |
-| `local/potion-code-16m-v2` | Model2Vec FP16 | 1,024 | 256 |
-| `local/potion-retrieval-32m` | Model2Vec FP32 | 1,024 | 512 |
-| `local/potion-multilingual-128m` | Model2Vec FP32 | 1,024 | 256 |
-| `local/all-minilm-l6-v2` | ONNX Q4 | 256 | 384 |
-| `local/bge-small-en-v1.5` | ONNX Q4 | 512 | 384 |
-| `local/multilingual-e5-small` | ONNX Q8 | 512 | 384 |
-| `local/jina-embeddings-v2-base-code` | ONNX Q8 | 8,192 | 768 |
-| `local/gte-modernbert-base` | ONNX Q4 | 8,192 | 768 |
-| `local/nomic-embed-text-v1.5` | ONNX Q4 | 8,192 | 768 |
-| `local/embeddinggemma-300m` | GGUF Q8_0 | 2,048 | 768 |
-| `local/qwen3-embedding-0.6b` | GGUF Q8_0 | 8,192 | 1,024 |
-| `qwen/text-embedding-v4` | Remote text | 8,192 | 1,024 |
-| `qwen/qwen3.7-text-embedding` | Remote text | 128,000 | 1,024 |
-| `qwen/qwen3-vl-embedding` | Remote multimodal | 32,000 | 2,560 |
-
-All catalog entries currently use cosine similarity. Exact model revisions are
-pinned by zvec-grep so the same reference resolves consistently for a given
-release.
-
-## Configure a default model
-
-Set a default for new indexes:
-
-```bash
-zg config model set local/potion-code-16m-v2 --default
-zg index
-```
-
-`ZVEC_GREP_EMBEDDING` provides a process-level default that takes priority over
-the configured global default for new indexes:
-
-```bash
-export ZVEC_GREP_EMBEDDING=local/potion-code-16m-v2
-zg index
-```
-
-An existing index always reuses its stored provider, model, dimensions, and
-metric unless `--embedding` and `--rebuild` explicitly change them. `zg index`
-forwards the current CLI environment default in server and auto modes; direct
-MCP calls use the environment inherited by the daemon.
-
-## Local runtime and device
-
-Select a device for local Transformer and GGUF models:
-
-```bash
-zg index \
-  --embedding local/jina-embeddings-v2-base-code \
-  --device auto
-```
-
-Supported values are `auto`, `cpu`, `metal`, `vulkan`, and `cuda`. Save a model
-preference globally with:
-
-```bash
-zg config model set local/jina-embeddings-v2-base-code --device metal
-```
-
-The equivalent environment override is `ZVEC_GREP_DEVICE`. Model2Vec models
-such as Potion use static vector lookup, so selecting a GPU does not improve
-their runtime.
-
-Override the download cache with `--model-cache` or `ZVEC_GREP_MODEL_CACHE`:
-
-```bash
-zg index \
-  --embedding local/potion-code-16m-v2 \
-  --model-cache /path/to/model-cache
-```
-
-## Remote Embedding and authorization
-
-Configure the Qwen provider credential and, optionally, a model endpoint:
-
-```bash
-zg config provider set qwen --api-key "$DASHSCOPE_API_KEY"
-zg config model set qwen/text-embedding-v4 --default
-```
-
-One-off values can be passed directly or through `ZVEC_GREP_API_KEY` and
-`ZVEC_GREP_ENDPOINT`:
-
-```bash
-zg index \
-  --embedding qwen/text-embedding-v4 \
-  --api-key "$DASHSCOPE_API_KEY" \
-  --allow-remote
-```
-
-Credentials configure access to a provider; they do not authorize data
-transfer. `--allow-remote` authorizes Remote Embedding only for the current
-command. To create a signed Workspace grant shared by the CLI and MCP server:
-
-```bash
-zg auth grant \
-  --capability embedding \
-  --scope workspace \
-  --embedding qwen/text-embedding-v4
-
-zg auth status
-zg auth revoke
-```
-
-Before granting access, confirm that the workspace content is permitted to be
-sent to the selected provider and endpoint. MCP tool approval is separate from
-this data authorization.
-
-When Qoder CLI reports that it has no handler for `elicitation/create`, or
-declines or cancels without displaying the form, the `AGENTS.md` guidance
-installed by `zg install --target qoder` uses the exact `AskUserQuestion` tool
-as a compatibility path. It offers workspace approval, local FTS only, or
-cancel.
-
-Qoder IDE uses the exact native tool name `ask_user_question`. Because Qoder IDE
-does not provide a supported global Rules location, the installer does not
-write equivalent IDE guidance. In stdio mode, the zvec-grep bridge instead
-turns a missing `elicitation/create` handler into an actionable error directing
-the top-level IDE agent to present the same choices with `ask_user_question`.
-This path still requires an end-to-end smoke test in a real Qoder IDE session.
-
-Either path runs the Workspace grant only after explicit approval and retries
-the original MCP search once. Choosing local FTS instead removes
-vector-producing query routes, sets `autoUpdate` to `false`, and keeps retrieval
-local without refreshing the remote-embedding index; a headless session never
-grants access automatically. Neither question tool should collect a token, API
-key, or password. `--allow-remote` is not a substitute for this path because it
-authorizes only the current CLI command, not a later MCP retry.
-
-## Input length and truncation
-
-zvec-grep uses the selected model's input limit when it creates fragments, but
-characters and tokens do not map one-to-one across languages. If an extracted
-input still exceeds the model limit, the provider truncates it and the index
-records the count.
-
-Inspect the index after changing models or file scope:
-
-```bash
-zg status
-```
-
-Prefer a model with a larger input limit or narrow the indexed content when
-`truncated_fragments` is unexpectedly high.
-
-## Change models
-
-Vector spaces from different models are incompatible, even when their
-dimensions match. Rebuild explicitly when changing models:
-
-```bash
-zg index --rebuild --embedding local/jina-embeddings-v2-base-code
-```
-
-Changing a remote endpoint also requires a rebuild because the endpoint is part
-of the stored index schema. API key and local device changes affect runtime and
-do not require rebuilding the vectors.

package/docs/08-roadmap.md

@@ -1,85 +0,0 @@
-# Roadmap
-
-[Documentation](./README.md) · [Agents](./01-agents.md) ·
-[CLI](./02-cli.md) · [MCP](./03-mcp.md) · [Pipeline](./04-pipeline.md) ·
-[Architecture](./05-architecture.md) · [Server](./06-server.md) ·
-[Embedding](./07-embedding.md) · [Roadmap](./08-roadmap.md)
-
-zvec-grep is under active development. The destination is one dependable,
-local-first search layer that people and agents can use without choosing among
-separate retrieval tools. This roadmap describes direction, not promised dates;
-priorities may change as we learn from real workloads.
-
-## Now — make the public preview dependable
-
-- Make installation, updates, and removal reliable across current platforms and
-  agent integrations.
-- Harden incremental indexing, freshness, Server recovery, and concurrent
-  workspace access.
-- Establish reproducible search-quality, performance, and agent-context
-  evaluation.
-- Stabilize the CLI, MCP contracts, configuration, diagnostics, and index
-  compatibility policy before a stable release.
-
-## Product direction
-
-The following four directions are part of the product vision, not optional
-experiments. Their sequencing may change, but each is required for zg to become
-the search layer we want it to be.
-
-### 1. Search more data formats, natively
-
-- Make PDF, PowerPoint (`.ppt` and `.pptx`), HTML, images, and other multimodal
-  documents first-class searchable inputs.
-- Add format-aware extraction that preserves useful structure, layout, metadata,
-  and relationships instead of flattening every file into plain text.
-- Combine text extraction, OCR, vision-language understanding, and multimodal
-  Embedding where each format benefits from them.
-- Use format- and content-aware retrieval so zg can choose a smarter strategy
-  for each query and source.
-
-### 2. Strengthen retrieval
-
-- Add knowledge-graph construction and graph retrieval to complement BM25,
-  vector search, and managed ripgrep.
-- Expand multi-route hybrid retrieval across lexical, vector, graph, structural,
-  and metadata signals.
-- Improve query planning so people and agents can express intent without
-  manually choosing retrieval routes.
-- Improve fusion, reranking, evaluation, and explainability while keeping the
-  returned context compact.
-
-### 3. Make zg more out of the box
-
-- Provide a local GUI for search, workspace management, indexing, model setup,
-  permissions, and diagnostics.
-- Support more installation paths beyond npm, including platform-native package
-  managers and installers where appropriate.
-- Make first-run setup, Agent discovery, model selection, updates, and recovery
-  increasingly automatic with useful defaults.
-- Preserve CLI and configuration control for advanced users while removing it
-  from the critical path for everyone else.
-
-### 4. Reach every platform, from PC to mobile
-
-- Deliver a consistent experience across macOS, Windows, and Linux desktops.
-- Extend the local search layer to iOS and Android instead of treating mobile as
-  a remote client to a required cloud service.
-- Adapt indexing, storage, and model execution to the memory, power, and lifecycle
-  constraints of mobile devices.
-- Keep the same local-first trust and permission model across desktop and mobile.
-
-## Guardrails
-
-- Local-first remains the default; remote content transfer always requires
-  explicit authorization.
-- zg should hide retrieval-tool choice from users without hiding useful control.
-- Better recall must not come at the cost of noisy, oversized agent context.
-- A hosted service must never be required for the core local workflow.
-
-## Help shape the roadmap
-
-Priorities should come from real use. Open a
-[GitHub issue](https://github.com/zvec-ai/zvec-grep/issues) to describe a
-workflow, limitation, or result that matters to you. Contributions are welcome;
-see the [Contributing Guide](../CONTRIBUTING.md).

package/docs/README.md

@@ -1,53 +0,0 @@
-# zvec-grep documentation
-
-**zg** is one local-first search layer for people and agents. These guides go
-beyond the short path in the project [README](../README.md) and explain how to
-connect an agent, shape an index, search it, and control the underlying
-interfaces.
-
-> [!IMPORTANT]
-> zvec-grep is a work in progress. Commands and configuration may change before
-> the first stable release.
-
-## Start here
-
-| I want to… | Read |
-| --- | --- |
-| Connect Codex, Claude Code, Qwen Code, Cursor, or OpenCode | [Agent integrations](./01-agents.md) |
-| Use zg directly from a terminal | [CLI guide](./02-cli.md) |
-| Understand the tools exposed to an agent | [MCP guide](./03-mcp.md) |
-| Understand indexing, updates, and search routes | [Retrieval pipeline](./04-pipeline.md) |
-| See how the components and trust boundaries fit together | [Architecture](./05-architecture.md) |
-| Choose between Auto, Server, and Direct execution | [Server and execution modes](./06-server.md) |
-| Choose and configure an Embedding model | [Embedding models](./07-embedding.md) |
-| See what is stable now and what comes next | [Roadmap](./08-roadmap.md) |
-
-## Recommended paths
-
-If you primarily use an agent, start with [Agent integrations](./01-agents.md),
-then read the [MCP guide](./03-mcp.md) and
-[Retrieval pipeline](./04-pipeline.md). The [Server guide](./06-server.md)
-explains the daemon that connects them.
-
-If you primarily use the terminal, start with the [CLI guide](./02-cli.md), then
-read the [Retrieval pipeline](./04-pipeline.md). Use the
-[Embedding guide](./07-embedding.md) when creating a new index. The default
-`auto` execution mode is explained in
-[Server and execution modes](./06-server.md).
-
-For the whole-system mental model and trust boundaries, read
-[Architecture](./05-architecture.md).
-
-The [Roadmap](./08-roadmap.md) tracks the path from work in progress to a stable
-release.
-
-The CLI remains the source of truth for flags in the installed version:
-
-```bash
-zg help
-zg help query
-zg help index
-```
-
-For development setup and pull request conventions, see
-[CONTRIBUTING.md](../CONTRIBUTING.md).