@orquentra/codebuddy 2.14.0 → 2.15.1

package/ARCHITECTURE.md

@@ -113,6 +113,104 @@ Rationale:
 
 - better to return slightly stale ranking than block on Hugging Face cold starts or rate limits
 
+## Local LLM and Intent Cards (2.15)
+
+**Runtime: node-llama-cpp.** The local small model (Qwen3-0.6B Q4_K_M by
+default; Qwen3-1.7B as `better`; Qwen only) runs through
+node-llama-cpp 3.22 (llama.cpp bindings). It was compared with Cactus on the
+four criteria in the 2.15 brief:
+
+| | node-llama-cpp 3.22.3 | Cactus |
+|---|---|---|
+| Install size | 41 MB package + one prebuilt binary (19 MB mac-arm64-metal, 33–34 MB x64) | npm ships `cactus-react-native` (152 MB, mobile) and `cactus-needle` (tool-calling model only) |
+| Cold start (M-series, Qwen3-0.6B) | 441 ms load, measured | no Node server binding to measure |
+| Tokens/s (M-series, Metal) | 100–200 tok/s measured; ~2 s per card incl. JSON grammar | n/a |
+| Prebuilt targets | macOS arm64/x64, Linux x64/arm64, Windows x64 (+CUDA/Vulkan) | iOS/Android focus |
+
+Cactus has no maintained Node.js binding for a desktop/server process, and its
+Node package hosts only the Needle tool-calling model, which the brief excludes
+(the job is plain English → code vocabulary, not tool selection). So
+node-llama-cpp. It is not a package dependency: `codebuddy` (or
+`codebuddy models add local-llm`) installs it with `npm -g` and downloads the pinned GGUF into
+`~/.cache/codebuddy/models/gguf/`, checking size and sha256 before moving it
+into place. Forcing `gpu: false` on macOS makes node-llama-cpp compile llama.cpp
+from source (34 s first load), so the backend uses `gpu: "auto"` (Metal on Apple
+Silicon, CPU elsewhere) with `threads` capped at cores − 1
+(`CODEBUDDY_LLM_THREADS` overrides).
+
+**Where it runs.** Generation happens on llama.cpp's native threads inside
+`codebuddy serve` (background project sync) or `codebuddy index --cards`, never
+on a request path except the bounded query expansion below. A single context
+sequence serialises calls. Hooks and MCP tool calls touch
+`.codebuddy/cache/foreground`; the card and fact passes wait (up to 15 s) while
+it is fresher than 2 s. Output is constrained by a JSON-schema grammar with
+`maxItems`/`maxLength` bounds (without them the 0.6B model writes paragraphs
+until the token cap cuts the JSON), then validated with zod; invalid output is
+retried once, then the file is skipped.
+
+**Lifetime.** The model is loaded on demand and unloaded `IDLE_UNLOAD_MS`
+(60 s) after its last generation; measured on Qwen3-0.6B, resident memory went
+from 1,274 MB to 452 MB (the rest is the llama.cpp/Metal runtime, freed when
+the process exits). `serve` frees it on shutdown, and the detached
+`index --cards` that setup starts exits when its pass ends. Setup skips the
+local model on machines under 4 GB RAM or with one core (`machineCanCard`) and
+uses cloud Qwen there when a key is set.
+
+**Model size.** The same four-file card test with the grammar bounds:
+SmolLM2-135M Q4_K_M (105 MB) wrote `...` summaries and raw code lines as
+symptoms; Gemma 3 270M Q4_K_M (253 MB) wrote one-word summaries and term lists
+as symptoms; Qwen3-0.6B Q4_K_M (397 MB) wrote usable summaries and symptoms.
+So CodeBuddy ships Qwen only. A machine below `machineCanCard` gets Qwen from
+the cloud instead (BYOK: `qwen/qwen3.8-27b` on Groq or OpenRouter, sent with
+`reasoning_effort: "none"`; `<think>` blocks are stripped), configured by
+setup only when a provider key is already in the environment.
+
+**Cards** extend the existing `llm-summaries.json` store (prompt version 2), so
+the BYOK and local backends fill the same cache. Each card is used three ways:
+a card vector per file in `file-vectors.json` (granite, next to the chunk
+vectors), one Markdown file per source file in `.codebuddy/cache/cards/`, which
+is its own zvec-grep workspace (English model `local/bge-small-en-v1.5`,
+`--no-ignore`, so the project's ignore rules never hide it), and card terms and
+symptoms in the content index at half weight.
+
+**Agreement rule** (`context/discovery.ts`). 2.14 dropped every vector hit
+outside the keyword set unless it shared task terms with the source and cleared
+`zvec.minScore`. That deleted exactly the hits a reworded task needs. In 2.15 a
+candidate is admitted when at least two signals put it in their top k (default
+5): zvec-grep (code and card workspaces fused), the source-chunk vector, the
+card vector, and the old term-overlap rule as one more vote. Every ranking has
+a top k even for an unrelated task, so a vector vote also needs its cosine at
+or above a floor (`chunkFloor` 0.80, `cardFloor` 0.75). zg prints no
+similarity, so its vote is rank-only, and two rank-only votes never agree.
+Abstention returns nothing only when no candidate passes the rule and keyword
+evidence is below the floor. Card text alone never admits a file; card
+evidence counts when another signal agrees. The floors are uncalibrated
+defaults. Tune them in `.codebuddy/config.json` under `discovery.agreement`
+against the unrelated-task files (the 6 in the Litestar harness plus
+`evals/unrelated-2.15.json`).
+
+**Confidence.** Discovery returns `high` (anchor, exact identifier, CLI
+command, or ≥ 3 votes counting a strong keyword match as two), `medium` (two),
+or `low` (dense signals present but no returned file has two). `low` returns no
+paths: the hook emits nothing, and `context_pack` adds a note telling the agent
+to search normally. Without any dense signal the result is 2.14's keyword path
+and is never suppressed.
+
+**Expansion.** When a task found nothing confident and names no identifier,
+discovery asks the warm local model for 5–8 code terms (≤ 800 ms in
+`context_pack`, ≤ 300 ms in the hook) and ranks once more. Answers are cached
+per normalised task in `.codebuddy/cache/expansions.json` (LRU, 200). The hook
+process never has a warm model, so it only benefits from cached expansions.
+
+**Index facts** (`llm/index-facts.ts`) are Markdown facts with
+`createdByAgent: codebuddy-index`, `paths: [source]`,
+`verification: ["source:Lline"]` and an `index:` idempotency key.
+Manifest and CI facts are read deterministically. Prose facts come from the
+model and must share words with the cited line and read as a statement.
+Sensitive candidates are dropped. Re-extraction replaces only index facts
+whose source hash changed. The cap is 40 per repository, and near-duplicates
+(Jaccard ≥ 0.8) of any existing fact are skipped.
+
 ## Sharing Model
 
 Shares are references by default.

package/README.md

@@ -1,11 +1,18 @@
-# CodeBuddy
+# CodeBuddy by Orquentra
 
 Local-first project context and memory for AI coding agents.
 
+Published by Orquentra as
+[`@orquentra/codebuddy` on npm](https://www.npmjs.com/package/@orquentra/codebuddy).
+
 CodeBuddy helps agents find relevant code, understand project constraints, and
 remember decisions across sessions. It works through MCP with Claude Code,
 Codex, OpenCode, Hermes, and other compatible clients.
 
+[npm package](https://www.npmjs.com/package/@orquentra/codebuddy) ·
+[Documentation](https://github.com/Orquentra-Inc/CodeBuddy/blob/main/docs/guide.md) ·
+[Source and issues](https://github.com/Orquentra-Inc/CodeBuddy)
+
 ## Install
 
 Requires Node.js 20 or later.
@@ -16,9 +23,12 @@ cd your-project
 codebuddy
 ```
 
-The setup wizard configures the project and agent integration. Project context
-works without a database; semantic memory recall optionally uses PostgreSQL
-with pgvector and a local Hugging Face embedding model (no token needed).
+That one command sets everything up: PostgreSQL (the bundled Docker database
+if none is running), the local embedding model (~90 MB), zvec-grep (installed
+and indexed), the local Qwen model (~400 MB; cloud Qwen with a key you enter on
+machines under 4 GB RAM), agent wiring for Claude and Codex, and a background
+pass that builds file vectors, intent cards and project facts. Skip pieces with
+`--no-local-llm`, `--no-graphify` or `--no-docker`.
 
 Run setup inside the project you want CodeBuddy to read. Claude Desktop
 registration saves that folder's absolute path, so restarting the desktop app
@@ -28,13 +38,27 @@ Older desktop registrations without a project path can be repaired by running
 `codebuddy claude install` from the initialized project folder, then restarting
 Claude Desktop.
 
+### Verify the package
+
+The npm package name is **`@orquentra/codebuddy`**; the installed command is
+**`codebuddy`**. Open the [npm package page](https://www.npmjs.com/package/@orquentra/codebuddy)
+directly to find its published versions, or query the public registry:
+
+```bash
+npm view @orquentra/codebuddy version --registry=https://registry.npmjs.org/
+```
+
+Search engine results and AI summaries may lag behind npm releases. If an
+installation fails, include the exact npm error and requested version in a
+[GitHub issue](https://github.com/Orquentra-Inc/CodeBuddy/issues).
+
 ## What it does
 
 - **Task context:** relevant files, symbols, code excerpts, nearby dependencies,
   and covering tests within a token budget.
 - **Project memory:** decisions, rules, and handoffs stored as readable Markdown,
   recalled for relevant tasks by hybrid lexical + vector search
-  ([how it works](docs/semantic-memory.md)).
+  ([how it works](https://github.com/Orquentra-Inc/CodeBuddy/blob/main/docs/semantic-memory.md)).
 - **Continuity:** a local journal of each session's goal, edits, checks and
   findings, re-injected after compaction and offered to the next session when
   a task was left unfinished. Native bridges cover Claude Code, Codex, OpenCode
@@ -50,6 +74,63 @@ Claude Desktop.
 - **Optional retrieval:** zvec semantic search, Graphify architecture maps, and
   a bring-your-own LLM layer.
 
+## Optional local LLM (2.15)
+
+A small local model can write a plain-English **intent card** for every source
+file, so a task worded nothing like the code (for example "the browser forgets
+how long to wait before reconnecting") still finds the right file. `codebuddy`
+/ `codebuddy` installs it as part of setup (Qwen3-0.6B Q4_K_M, ~400 MB,
+checksum-verified; skip with `--no-local-llm` or `CODEBUDDY_LOCAL_LLM=0`), and it
+runs offline after that. To manage it by hand:
+
+```bash
+codebuddy models add local-llm            # Qwen3-0.6B Q4_K_M, ~400 MB, checksum-verified
+codebuddy models add local-llm --size better   # Qwen3-1.7B, ~1.1 GB
+codebuddy models remove local-llm         # back to keyword + vector discovery
+```
+
+Setup also installs the runtime (`node-llama-cpp`, prebuilt for macOS, Linux
+and Windows; `models add` asks first). CodeBuddy uses Qwen only. On a machine
+under 4 GB of RAM or with one CPU core, setup uses Qwen in the cloud instead
+(`qwen/qwen3.8-27b` on Groq, free-tier key; or OpenRouter). It uses
+`GROQ_API_KEY` / `OPENROUTER_API_KEY` from the environment, or asks for a Groq
+key (free at console.groq.com/keys) and saves it in `~/.codebuddy/global.json`
+(mode 0600, never in the project). Bounded, redacted file heads and
+signatures are then sent to that provider ([details](docs/llm-byok.md)). Skip
+the prompt and keyword + vector discovery keep working.
+The model runs inside `codebuddy serve` (not in Docker; Docker only hosts
+PostgreSQL). It is unloaded 60 s after its last card and freed when the agent
+closes `serve`, so it never keeps running in the background.
+
+Qwen3-0.6B is the smallest model that wrote usable cards in our checks:
+SmolLM2-135M (105 MB) returned placeholders and raw code lines, and Gemma 3
+270M returned keyword lists instead of descriptions, so neither ships.
+Nothing leaves your machine, and without the model every feature falls back to
+the 2.14 behaviour.
+
+- **Intent cards:** a summary, 3–6 "symptoms" in bug-report wording, the code
+  terms to search for, and a line per main symbol. Written in the background by
+  `codebuddy serve` (or `codebuddy index --cards`) for new and changed source
+  files only; tests, docs, generated, vendored and secret-bearing files are
+  skipped. Cards feed a second vector per file, a zvec-grep card workspace, and
+  (down-weighted) the keyword index. Progress: `codebuddy models`, `codebuddy
+  storage`.
+- **Agreement rule:** a file the keywords missed is admitted when two
+  independent signals agree (zvec-grep, source-chunk vectors, card vectors,
+  term overlap). When no returned file has two, CodeBuddy says so instead of
+  guessing: `context_pack` returns `confidence: "low"` with a note to search
+  normally, and the prompt hook stays silent.
+- **Index facts:** up to 40 project facts drawn from `package.json`,
+  `pyproject.toml`, CI workflows, README / CONTRIBUTING / AGENTS.md /
+  CLAUDE.md and deprecation or release policies, each with its source line.
+  They are ordinary Markdown facts; `codebuddy memory list --origin index`
+  shows them and `codebuddy memory prune --origin index` removes them. Facts a
+  person or agent wrote are never replaced.
+
+`codebuddy` also builds the zvec-grep index and enables it when `zg` 0.2.x
+is installed, and `codebuddy serve` keeps a `zg server` running so the prompt
+hook gets vector candidates within its deadline.
+
 ## What it costs
 
 Context is not free: tool schemas, the rules block and the injected context are
@@ -98,7 +179,7 @@ needs more detail. Inspect missing or truncated evidence before editing.
 
 ## Documentation
 
-- [Full guide and examples](https://github.com/Orquentra-Inc/CodeBuddy/blob/348bb7c/README.md)
+- [Full guide and examples](https://github.com/Orquentra-Inc/CodeBuddy/blob/main/docs/guide.md)
 - [Telemetry and the admin panel](https://github.com/Orquentra-Inc/CodeBuddy/blob/main/docs/telemetry.md)
 - [Token budgets](https://github.com/Orquentra-Inc/CodeBuddy/blob/main/docs/token-budgets.md)
 - [Optional LLM setup](https://github.com/Orquentra-Inc/CodeBuddy/blob/main/docs/llm-byok.md)