@octocodeai/freellama-mcp-server 0.6.2 → 0.9.0

package/docs/TUNING.md

@@ -0,0 +1,86 @@
+# Make local models faster and lighter
+
+Moved from the root README.
+
+## Make local models faster and lighter
+
+Decode speed is fixed by the model and hardware. A task's real cost is mostly around decoding: cold
+loads, reloads, re-evaluating a prompt the runner already saw, unneeded thinking and output, and
+running out of memory. Best payoff first:
+
+1. **Avoid reloads.** Ollama reuses a loaded runner only when `num_ctx`, `num_batch`, `num_gpu`,
+   `num_thread`, `use_mmap` and the context-shift setting match ([scheduler](https://github.com/ollama/ollama/blob/main/server/sched.go)).
+   Managed chat sends `truncate:false`, and by default `shift:false`, so an overflow is refused
+   instead of silently dropping context; raw clients that keep Ollama's default shift therefore
+   reload a runner FreeLlama loaded, and the reverse. Set `context_shift = "ollama_default"` in the
+   runtime file to omit `shift` so both share one runner (`truncate:false` is still sent; a long
+   generation may then drop its earliest tokens), or give raw and managed traffic separate models
+   or backends when both are busy. `freellama config` shows the active value and its source.
+   FreeLlama sizes `num_ctx` in fixed buckets (2k–32k), reuses a loaded runner's larger context, and
+   leaves thread selection to Ollama unless the operator or caller explicitly overrides it. Keep one model's callers on one
+   `contextTokens`, warm at that context, and compare `models`' `context_default` and
+   `resident_context`.
+2. **Keep prompts prefix-stable.** Ollama's backends own prompt caching; its llama.cpp backend
+   reuses matching prompt prefixes. Cache behavior varies by engine and model. Put the stable part first (system prompt, tool list, documents) and
+   append, never edit. Warm a long prefix once with `ollama_manage{action:"warm",model,systemPrompt}` (or
+   `[preload]` `messages`) and use `scope` for follow-ups, which reuse the whole history. Receipts
+   and the usage ledger report `metrics.cached_prompt_tokens` and `uncached_prompt_tokens` from
+   Ollama's `prompt_eval_cached_count` when reported; missing counters stay unknown. Context changes
+   can reload a runner and lose its prefix cache.
+3. **Spend output only where it helps.** Completion, tools, browser and vision tasks send
+   `think:false` (Ollama otherwise enables thinking on capable models); coding, code-repair and
+   long-context tasks send `think:"medium"` to thinking models. `stopOnRepetition` cut `glm-ocr` on 12
+   report strips from 12,948 to 6,452 output tokens with every label still read.
+4. **Unload instead of swapping.** A load that does not fit waits instead of swapping, idle models are
+   unloaded under memory pressure, and `unload` frees a big model before a switch without ever
+   loading it.
+5. **Shrink memory per runner.** Quantize the KV cache, keep `OLLAMA_NUM_PARALLEL=1` on a laptop
+   (every slot multiplies KV memory by the context), and bound llama-server's hidden prompt cache
+   with `LLAMA_ARG_CACHE_RAM`; see [Tune Ollama](#tune-ollama-for-local-models).
+6. **Keep the agent's context small.** See [small agent context](FEATURES.md#small-agent-context).
+
+Measured here (Apple M2 Pro, 32 GB, Ollama 0.40.13, FreeLlama 0.8.0, 2026-10-31, light load only):
+`serve` prepared in 69–143 ms; a cold `nomic-embed-text:v1.5` load took 0.5 s and a resident embedding
+13–26 ms in Ollama, plus 4 ms (median) for FreeLlama; `unload` took 5–56 ms; an MCP embedding result
+was 2.6 KB inline with the vectors in its `response_file`.
+
+### Tune Ollama for local models
+
+FreeLlama never changes Ollama's settings. These matter most, per Ollama's docs and measured reports:
+
+| Setting | What it does | Suggested start |
+|---|---|---|
+| `OLLAMA_NUM_PARALLEL` | Decoding slots per model, each with its own prompt cache; RAM grows with slots times context ([Ollama FAQ](https://github.com/ollama/ollama/blob/main/docs/faq.mdx)). Small gain on Apple silicon; some engines run one request at a time ([MLX engine](https://github.com/ollama/ollama/issues/17280)) | `1`, raised only after measuring. FreeLlama's budget follows it |
+| `OLLAMA_FLASH_ATTENTION` + `OLLAMA_KV_CACHE_TYPE=q8_0` | A quantized KV cache roughly halves its memory and needs flash attention ([Ollama FAQ](https://github.com/ollama/ollama/blob/main/docs/faq.mdx)); with `OLLAMA_FLASH_ATTENTION` unset, Ollama enables it automatically on supported backends. See [KV cache and flash attention](#kv-cache-and-flash-attention) | On, when the model's quality holds |
+| Context length | Ollama defaults to 4k below 24 GiB of VRAM and recommends at least 64k for agent and coding work ([context length](https://github.com/ollama/ollama/blob/main/docs/context-length.mdx)) | Size per task. FreeLlama keeps Ollama's default when it covers the prompt, otherwise sizes text tasks to the prompt; `contextTokens` overrides |
+| `OLLAMA_MAX_LOADED_MODELS` | How many runners stay resident. Unset and `0` mean 3 per GPU, then Ollama unloads by count | More than the model memory budget can hold, so the budget decides by size and reload cost. `1` reloads on every switch |
+| `keep_alive` | How long a runner stays loaded after use | Leave it out. FreeLlama picks a finite time from measured reuse |
+| `LLAMA_ARG_CACHE_RAM` | Host RAM llama-server may use for displaced prompt caches (`--cache-ram`); default 8192 MiB per runner ([llama.cpp server](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)), counted by no footprint estimate. Set it on the Ollama server, which passes its environment to llama-server; `doctor` checks it | `2048` on 32 GB or less; `0` disables it at the cost of re-evaluating prefixes |
+
+#### KV cache and flash attention
+
+FreeLlama sizes each backend's KV cache from that Ollama process's settings. A model whose metadata
+does not describe its cache (hybrid, recurrent or sliding-window layers) is assumed to need a quarter
+of its file size per 16K tokens of context, for each parallel slot, until its first load is
+measured (`source: model_file_plus_assumed_kv_plus_graph` in the receipt):
+
+| `OLLAMA_FLASH_ATTENTION` | State | KV share of f16 with `q8_0` / `q4_0` |
+|---|---|---|
+| unset | `auto`: Ollama enables flash attention where the backend supports it (Metal, CUDA) | 50% / 25% |
+| `1`, `true` | `on` | 50% / 25% |
+| `0`, `false` | `off`: Ollama ignores `OLLAMA_KV_CACHE_TYPE` | 100% |
+
+`auto` is sized like `on`. A model or backend without flash attention falls back to an f16 cache, so
+its first load can be larger than the estimate; the measured footprint, keyed by
+`OLLAMA_NUM_PARALLEL` and the KV cache share, then sizes later loads. `status` reports the result
+under `ollama.config.effective` (`cpu_config` for a CPU backend) and `config` under
+`ollama.effective`: `flash_attention`, `kv_cache_type` and `kv_cache_percent_of_f16`.
+
+Ollama's own fit check still predicts an f16 cache, and its scheduler evicts other runners or
+offloads layers when that prediction passes 80% of free memory. FreeLlama reserves the quantized
+estimate (or a measurement) and keeps Ollama's f16 figure beside it as
+`memory_reservation.ollama_f16_preflight_bytes`. When the quantized estimate fits within 80% of the
+available memory FreeLlama admitted against but the f16 figure does not, the receipt adds
+`ollama_may_evict_or_offload: true` and `ollama_preflight_reason`. This applies to CPU and
+unified-memory backends; on a discrete GPU Ollama compares against VRAM, which FreeLlama does not
+attribute to a backend.

package/docs/cookbooks/README.md

@@ -0,0 +1,22 @@
+# FreeLlama cookbooks
+
+Short recipes for one job each: when to use it, which model, the exact call, and how to check the result. To see what is installed and fits right now, ask `models {view:"catalog"}` or run `freellama catalog`.
+
+| Job | Recipe |
+|---|---|
+| Classify, triage, yes/no | [classify](classify.md) |
+| Find the code for an issue | [find-code](find-code.md) |
+| Find duplicates | [duplicates](duplicates.md) |
+| Read text from an image | [ocr](ocr.md) |
+| Understand what an image shows | [image-recognition](image-recognition.md) |
+| Find an image by describing it | [image-search](image-search.md) |
+| Recording to text | [speech-to-text](speech-to-text.md) |
+| Text to speech | [text-to-speech](text-to-speech.md) |
+| Small coding tasks | [coding](coding.md) |
+| Research a repository | [research-agent](research-agent.md) |
+| Rewrite or summarize many texts | [batch-text](batch-text.md) |
+| Check a code tree, log or site in small saved chunks | [sweep](sweep.md) |
+| Let FreeLlama pick the model and prompt for a job | [jobs](jobs.md) |
+| Memory, context and caching | [memory-and-caching](memory-and-caching.md) |
+
+Every call here is checked against the tool schemas by a test. Measured results come from one machine: an Apple M2 Pro with 32 GB. Measure on yours before you rely on them.

package/docs/cookbooks/batch-text.md

@@ -0,0 +1,19 @@
+# Rewrite, summarize or extract across many texts
+
+**Use when** you have 10 or more similar short texts. Send what they share once, and name each input by path.
+
+**Skip it when** a pattern or script can do it exactly: checking a format with a regex beats any model.
+
+**Model:** run `models {view:"catalog", job:"coding"}`; the same chat models apply.
+
+```json
+{"tool": "run_task", "arguments": {"task": "completion", "model": "bonsai-2:9b", "systemPrompt": "Summarize this release note in one line.", "options": {"num_predict": 120}, "tasks": [{"id": "v1", "independent": true, "task": {"prompt": "Release note:", "files": ["releases/v1.md"]}}, {"id": "v2", "independent": true, "task": {"prompt": "Release note:", "files": ["releases/v2.md"]}}]}}
+```
+
+Up to 64 items per call. Top-level `task`, `model`, `systemPrompt`, `questions`, `format` and `options` apply to every item; `prompt`, `state`, `input`, `files` and `images` stay per item.
+
+`job:"summarize"` in place of `model` and `systemPrompt` lets FreeLlama pick the model and its summary prompt ([jobs](jobs.md)). On a single call, `saveAs` writes the answer to a handoff file a later task reads with `files`.
+
+**Read the result:** one `## <id>` section per item, in order.
+
+**Check it:** read 3 to 5, more where a mistake costs more.

package/docs/cookbooks/classify.md

@@ -0,0 +1,25 @@
+# Classify, triage and yes/no checks
+
+**Use when** you have many items to label, route or check against a rule. A decision model answers in one forward pass with a probability, and generates no text to parse.
+
+**Skip it when** there is one item you already understand, or a wrong label would be costly and unchecked.
+
+**Model:** `strands-decider:2b-nvfp4` (1.3 GB). See `models {view:"catalog", job:"classify"}` for what is installed and fits now.
+
+One item, with the question asked once:
+
+```json
+{"tool": "run_task", "arguments": {"task": "decision", "model": "strands-decider:2b-nvfp4", "state": "Checkout returns 500 since 9am.", "questions": {"kind": {"type": "choice", "instructions": "What kind of issue is this?", "criteria": {"bug": "Something is broken", "feature": "A request for new capability", "question": "A question"}}, "urgent": {"type": "noul", "instructions": "Does this need action today?"}}}}
+```
+
+Many items: send the question once and let each item carry only its own text or file:
+
+```json
+{"tool": "run_task", "arguments": {"task": "decision", "model": "strands-decider:2b-nvfp4", "questions": {"kind": {"type": "choice", "instructions": "What kind of issue is this?", "criteria": {"bug": "Something is broken", "feature": "A request for new capability"}}}, "tasks": [{"id": "a", "independent": true, "task": {"files": ["issues/a.md"]}}, {"id": "b", "independent": true, "task": {"files": ["issues/b.md"]}}]}}
+```
+
+CLI: `freellama task --task decision --questions @questions.json --input-file issue.md`
+
+**Read the result:** one line per question, such as `kind: bug (p 0.95, confidence 0.79)`.
+
+**Check it:** spot-check 3 to 5 answers, and treat a low probability as unknown. Add an explicit `other` option when no listed label may fit. Do not use this model's `score` answers for severity.

package/docs/cookbooks/coding.md

@@ -0,0 +1,17 @@
+# Offload small coding tasks
+
+**Use when** the task is small and self-contained, and you can check the answer cheaply: explain a function, write a regex, convert a snippet, draft a test.
+
+**Skip it when** the change spans files, needs judgment you must stand behind, or is shorter than describing it. Local code review is unreliable: verify every finding.
+
+**Model:** `lfm3:1.7b-q4_K_M` (1.1 GB, about 3.3 GB available needed). Measured here on 25 small questions over three rounds: lfm3 61 right, `bonsai-2:9b` 55, and Nemotron-3.5-Bolt 68, which needs about 13.5 GB available. Run `models {view:"catalog", job:"coding"}` for what fits now.
+
+Pass the code by path:
+
+```json
+{"tool": "run_task", "arguments": {"task": "coding", "model": "lfm3:1.7b-q4_K_M", "prompt": "Write a unit test for the function in this file. Reply with code only.", "files": ["src/parse.ts"], "options": {"num_predict": 800}}}
+```
+
+CLI: `freellama task --task coding --model lfm3:1.7b-q4_K_M --input-file prompt.txt`
+
+**Check it:** run the code. Set `num_predict` high enough for a thinking model's reasoning, which counts as output, or pass `think:false` for a short answer: an answer that comes back empty says which.

package/docs/cookbooks/duplicates.md

@@ -0,0 +1,17 @@
+# Find duplicates among issues or texts
+
+**Use when** you need likely duplicate pairs from many short texts.
+
+**Model:** `nomic-embed-text:v1.5` (0.3 GB).
+
+Leave out `query`, and `nearest` returns the most similar pairs:
+
+```json
+{"tool": "run_task", "arguments": {"task": "embedding", "model": "nomic-embed-text:v1.5", "files": ["issues/*.md"], "nearest": {"top": 10, "documentPrefix": "search_document: "}}}
+```
+
+Two chunks of one file are never reported as a pair.
+
+**Read the result:** one line per pair, with a cosine score.
+
+**Check it:** a high score is a candidate, never a verdict. Read both before you close anything. Measured here: 8 of 12 real related issues found their partner as the nearest neighbour.

package/docs/cookbooks/find-code.md

@@ -0,0 +1,21 @@
+# Find the code for an issue or question
+
+**Use when** you do not know which file to read and have no exact word to search for. The server embeds the files and returns ranked `path:Lstart-Lend` labels, so you read a few ranges instead of a repository.
+
+**Skip it when** you know an identifier: `localSearch` or `grep` is faster and exact.
+
+**Model:** `embeddinggemma-2:740m-q8_0` (1.3 GB). `nomic-embed-text:v1.5` (0.3 GB) indexes faster but found the right file less often.
+
+```json
+{"tool": "run_task", "arguments": {"task": "embedding", "model": "embeddinggemma-2:740m-q8_0", "files": ["src/**/*.rs"], "nearest": {"query": "task: search result | query: why does saving a long path crash", "top": 5, "documentPrefix": "title: none | text: "}}}
+```
+
+The prefixes belong to the model: `task: search result | query: ` and `title: none | text: ` for embeddinggemma; `search_query: ` and `search_document: ` for nomic. Other models have their own; check the model's page.
+
+CLI, for items one per line: `freellama task --task embedding --model embeddinggemma-2:740m-q8_0 --input-file items.txt --nearest "task: search result | query: saving crashes" --top 5`
+
+**Read the result:** one line per file, best chunk first, with a cosine score.
+
+**Check it:** read the top ranges before you rely on them. Measured here on 10 questions about this repository: embeddinggemma put the right file first for 9 and in the top 3 for 9; nomic, 6 and 8. A miss is still normal; widen `top` or rephrase before you conclude anything.
+
+Each call embeds the files again, so narrow the glob. A glob over about 900 chunks took about 67 s with embeddinggemma and 42 s with nomic.

package/docs/cookbooks/image-recognition.md

@@ -0,0 +1,15 @@
+# Understand what an image shows
+
+**Use when** you need to know what a picture, chart or screen shows, not just its text.
+
+**Skip it when** you only need the words: see [OCR](ocr.md).
+
+**Model:** `lfm3-vl:2b-q4_K_M` (1.5 GB). See `models {view:"catalog", job:"image-recognition"}` for the tested alternatives.
+
+```json
+{"tool": "run_task", "arguments": {"task": "vision", "model": "lfm3-vl:2b-q4_K_M", "prompt": "Which label has the tallest bar? Answer briefly.", "files": ["charts/usage.png"]}}
+```
+
+CLI: `freellama task --task vision --model lfm3-vl:2b-q4_K_M --image chart.png "Which label has the tallest bar?"`
+
+**Check it:** ask a specific question, not "describe this". Tested here on drawn images only; expect less on photos, and verify anything you act on.

package/docs/cookbooks/image-search.md

@@ -0,0 +1,13 @@
+# Find an image by describing it
+
+**Use when** you have a folder of screenshots or images and need the one that matches a description.
+
+**Model:** `embeddinggemma-2:740m-q8_0` (1.3 GB). Its text-only tags do not embed images.
+
+```json
+{"tool": "run_task", "arguments": {"task": "embedding", "model": "embeddinggemma-2:740m-q8_0", "files": ["screenshots/*.png"], "nearest": {"query": "task: search result | query: a compiler error in a terminal", "top": 3}}}
+```
+
+Image files are embedded whole, and each is one item. WAV files work the same way with a model that has audio input, such as the same model; audio has not been measured here.
+
+**Check it:** open the top match. Measured here: 5 of 5 on five drawn screens, too few to say more.

package/docs/cookbooks/jobs.md

@@ -0,0 +1,60 @@
+# Let FreeLlama pick the model and prompt (job mode)
+
+**Use when** one of the jobs below fits the work and you would rather not write the local model's prompt, options and embedding prefixes yourself. Pass `job` to `run_task`: FreeLlama runs the job's first catalog pick that is installed and fits now, with its own prompt for that job. Any field you pass wins, and `job_profile.applied` in the result lists what job mode filled in.
+
+**Skip it when** you need your own wording or decoding: pass `model`, `prompt` and `options` yourself, as the other recipes do.
+
+| `job` | Task | First pick | What job mode fills in |
+|---|---|---|---|
+| `classify` | decision | `strands-decider:2b-nvfp4` | task and model; you send `questions` |
+| `find-code` | embedding | `embeddinggemma-2:740m-q8_0` | `task: code retrieval \| query: ` before your query, `title: none \| text: ` before each chunk |
+| `duplicates` | embedding | `nomic-embed-text:v1.5` | `clustering: ` before every item; the most similar pairs when you send no `nearest` |
+| `ocr` | vision | `hf.co/openbmb/MiniCPM-OCR-1.0-gguf:Q4_K_M` | prompt `Text Recognition:`; send `Table Recognition:` or `Formula Recognition:` as `prompt` for those |
+| `image-recognition` | vision | `lfm3-vl:2b-q4_K_M` | a brief-answer system prompt and the card's sampling; you send the question as `prompt` |
+| `image-search` | embedding | `embeddinggemma-2:740m-q8_0` | `task: search result \| query: ` before your query; images go in bare |
+| `coding` | coding | `lfm3:1.7b-q4_K_M` | a system prompt asking for SEARCH/REPLACE edits, and the card's sampling; you send the change as `prompt` |
+| `summarize` | completion | `lfm3:1.7b-q4_K_M` | a short-bullets system prompt that quotes numbers and names and says "not stated" rather than guess |
+
+The later picks of each job are the catalog's (`models {view:"catalog", job}`). Every value, with the model card it comes from, is in [local-prompts.ts](../../packages/mcp/src/local-prompts.ts); a sampling value is used only where the model's own card documents it.
+
+Read the text in one image:
+
+```json
+{"tool": "run_task", "arguments": {"job": "ocr", "files": ["screenshots/error.png"]}}
+```
+
+One answer per image, with the job sent once:
+
+```json
+{"tool": "run_task", "arguments": {"job": "ocr", "tasks": [{"id": "a", "independent": true, "task": {"files": ["screenshots/a.png"]}}, {"id": "b", "independent": true, "task": {"files": ["screenshots/b.png"]}}]}}
+```
+
+Find the code for an issue:
+
+```json
+{"tool": "run_task", "arguments": {"job": "find-code", "files": ["src/**/*.rs"], "nearest": {"query": "why does saving a long path crash", "top": 5}}}
+```
+
+Draft one small change:
+
+```json
+{"tool": "run_task", "arguments": {"job": "coding", "files": ["src/parse.ts"], "prompt": "Make parsePort reject ports above 65535."}}
+```
+
+**Apply coding edits yourself.** The answer is blocks of a path line, `<<<<<<< SEARCH`, the current lines, `=======`, the replacement and `>>>>>>> REPLACE`, or a path and one fenced block for a whole new file. Check that each SEARCH text occurs exactly once in its file, indentation included, replace it, skip a block that matches zero or several times, then run the tests.
+
+**Hand a result to the next task.** `saveAs` writes the answer to `handoffs/<name>.md` in the FreeLlama home, with a `## Summary` and a `## Result`, and returns its path as `handoff_file`. Another task reads it with `files`, so you never paste it back:
+
+```json
+{"tool": "run_task", "arguments": {"job": "summarize", "files": ["logs/build.log"], "prompt": "Which steps failed, and why?", "saveAs": "build-failures"}}
+```
+
+```json
+{"tool": "run_task", "arguments": {"job": "coding", "files": ["/Users/me/.octocode/freellama/handoffs/build-failures.md", "scripts/build.mjs"], "prompt": "Fix the first failure the summary names."}}
+```
+
+**When nothing fits now,** the refusal lists each pick, the memory it needs and what is free. Tell the user; pass `model` for an installed model with the same capability, or ask them to approve pulling one exact tag at its size.
+
+**MCP prompts.** Clients that show prompts (often as slash commands) list one per job: `ocr`, `image`, `classify`, `find-code`, `duplicates`, `coding`, `summarize`, `research`, `github-research` and `sweep`. Each returns the exact call to make; `freellama` explains when to offload and how to check results.
+
+**Check it:** as the job's own recipe says; read 3 to 5 results before relying on the rest.

package/docs/cookbooks/memory-and-caching.md

@@ -0,0 +1,31 @@
+# Keep local work light: memory, context and caching
+
+**See what fits before you start:**
+
+```json
+{"tool": "models", "arguments": {"view": "catalog", "job": "classify"}}
+```
+
+```json
+{"tool": "doctor", "arguments": {"view": "status"}}
+```
+
+**Load before a burst.** This loads the model and evaluates a shared prompt once, so later calls that send the same prefix skip that work. It saves local time, not your tokens: you still send the prefix.
+
+```json
+{"tool": "ollama_manage", "arguments": {"action": "warm", "model": "bonsai-2:9b", "systemPrompt": "You review release notes.", "contextTokens": 8192}}
+```
+
+**Free memory when a burst ends.** Unload a model you warmed, or one the user named. It waits for that model's running tasks.
+
+```json
+{"tool": "ollama_manage", "arguments": {"action": "unload", "model": "bonsai-2:9b"}}
+```
+
+**Caching.** Reuse needs the same text at the start of the prompt, so put stable text first and append rather than edit. `metrics.cached_prompt_tokens` shows what was reused. Measured here on `gpt-oss:20b`: 1,251 of 1,266 tokens reused, with prompt evaluation falling from 544 ms to under 40 ms.
+
+**Context window.** `contextTokens` is input plus output. A larger window uses more memory and can reload the model, so ask for what the prompt needs. FreeLlama never cuts a prompt silently.
+
+**When a load is refused,** the message says how much memory is available and needed. Tell the user, and never stop their processes. Retry once they free memory, or use a smaller installed model with the same capability; do not retry in a loop.
+
+Each pick's memory need is `memory_gb` in `models {view:"catalog"}`; measured models and their speeds are in [Choose local models](../MODEL_SELECTION.md).

package/docs/cookbooks/ocr.md

@@ -0,0 +1,17 @@
+# Read text from an image (OCR)
+
+**Use when** a screenshot, scan or photo of text must become text. Name the image by path: base64 pasted into your call costs you tens of thousands of tokens.
+
+**Skip it when** you need to know what an image shows rather than what it says: see [image recognition](image-recognition.md).
+
+**Model:** `hf.co/openbmb/MiniCPM-OCR-1.0-gguf:Q4_K_M` (1.4 GB), or `glm-ocr:latest` (2.2 GB).
+
+```json
+{"tool": "run_task", "arguments": {"task": "vision", "model": "hf.co/openbmb/MiniCPM-OCR-1.0-gguf:Q4_K_M", "prompt": "Text Recognition:", "files": ["screenshots/error.png"]}}
+```
+
+Both models document the prompt `Text Recognition:`. Paths must be under the server's working directory, the temp directory or `FREELLAMA_FILE_ROOTS`.
+
+CLI: `freellama task --task vision --model glm-ocr:latest --image shot.png "Text Recognition:"`
+
+**Check it:** crop to the text first; a whole page with one line on it can read as garbage. A note that the model looped means low confidence, so check that text. Measured here on clean rendered text only, not scans.

package/docs/cookbooks/research-agent.md

@@ -0,0 +1,27 @@
+# Ask a local agent to research a repository
+
+**Use when** a question would take you several searches and reads, and the answer is cheap to check: where is X, what calls Y, which PR added Z.
+
+**Skip it when** one search answers it, or when the answer needs your judgment.
+
+**Model:** `bonsai-2:9b` (3.2 GB), needing about 8.6 GB available.
+
+Local code, using the installed Octocode CLI's read-only tools. Pass the project's absolute path; a relative one resolves against the server's working directory, which is often not your project. The path must be under `FREELLAMA_MCP_ALLOWED_ROOTS`, which a published install requires:
+
+```json
+{"tool": "delegate_research", "arguments": {"question": "Which files call the function memory_holders?", "workspacePath": "/Users/me/code/my-project", "adapter": "octocode", "model": "bonsai-2:9b"}}
+```
+
+GitHub, without cloning or a `workspacePath`:
+
+```json
+{"tool": "delegate_research", "arguments": {"question": "Which file in ollama/ollama defines SystemOneRequest?", "adapter": "github", "model": "bonsai-2:9b"}}
+```
+
+The agent's instructions are [local](../octocode/LOCAL.md) and [GitHub](../octocode/GITHUB.md), which also work for any agent using the CLI directly.
+
+With `adapter:"bash"` (the default) the model gets structured `search`, `read` and `list` actions that FreeLlama runs correctly and confined (a literal search by default; a path written relative to a package is resolved; a regex sent as a literal is retried as one), plus a read-only shell for `rg`, `grep`, `find`, `sed -n` and `git`. The GitHub adapter also takes the GitHub CLI read-only (`gh search`, `gh repo view`, `gh pr view`, `gh api` GET) through your existing `gh` login.
+
+Every run starts with a short workspace overview (top level, file types, source folders). An answer that cites a file that does not exist, or names no source although its results did, gets one repair turn; an answer that still cites a missing file is marked unverified. Bigger models get more time (180 s up to 4 GB, then 20 s per GB); pass `agent.timeoutSeconds` to choose.
+
+**Check it:** discard `verification.recommendation:"escalate"`, and verify the citations otherwise. Measured here it was often right but also wrong with confidence, so treat answers as leads.

package/docs/cookbooks/speech-to-text.md

@@ -0,0 +1,19 @@
+# Turn a recording into text
+
+**Use when** a WAV recording must become text. You cannot hear audio yourself, so this has no self-service alternative.
+
+**Model:** `lfm3-audio:2b-q4_K_M` (1.7 GB), or `lfm3-omni:2b-q4_K_M` (2.0 GB), which also does images.
+
+```json
+{"tool": "audio", "arguments": {"action": "transcribe", "model": "lfm3-audio:2b-q4_K_M", "file": "recordings/standup.wav", "language": "en"}}
+```
+
+Translate into another language:
+
+```json
+{"tool": "audio", "arguments": {"action": "translate", "model": "lfm3-audio:2b-q4_K_M", "file": "recordings/standup.wav", "toLanguage": "fr"}}
+```
+
+The input must be WAV; convert other formats first (`ffmpeg -i a.mp3 a.wav`). Ollama splits long audio at pauses.
+
+**Check it:** numbers and names are the usual misses. Measured here on generated speech only (1.7% word error); expect more errors on real recordings.

package/docs/cookbooks/sweep.md

@@ -0,0 +1,66 @@
+# Sweep a large job in small, checked chunks
+
+**Use when** a check is too big to read yourself but splits cleanly: every file of a code tree, a long log, the pages of a docs site. A small local model judges each chunk with your questions, in the background and one chunk at a time. Every result is saved as it lands, and you read only what was flagged.
+
+**Skip it when** there are only a few items (use `run_task` with `tasks`), or when a missed finding would be costly and you will not verify the flagged ones.
+
+**Models:** a decision model for triage, such as the catalog's `classify` pick `strands-decider:2b-nvfp4`. Optionally add a stronger `verifyModel` that re-asks only what was flagged or unsure, and an `explainModel` with an `explainPrompt` that writes one note per confirmed flag. See `models {view:"catalog", job:"classify"}` for what is installed and fits now.
+
+## How it works
+
+1. **Split.** `files` (paths or globs) or `urls` (a polite crawl of public pages: robots.txt honored, GET only, at most 200 pages, no local or private hosts) are cut into labelled chunks of about 750 tokens: `path:Lstart-Lend` or `url#n`.
+2. **Triage.** The decision model answers your `questions` for every chunk. Yes/no (`noul`) questions flag a chunk at probability 0.5 or more; `flags` sets other rules, such as `{question:"kind", when:"bug", atLeast:0.7}`. A probability just below a threshold is **unsure**, not clear.
+3. **Verify and explain.** Flagged and unsure chunks are re-asked by `verifyModel`. Chunks that stay flagged get a short note from `explainModel`.
+4. **Save.** Each result is appended to `~/.octocode/freellama/sweeps/<id>/results.jsonl` before the next chunk starts. Transient refusals retry; memory refusals wait for memory. After a cancel or a server restart, `resume` continues where the sweep stopped.
+5. **Compare.** Pass `baseline` (an earlier sweep of the same source) and unchanged chunks reuse its results. The status then reports what is new, changed, unchanged and removed, so a re-run costs only what changed.
+
+## Calls
+
+Start a code check; it returns at once:
+
+```json
+{"tool": "sweep", "arguments": {"action": "start", "name": "error handling", "files": ["src/**/*.ts"], "model": "strands-decider:2b-nvfp4", "questions": {"swallowed": {"type": "noul", "instructions": "Does this code catch an error and silently ignore it?"}}, "verifyModel": "bonsai-2:9b"}}
+```
+
+Follow it. Each call blocks up to 50 s and streams progress when the client sends a progressToken:
+
+```json
+{"tool": "sweep", "arguments": {"action": "wait", "sweepId": "00000000-0000-4000-8000-000000000000"}}
+```
+
+Read the flagged chunks, then open the cited ranges yourself:
+
+```json
+{"tool": "sweep", "arguments": {"action": "results", "sweepId": "00000000-0000-4000-8000-000000000000", "filter": "flagged"}}
+```
+
+Hand every flagged chunk to another task: `saveAs` writes them, under a summary of the counts, to `handoffs/<name>.md` and returns `handoff_file`, which a later `run_task` reads with `files`:
+
+```json
+{"tool": "sweep", "arguments": {"action": "results", "sweepId": "00000000-0000-4000-8000-000000000000", "filter": "flagged", "saveAs": "swallowed-errors"}}
+```
+
+Check a docs site for outdated install steps, two link hops deep:
+
+```json
+{"tool": "sweep", "arguments": {"action": "start", "urls": ["https://docs.example.com/"], "crawl": {"maxPages": 40, "maxDepth": 2}, "model": "strands-decider:2b-nvfp4", "questions": {"outdated": {"type": "noul", "instructions": "Does this page tell the reader to run a command or install a version that looks deprecated?"}}}}
+```
+
+Re-run next week and see only what happened since:
+
+```json
+{"tool": "sweep", "arguments": {"action": "start", "files": ["src/**/*.ts"], "model": "strands-decider:2b-nvfp4", "questions": {"swallowed": {"type": "noul", "instructions": "Does this code catch an error and silently ignore it?"}}, "baseline": "00000000-0000-4000-8000-000000000000"}}
+```
+
+```json
+{"tool": "sweep", "arguments": {"action": "results", "sweepId": "00000000-0000-4000-8000-000000000000", "filter": "changed"}}
+```
+
+## Split the work well
+
+- One question per concern, phrased about one chunk ("Does this code…"), never about the whole repository.
+- Put judgment you must stand behind in your own review of the flagged chunks, not in the questions.
+- Keep one model per sweep. `maxParallelism` stays 1 unless the machine has spare slots (`doctor {view:"status"}`).
+- Split a source of more than 5,000 chunks into several sweeps, for example one per directory.
+
+**Check it:** read 3 to 5 flagged chunks and 2 clear ones before trusting the rest. Raise a threshold with `atLeast` when too much is flagged, and add `verifyModel` when too little of what is flagged holds up.

package/docs/cookbooks/text-to-speech.md

@@ -0,0 +1,15 @@
+# Speak text aloud
+
+**Use when** the user wants to hear a result. It does not save tokens: it is for the person, not for you.
+
+**Model:** `phonon:en` (0.1 GB, Apple silicon, Ollama 0.40.9 or later). See `models {view:"catalog", job:"text-to-speech"}` for rivals and languages.
+
+```json
+{"tool": "audio", "arguments": {"action": "speak", "model": "phonon:en", "text": "Tests passed. The release is ready.", "outputFile": "reply.wav"}}
+```
+
+Without `outputFile`, the WAV goes to the temp directory. The result is its path and length, never the audio. Input is limited to 4,096 characters; omit `voice` for the default.
+
+On macOS, play it with `afplay reply.wav`.
+
+**Check it:** listen once. The score in the catalog measures how well a machine transcribes the audio, not how natural it sounds.

package/docs/octocode/GITHUB.md

@@ -0,0 +1,30 @@
+Octocode's GitHub tools research public and private repositories without cloning them: find a repo, its code, its history, and its packages.
+Call: octocode <tool> '{"queries":[{...}]}'  (1-5 queries per call)
+Result: JSON with results[].data. Exit 0 found, 1 nothing found, 2 bad input (the message names the field), 3 not found, 4 not logged in (octocode auth login), 7 rate limited (wait).
+More: follow a printed next or {"cursor":"..."}; no next and no hasMore means the result is complete.
+
+ghSearchRepo: find a repository when you do not know its owner or name.
+  {"operation":"search","keywords":["local llm router"],"filters":{"language":"Rust"}}
+  {"operation":"get","owner":"ollama","repo":"ollama"}  one repo: description, stars, default branch
+artifactSearch: find a package's repository and versions.
+  {"operation":"resolve","ecosystem":"npm","packageName":"zod"}  also pypi, maven
+ghStructure: see a repo's layout; operation "refs" lists branches, "languages" its languages.
+  {"operation":"tree","owner":"ollama","repo":"ollama","path":"server","maxDepth":2}
+ghSearchCode: find a word or name inside a repo when you do not know the file.
+  {"owner":"ollama","repo":"ollama","keywords":["SystemOneRequest"]}  returns paths, lines and a commitSha
+ghGetFileContent: read a file you found; ask for the least that answers.
+  {"owner":"ollama","repo":"ollama","path":"api/types.go","view":"symbols"}  outline
+  {"owner":"ollama","repo":"ollama","path":"api/types.go","ranges":[{"startLine":600,"endLine":640}]}
+  {"owner":"ollama","repo":"ollama","path":"api/types.go","matchString":["type SystemOneRequest"]}
+ghSearchHistory: find why something changed: pull requests, issues, commits.
+  {"operation":"pullRequest","owner":"ollama","repo":"ollama","keywords":["systemone"]}
+  {"operation":"commit","owner":"ollama","repo":"ollama","keywords":["decision"]}  or operation "issue" with "filters":{"state":"open"}
+ghGetHistoryItem: read one pull request, or issue, you found.
+  {"operation":"pullRequest","owner":"ollama","repo":"ollama","number":18606,"sections":["body","files"]}
+
+Work smart:
+1. Narrow: repo, then search, then read only the ranges you need. Never read whole files first.
+2. Reuse what a result returns: its path, commitSha (as ref) and the next step it suggests.
+3. Search words are exact code words, not sentences: one to three identifiers beat a question.
+4. Nothing found proves little: try another word or check the repo name before saying it does not exist.
+5. Answer with owner/repo, path:line or PR number as evidence, and stop as soon as the question is answered.

package/docs/octocode/LOCAL.md

@@ -0,0 +1,29 @@
+Octocode is a read-only code research CLI: it finds code in a directory so you read a few exact lines, not whole files.
+Call: octocode <tool> '{"queries":[{...}]}'  (1-5 queries per call; paths relative to the working directory)
+Result: lines of findings, then "#" notes. Exit 0 found, 1 nothing found, 2 bad input (the message names the field), 3 path not found.
+More: copy a printed {"cursor":"..."} as the whole input. No cursor means the result is complete.
+
+structureSearch: see the layout or list files, when you do not know where to look.
+  {"operation":"tree","path":".","maxDepth":2}
+  {"operation":"files","path":".","include":["**/*.rs"],"view":"paths"}
+localSearch: find a word, name or error text, when you do not know the file.
+  {"path":".","matchString":["memory_holders"],"view":"files"}  which files
+  {"path":".","matchString":["memory_holders","holders"]}  matching lines with line numbers; several words mean any of them
+localFetch: read a file an earlier result named (src/a.rs below is a placeholder); ask for the least that answers.
+  {"path":"src/a.rs","view":"symbols"}  outline with line numbers
+  {"path":"src/a.rs","ranges":[{"startLine":40,"endLine":90}]}
+  {"path":"src/a.rs","matchString":["fn run"],"contextLines":{"before":2,"after":15}}
+astSearch: find code by shape, not by text. Use it to find where something is declared.
+  {"operation":"symbols","path":".","symbolFilter":{"terms":["run"],"mode":"exact"}}  declarations named run
+  {"operation":"match","path":".","rule":{"pattern":"eval($X)"}}  every call of this shape
+lspSearch: who defines, uses or calls a symbol; exact where text search only guesses. Take path and line from an earlier result, never guess them.
+  {"path":"src/a.rs","symbolName":"run","line":35}  its definition
+  {"operation":"references","path":"src/a.rs","symbolName":"run","line":35,"view":"files"}  where it is used
+  {"operation":"callers","path":"src/a.rs","symbolName":"run","line":35}  also callees, hover, documentSymbols
+
+Work smart:
+1. Narrow before reading: list files or matching lines first, then read only those ranges.
+2. Try several words at once: put synonyms in one matchString, or several queries in one call.
+3. Build and dependency folders are skipped already; add "testCode":"exclude" to skip tests.
+4. Nothing found proves little: widen the words or the path before saying something does not exist. If lspSearch is empty, use localSearch.
+5. Answer with file:line evidence, and stop as soon as the question is answered.

package/native/index.d.ts

@@ -1,5 +1,6 @@
 /* auto-generated by NAPI-RS */
 /* eslint-disable */
+
 /**
  * Cancels one deferred task and waits for its local admission permits to be released.
  *
@@ -139,17 +140,6 @@ export declare function removeTaskJob(endpoint: string | undefined | null, jobId
  */
 export declare function route(endpoint: string | undefined | null, task: string, objective?: string | undefined | null, model?: string | undefined | null, sessionId?: string | undefined | null, contextTokens?: number | undefined | null, requiredCapabilities?: Array<string> | undefined | null, minConfidence?: string | undefined | null, executionPreference?: string | undefined | null, minPlacementEvidence?: string | undefined | null): Promise<string>
 
-/**
- * Executes caller-declared independent managed tasks with bounded, priority-fair dispatch.
- *
- * # Errors
- *
- * Returns an error if `freellama serve` is unreachable or rejects the batch envelope. Individual
- * task failures remain explicit entries in the successful batch response so sibling work is not
- * discarded.
- */
-export declare function runTaskBatchRequest(endpoint: string | undefined | null, request: unknown): Promise<string>
-
 /**
  * Object-based managed task API for callers that need the complete task contract without a long,
  * ABI-fragile positional argument list. The request is forwarded to `/_freellama/v1/tasks`, where