@octocodeai/freellama-mcp-server 0.6.1 → 0.7.0

package/docs/ARCHITECTURE.md

@@ -395,9 +395,11 @@ authority. Configured settings alone do not prove physical processor placement.
 The shared [resource governor](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/resources/mod.rs) samples local
 available-memory estimates, load per logical CPU, OS memory pressure, active swap-out growth, and
 thermal throttling when reported. OS memory pressure holds every local demand; active swap-out
-growth holds new loads; CPU-load and thermal holds apply to CPU-assigned work and to cold loads
-(decoding weights is CPU work and a new heat source) but not to a bounded request on a runner
-already resident on the GPU, which neither waits for the cores nor cools the package by waiting (`ResourceDemand::is_held_by`; the assessment's
+growth holds new loads; the CPU-load hold applies only to CPU-assigned work (saturated cores slow
+nothing on the GPU, so raw passthrough, which cannot know residency, is not refused for it), and to
+all work on a host whose sampler sees no accelerator; the
+thermal hold applies to CPU-assigned work and to cold loads (a new heat source) but not to a bounded
+request on a runner already resident on the GPU (`ResourceDemand::is_held_by`; the assessment's
 `held_by` lists the holds that applied). Low available RAM (below the larger of 15% or 1 GiB, clearing after two
 healthy samples above the larger of 20% or 2 GiB) depends on the platform (`ResourcePolicy::low_memory`).
 A new load must leave that reserve free, but never more than its own size above the 1 GiB floor.

package/docs/CLI.md

@@ -148,10 +148,13 @@ cache (and is the hard cap in a container), the same shortfall unloads idle runn
 new load with `held_insufficient_capacity` until capacity returns, refusing with 503 at its deadline.
 OS memory pressure holds all work where the OS reports it (macOS, and Linux when the kernel
 exposes `/proc/pressure/memory`; Windows has no host collector yet). Active swapping and low RAM hold new loads only:
-an already-resident runner adds no memory and stays admitted. CPU-load (one-minute load at or above
-1.5 per logical CPU) and thermal-throttling holds apply to CPU-assigned work and to cold loads, which
-decode weights on the host cores and add a new heat source; a bounded request on a runner already
-resident on the GPU neither waits for the cores nor cools the package by waiting, so it proceeds. The receipt's `resource_admission.assessment.held_by`
+an already-resident runner adds no memory and stays admitted. The CPU-load hold (one-minute load at
+or above 1.5 per logical CPU) applies only to CPU-assigned work, which decodes on those cores; GPU
+work, loaded or not, proceeds, so another program's build never blocks a GPU model. On a host where
+the sampler sees no accelerator (no GPU device or GPU memory), every model runs on the CPU, so the
+hold applies to all work there. The
+thermal-throttling hold applies to CPU-assigned work and to cold loads, a new heat source; a bounded
+request on a runner already resident on the GPU proceeds. The receipt's `resource_admission.assessment.held_by`
 lists the holds that applied to the request.
 Both `serve` and `proxy` accept `--resource-telemetry-policy`:
 
@@ -231,7 +234,7 @@ Useful task options include:
 - `--context-tokens` for the complete input and output context window (`num_ctx`).
 - repeatable `--required-capability` constraints.
 - repeatable `--image` paths for vision tasks.
-- `--input-file` for batched embedding input, one item per line.
+- `--input-file` reads the input from a file: one item per line for `--task embedding` (batched in one request), otherwise the whole file is the prompt.
 - `--min-confidence low|medium` to refuse insufficiently evidenced routes before generation.
 - `--priority interactive|normal|background` for fair admission scheduling.
 - `--max-wait-seconds` for a shorter admission/resource wait budget.

package/docs/MODEL_SELECTION.md

@@ -4,8 +4,9 @@ To offload a task, select an installed model with the required capabilities and
 then check its memory fit under the intended concurrency. FreeLlama applies model eligibility
 and admission; the calling agent still verifies the output.
 
-The measurements in this guide come from one Apple M4 Pro machine with 48 GB unified memory and
-are not compiled into routing. Re-run
+The measurements in this guide come from single machines and are not compiled into routing:
+[Recommended starting points](#recommended-starting-points-by-form) from an Apple M2 Pro with 32 GB,
+the sections after it from an Apple M4 Pro with 48 GB unified memory. Re-run
 `npx @octocodeai/freellama doctor`, `models`, `bench-all`, and your quality suite on the target hardware.
 `doctor.machine.memory_bytes` is host RAM; a discrete GPU's usable VRAM must be observed separately.
 
@@ -26,6 +27,46 @@ flowchart TD
     M -->|"Yes"| B["Benchmark locally and add policy evidence"]
 ```
 
+## Recommended starting points by form
+
+Measured on an Apple M2 Pro with 32 GB, Ollama 0.40.8 and FreeLlama 0.6.1 on 2026-10-30, while
+other builds kept the one-minute load near 14 on 12 cores. Every check had a known answer; the
+batteries are small, so treat these as first picks to re-check on your machine, not rankings.
+
+| Form | Model (download) | Measured | Speed (warm) | How an agent calls it |
+|---|---|---|---|---|
+| Text, tools, JSON, reasoning | `gpt-oss:20b` (13.8 GB) | 7/7: exact output, arithmetic, bat-and-ball, code reading, JSON extraction, tool call, 9/9 ticket routing | 1.4 s per routed ticket | `run_task {model:"gpt-oss:20b", prompt, think:"low"}`; add `tools` or `format:"json"` |
+| Embeddings | `nomic-embed-text:v1.5` (274 MB) | Retrieval 4/4 top-1; ticket routing 9/9 from two examples per team | 51 ms per item, batched | `run_task {task:"embedding", input:[...], returnEmbeddings:true}` |
+| Decisions (classify, route, score) | `strands-decider:2b-nvfp4` (1.3 GB) | Ticket routing 9/9, return-policy yes/no 3/3, damage rubric 2/3 | 0.17 s per question | `POST <serve>/v1/systemone {model, state, questions}`: passthrough, no MCP tool yet |
+| OCR | `glm-ocr:latest` (2.2 GB) | 4/4 phrases from a dense report strip; 0/7 on text tasks | 11 s per strip at a 400-token cap | `run_task {task:"vision", model:"glm-ocr", prompt:"Text Recognition:", images:[...], stopOnRepetition:true}` |
+
+`gpt-oss:20b` reasons before answering; the battery ran it with `think:"low"` and a 2,048-token
+output cap, so leave room for the reasoning when you set `num_predict`. `glm-ocr` reads text from images only and keeps generating
+after the visible text ends, so cap its output or set `stopOnRepetition`. The decision model needs
+Apple silicon and answers only through Ollama's decision API; reach it through `serve` so the call
+still passes FreeLlama's memory checks and stream cap (from 0.6.2 a busy CPU no longer refuses it).
+
+### Classify text three ways
+
+The same nine support tickets, routed to billing, technical or returns:
+
+| Approach | Model | Correct | Per item | Use when |
+|---|---|---:|---:|---|
+| Embeddings and per-class centroids | `nomic-embed-text:v1.5` | 9/9 | 51 ms | Many items, stable classes, a few labeled examples per class |
+| Decision model | `strands-decider:2b-nvfp4` | 9/9 | 0.5 s (first call includes the load) | You need probabilities, yes/no policy checks or rubric scores |
+| Instruction model with JSON output | `gpt-oss:20b` | 9/9 | 1.4 s | Classes change per request or need explanation |
+
+A decision API request names its questions; each answer comes back under the same name:
+
+```bash
+curl -s http://127.0.0.1:11435/v1/systemone -H 'content-type: application/json' -d '{
+  "model": "strands-decider:2b-nvfp4",
+  "state": "Why did my credit card get billed two times?",
+  "questions": {"team": {"type": "choice", "instructions": "Which team handles this ticket?",
+    "criteria": {"billing": "Charges and invoices", "technical": "App errors", "returns": "Returns and refunds"}}}
+}'
+```
+
 ## Text, code, and vision
 
 `qwen3.8:27b-mlx` was the strongest general model in the local evaluations. It handled coding,

package/docs/PRODUCTION.md

@@ -11,8 +11,8 @@ not provide tenant isolation or billing.
 ## Meet the prerequisites
 
 - Install one Ollama CLI and server version. Run `freellama doctor` and resolve any mismatch.
-- Install Node.js 20 or later for MCP and npm distribution use.
-- Use the release artifacts for your operating system, or build with Rust 1.88 or later.
+- Install Node.js 22 or later for MCP and npm distribution use.
+- Use the release artifacts for your operating system, or build with Rust 1.89 or later.
 - Preinstall and benchmark exact model tags. Discovery never authorizes a pull.
 - Keep the primary and optional CPU Ollama processes on distinct loopback sockets.
 - In a slim or distroless container, install the CA bundle (`ca-certificates`) or set
@@ -42,8 +42,10 @@ release-artifacts release`, then `yarn release:verify:publish`. The publish veri
 Live E2E and hardware qualification remain separate operator checks; verify those receipts before tagging.
 
 Attach `release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
-first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Configure
-registry credentials in the release environment; never store them in the repository.
+first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Publish
+the Rust crates from the same commit with `cargo publish --workspace` (it uploads `freellama-core`
+before `freellama-cli`; run it with `--dry-run` first). Configure registry credentials (`npm login`,
+`cargo login`) in the release environment; never store them in the repository.
 
 ## Configure Ollama explicitly
 

package/docs/TESTING.md

@@ -210,7 +210,8 @@ yarn release:verify:publish
 Build every claimed CLI/native target in a clean environment, assemble `SHA256SUMS`, run the full
 deterministic suite, and inspect every platform package dry run before publishing explicitly.
 Publish the eight `@octocodeai/freellama-native-*` packages before `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server`, all
-at the exact same version. A successful local pack dry run proves package contents; it does not
+at the exact same version, then the crates with `cargo publish --workspace` (`--dry-run` first packages
+and compiles both crates from their published sources). A successful local pack dry run proves package contents; it does not
 prove registry publication or another hardware class.
 
 ## Test one layer while iterating

package/native/index.d.ts

@@ -1,5 +1,6 @@
 /* auto-generated by NAPI-RS */
 /* eslint-disable */
+
 /**
  * Cancels one deferred task and waits for its local admission permits to be released.
  *

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@octocodeai/freellama-mcp-server",
-  "version": "0.6.1",
+  "version": "0.7.0",
   "description": "MCP server exposing FreeLlama core as tools, via native NAPI bindings into packages/rust-core.",
   "repository": {
     "type": "git",
@@ -16,7 +16,7 @@
     "coding-agent"
   ],
   "engines": {
-    "node": ">=20.0.0"
+    "node": ">=22"
   },
   "type": "module",
   "main": "dist/index.js",
@@ -46,18 +46,18 @@
     "build:native": "yarn --cwd ../.. build:native"
   },
   "dependencies": {
-    "@modelcontextprotocol/sdk": "^1.30.0",
-    "zod": "^3.24.1"
+    "@modelcontextprotocol/sdk": "^1.32.2",
+    "zod": "^4.6.7"
   },
   "optionalDependencies": {
-    "@octocodeai/freellama-native-darwin-arm64": "0.6.1",
-    "@octocodeai/freellama-native-darwin-x64": "0.6.1",
-    "@octocodeai/freellama-native-linux-arm64-gnu": "0.6.1",
-    "@octocodeai/freellama-native-linux-arm64-musl": "0.6.1",
-    "@octocodeai/freellama-native-linux-x64-gnu": "0.6.1",
-    "@octocodeai/freellama-native-linux-x64-musl": "0.6.1",
-    "@octocodeai/freellama-native-win32-arm64-msvc": "0.6.1",
-    "@octocodeai/freellama-native-win32-x64-msvc": "0.6.1"
+    "@octocodeai/freellama-native-darwin-arm64": "0.7.0",
+    "@octocodeai/freellama-native-darwin-x64": "0.7.0",
+    "@octocodeai/freellama-native-linux-arm64-gnu": "0.7.0",
+    "@octocodeai/freellama-native-linux-arm64-musl": "0.7.0",
+    "@octocodeai/freellama-native-linux-x64-gnu": "0.7.0",
+    "@octocodeai/freellama-native-linux-x64-musl": "0.7.0",
+    "@octocodeai/freellama-native-win32-arm64-msvc": "0.7.0",
+    "@octocodeai/freellama-native-win32-x64-msvc": "0.7.0"
   },
   "author": "Guy Bary",
   "license": "Apache-2.0 OR MIT",
@@ -66,9 +66,10 @@
     "registry": "https://registry.npmjs.org/"
   },
   "devDependencies": {
-    "@types/node": "^22.10.0",
+    "@types/node": "^22.20.5",
     "esbuild": "^0.28.2",
-    "typescript": "^5.7.2",
-    "vitest": "^4.1.11"
+    "typescript": "^7.0.2",
+    "vite": "^8.3.5",
+    "vitest": "^5.0.5"
   }
 }