@octocodeai/freellama-mcp-server 0.5.0 → 0.6.1

package/docs/PRODUCTION.md

@@ -12,9 +12,12 @@ not provide tenant isolation or billing.
 
 - Install one Ollama CLI and server version. Run `freellama doctor` and resolve any mismatch.
 - Install Node.js 20 or later for MCP and npm distribution use.
-- Use the release artifacts for your operating system, or build with Rust 1.85 or later.
+- Use the release artifacts for your operating system, or build with Rust 1.88 or later.
 - Preinstall and benchmark exact model tags. Discovery never authorizes a pull.
 - Keep the primary and optional CPU Ollama processes on distinct loopback sockets.
+- In a slim or distroless container, install the CA bundle (`ca-certificates`) or set
+  `SSL_CERT_FILE` if any upstream uses `https://`. Without one, FreeLlama starts with an empty trust
+  store and warns: plain-HTTP upstreams work, HTTPS upstreams fail their certificate check.
 
 After a tag has been published, install that exact checksummed release on macOS or Linux:
 
@@ -31,18 +34,14 @@ published tag:
 powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1 -Version VERSION
 ```
 
-The [release workflow](../.github/workflows/release.yml) builds CLI and native artifacts for eight
-targets, assembles packages, verifies package contents, and publishes on version tags. The
-[CI workflow](../.github/workflows/ci.yml) runs Rust checks on macOS, Linux, and Windows, plus
-JavaScript/native checks on macOS and Linux. These workflows do not prove driver compatibility or
-complete the hardware promotion checklist below. Release artifact builds depend on the reusable CI checks.
-The publish step also verifies that a version tag matches the workspace version.
+The repository has no CI or release workflows; they were removed (see git history). Release from a
+clean checkout: `yarn release:version <version>`, `yarn release:build` (builds each target's
+executable and addon into `release-artifacts/<target>/`), `node scripts/assemble-release.mjs
+release-artifacts release`, then `yarn release:verify:publish`. The publish verifier checks that
+`FREELLAMA_RELEASE_TAG`, when set, matches the workspace version.
 Live E2E and hardware qualification remain separate operator checks; verify those receipts before tagging.
 
-For manual assembly, build and test every claimed target in a clean
-environment, place each executable and matching addon under `release-artifacts/<target>/`, run
-`yarn release:assemble release-artifacts release` and `yarn release:verify:publish`, then attach
-`release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
+Attach `release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
 first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Configure
 registry credentials in the release environment; never store them in the repository.
 
@@ -82,7 +81,7 @@ FreeLlama and Ollama have separate queues. FreeLlama caps retained managed waite
 `--max-queued-tasks`/`--cpu-max-queued-tasks`, then waits for at most
 `--max-queue-wait-seconds`. An admitted request can then enter Ollama's internal queue,
 which is bounded by `OLLAMA_MAX_QUEUE`. Raw compatibility traffic bypasses weighted managed
-admission and enters the primary Ollama queue through a separate one-stream default cap in `serve`.
+admission and, for generation, enters the primary Ollama queue through a separate one-stream default cap in `serve`.
 Set all limits from observed latency and overload behavior; changing one does not configure the
 others.
 
@@ -91,7 +90,7 @@ The following flow shows where each production setting applies:
 ```mermaid
 flowchart LR
     C["Client"] --> FQ["FreeLlama admission<br/>weighted budget + wait deadline"]
-    FQ -->|"refused"| E503["503 server busy"]
+    FQ -->|"refused"| E503["429 or 503 server busy"]
     FQ -->|"admitted"| OQ["Ollama queue<br/>OLLAMA_MAX_QUEUE"]
     RAW["Raw /api/* or /v1/*"] --> RC["Raw stream cap<br/>default 1 in serve"]
     RC -->|"refused"| E503
@@ -102,14 +101,24 @@ flowchart LR
 
 These values are conservative starting points, not universal constants. NVIDIA, AMD, Apple, and
 CPU-only hosts must pass hardware validation before promotion. See
-[Ollama and FreeLlama optimization](dev/OLLAMA_SYSTEM_OPTIMIZATION.md) for the ownership boundary and
 [Run models on CPU and GPU](CPU_GPU_ROUTING.md) for device-specific controls.
 
+Ollama already picks the accelerator, fits the model to device memory, splits layers between CPU
+and GPU, and schedules requests; FreeLlama decides which tasks reach it. These changes do not help:
+
+- Forcing one model's layers between CPU and GPU: partial CPU offload usually lowers decode speed.
+  Run a separate CPU-only Ollama process for a second model instead.
+- Running Ollama in Docker on macOS: Docker Desktop gives it no GPU acceleration there.
+- Raising context globally: K/V memory grows with context; size `num_ctx` per request.
+- Pinning several large models in one process without the memory arithmetic: weights, runtime
+  buffers and every active context must fit together.
+- `q4_0` K/V cache as a default: it trades more quality than `q8_0` for the memory it saves.
+
 Put the values in the service manager that owns each Ollama process: launchd on macOS, systemd or
 the container definition on Linux, and the Windows service wrapper. Shell exports and
 `launchctl setenv` are useful diagnostics but are login-session state, not a reboot-persistent
-production configuration. After a real service restart, rerun `doctor` and `skills/freellama/scripts/check.sh` and
-require the same visible values before admitting work.
+production configuration. After a real service restart, rerun `freellama doctor` and `freellama status`
+and require the same visible values before admitting work.
 
 ## Create the security and state files
 
@@ -170,7 +179,8 @@ only model identifiers, scale, and sample count; they contain no prompts.
 Keep adapter context settings distinct from Ollama server context. `FREELLAMA_AGENT_NUM_CTX`
 budgets the adapter conversation; older tool observations are compacted inside that budget while
 the system prompt and original question remain pinned by default. `OLLAMA_CONTEXT_LENGTH` controls
-direct Ollama requests, and managed FreeLlama tasks send their own `num_ctx`.
+direct Ollama requests; managed FreeLlama tasks size their own context and send `num_ctx` unless
+Ollama's known default covers it.
 
 ## Verify lifecycle and placement
 
@@ -187,22 +197,19 @@ Require these health contracts before admitting traffic:
 - `placement_observation: ollama_api_ps_after_execution`
 - `placement_evidence_gate: configured_or_observed`
 
-Run `skills/freellama/scripts/check.sh` with `FREELLAMA_AUTH_TOKEN_FILE` set. It verifies the
-active versions, visible Ollama settings, persisted-feedback contract, authentication contract,
-backend topology, and binary freshness. Also inspect the service definition for any setting that
-the Ollama process does not expose to FreeLlama.
+With `FREELLAMA_AUTH_TOKEN_FILE` set, run `freellama doctor` (Ollama and CLI versions, visible Ollama
+settings, machine) and read `/_freellama/v1/health` (FreeLlama version, `security`, `feedback.persistence`,
+`backends`) and `freellama status` (backend topology, loaded models, admission). Also inspect the
+service definition for any setting that the Ollama process does not expose to FreeLlama.
 
 ## Validate every hardware class
 
 Run compile and deterministic test checks from clean checkouts on Windows, Linux, and macOS. Live
 accelerator acceptance is separate because build machines do not necessarily represent production
 GPU drivers or memory topology. Prepare Apple Metal, NVIDIA Linux, AMD Linux, and NVIDIA Windows
-hosts, then run `benchmark/hardware/run_validation.py` directly on each host.
+hosts, then run a hardware acceptance check directly on each host.
 
-Each environment supplies its exact GPU and CPU model tags and a running FreeLlama endpoint. The
-portable runner in `benchmark/hardware/run_validation.py` executes GPU coding and CPU embedding
-concurrently, verifies physical processor receipts, checks admission receipts, and writes a JSON
-artifact. Add `--vision-model` and `--vision-image` for that host's OCR/vision gate.
+The portable hardware runners were removed from the repository; they remain in git history.
 
 Do not mark a hardware row validated until its archived receipt says `verdict: accept`. Missing
 hardware is an explicit release gap, not a result that documentation can waive.

package/docs/README.md

@@ -2,39 +2,31 @@
 
 FreeLlama helps agents offload bounded work to local models through Ollama. The calling agent owns
 the task and verifies the answer; FreeLlama manages model selection and resources around execution.
-Start with the [repository quick start](../README.md#quick-start) to run one managed task.
+Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/main/README.md#mcp-server) to run one managed task.
 
 ## Choose a guide
 
 | Goal | Guide |
 |---|---|
-| Connect an agent and choose a tool | [MCP server](../packages/mcp/README.md) |
-| Follow the agent's operating workflow | [FreeLlama skill](../skills/freellama/README.md) |
+| Connect an agent and choose a tool | [MCP server](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md) |
+| Follow the agent's operating workflow | [FreeLlama skill](https://github.com/bgauryy/FreeLlama/blob/main/skills/freellama/SKILL.md) |
 | Look up commands and task controls | [CLI reference](CLI.md) |
 | Understand task and resource ownership | [Architecture](ARCHITECTURE.md) |
-| Choose a model from measured evidence | [Model selection](MODEL_SELECTION.md) |
-| Interpret capabilities and public model metadata | [Model metadata](MODEL_METADATA.md) |
+| Choose a model and interpret its metadata | [Model selection](MODEL_SELECTION.md), [model metadata](MODEL_SELECTION.md#model-metadata) |
 | Configure CPU helpers alongside GPU work | [CPU/GPU routing](CPU_GPU_ROUTING.md) |
 | Inspect queues, memory, usage, and live settings | [Monitoring](MONITORING.md) |
 | Retain history, affinity, or residency | [Scopes and warming](SCOPES_AND_WARMING.md) |
-| Understand raw Ollama compatibility | [Sidecar boundary](OLLAMA_SIDECAR.md) |
-| Evaluate context offload and its cost | [Token economics](ECONOMICS.md) |
+| Understand raw Ollama compatibility | [Ollama sidecar](ARCHITECTURE.md#ollama-sidecar) |
+| Evaluate context offload and its cost | [Token economics](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#token-economics) |
 | Deploy the management layer | [Production runbook](PRODUCTION.md) |
-| Run development and release checks | [Testing](TESTING.md), [release procedure](../RELEASE.md) |
-| Embed the routing core | [Rust core](../packages/rust-core/README.md) |
-| Use the read-only dashboard | [Runtime view](../packages/view/README.md) |
+| Run development and release checks | [Testing](TESTING.md), [release packages](TESTING.md#verify-release-packages) |
+| Embed the routing core | [Rust core](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/README.md) |
+| Use the read-only status page | `/_freellama/ui` on a running `serve` (see [Monitoring](MONITORING.md)) |
 
 ## Evidence and design
 
 | Goal | Source |
 |---|---|
-| Describe the product and its limits | [Product positioning](PRODUCT_POSITIONING.md) |
-| Review dated findings and project comparisons | [Findings report](dev/FINDINGS_AND_POSITIONING.md) |
-| Measure inference, device activity, and thermal behavior | [Local performance testing](dev/LOCAL_PERFORMANCE_TESTING.md) |
-| Separate management controls from inference tuning | [Ollama optimization boundary](dev/OLLAMA_SYSTEM_OPTIMIZATION.md) |
-| Understand backend selection decisions | [Resource-routing ADR](dev/ADR_RESOURCE_AWARE_BACKEND_ROUTING.md) |
-| Choose a model, adapter, or hardware benchmark | [Benchmark index](../benchmark/README.md) |
-| Inspect research adapter contracts and caveats | [Adapter reference](../benchmark/local/docs/07-adapter-contracts.md) |
 
 Historical measurements describe one model, machine, and workload. Inspect current state and
 repeat the relevant checks before using them to select models or set resource limits.

package/docs/SCOPES_AND_WARMING.md

@@ -41,15 +41,15 @@ Create accepts the following optional fields:
 | HTTP field | MCP field | Meaning |
 |---|---|---|
 | `messages` | `messages` | Initial Ollama messages, preserving fields, content, and order. |
-| `route_defaults` | `routeDefaults` | Task-routing defaults for later scoped execution. |
-| `limits` | `limits` | Per-scope bounds within the server's configured limits. |
+| `route_defaults` | — | Task-routing defaults for later scoped execution (HTTP/CLI only). |
+| `limits` | — | Per-scope bounds within the server's configured limits (HTTP/CLI only). |
 
 Messages require a supported role and can omit content, including for assistant tool calls.
 When present, content must be a string.
 
 Routing defaults accept `task`, `objective`, `model`, `required_capabilities`, `context_tokens`,
-`execution_preference`, `min_placement_evidence`, and `min_confidence`. MCP uses the corresponding
-camelCase field names. Defaults do not grant admission or qualify a model. Explicit task fields
+`execution_preference`, `min_placement_evidence`, and `min_confidence`. MCP callers set routing per
+`run_task` call instead. Defaults do not grant admission or qualify a model. Explicit task fields
 take precedence, and execution checks current eligibility and resources.
 
 The metadata receipt contains `scope_id`, `revision`, `route_defaults`, `limits`, `message_count`,
@@ -59,7 +59,7 @@ Token estimates derive from the full serialized message bytes and count both med
 This storage-budget estimate can refuse media-heavy history before inference; it does not measure
 image token cost or determine the model context required to execute the history.
 
-Fork requires `revision` and accepts optional replacement routing defaults and limit overrides. It creates a
+Fork requires `revision`; HTTP/CLI forks also accept replacement routing defaults and limit overrides. It creates a
 separate history snapshot with revision `0`; later writes to either scope do not change the other.
 Fork independent branches before parallel execution instead of submitting concurrent writes to one
 scope.
@@ -138,15 +138,14 @@ precedence. Inspect effective values and their sources with
 | `total_max_bytes` | `FREELLAMA_SCOPE_TOTAL_MAX_BYTES` | `16777216` | Aggregate retained history-byte budget. |
 | `ttl_seconds` | `FREELLAMA_SCOPE_TTL_SECONDS` | `3600` | Idle expiry since create, fork, or successful append; reads do not renew it. |
 
-Per-scope `limits` accepts `max_messages`, `max_bytes`, `max_estimated_tokens`, and `ttl_seconds`;
-MCP uses `maxMessages`, `maxBytes`, `maxEstimatedTokens`, and `ttlSeconds`. A request can lower these
-bounds within server policy. Finite safety bounds remain operator-owned. Lowering runtime caps
+Per-scope HTTP/CLI `limits` accepts `max_messages`, `max_bytes`, `max_estimated_tokens`, and
+`ttl_seconds` and can only lower these bounds within server policy. Finite safety bounds remain operator-owned. Lowering runtime caps
 does not summarize or trim stored transcripts; subsequent writes must satisfy the active caps.
 Expired entries are removed during store operations.
 
 ## Warm API
 
-`POST /warm`, MCP `warm_model`, and CLI `warm` load an exact installed model with a matching
+`POST /warm`, MCP `run_task {warm:true}`, and CLI `warm` load an exact installed model with a matching
 non-embedding task profile. They do not accept prompts or messages, pull a model, or store conversation
 history. The default task profile is `completion`.
 

package/docs/TESTING.md

@@ -19,24 +19,27 @@ flowchart LR
 ## Install and build
 
 ```bash
+corepack enable   # Yarn version comes from package.json "packageManager"
 yarn install
 yarn build
 ```
 
-The root build compiles the host Rust release binary, NAPI addon, and single-file MCP JavaScript
-bundle. The native `.node` addon remains external to the JavaScript bundle because it is selected
+The root build compiles the host Rust release binary, NAPI addon, and MCP JavaScript bundles
+(server and research adapters). The native `.node` addon remains external to the JavaScript bundle because it is selected
 by platform triple at runtime. Published packages resolve it from an OS/CPU/libc-specific optional
 dependency rather than compiling Rust during installation.
 
 ## Run the test tiers
 
+The root `yarn build`, `yarn lint`, and `yarn test` run across every package: the Rust crates
+(`rust-core`, `cli`) through Cargo and the JavaScript workspaces (`mcp`, `cli`, `native/*`) through
+Yarn. Use the `:js` or `:rust` variants to run one side only.
+
 | Tier | Command | External requirements |
 |---|---|---|
-| JavaScript unit | `yarn test` | None |
-| JavaScript watch | `yarn test:watch` | None |
+| All package unit tests (JS + Rust) | `yarn test` | Rust toolchain |
+| JavaScript unit | `yarn test:js` | None |
 | Rust contracts | `yarn test:rust` | Rust toolchain |
-| Local-agent contracts | `yarn test:agents` | Python 3 standard library |
-| Hardware runner contracts | `python3 -m unittest discover -s benchmark/hardware -p 'test_*.py'` | Python 3 standard library; no inference |
 | TypeScript | `yarn typecheck` | None after install |
 | MCP integration | `yarn test:integration` | Ollama at `FREELLAMA_OLLAMA_ENDPOINT` (default `127.0.0.1:11434`) |
 | MCP end to end | `yarn test:e2e` | Ollama, `freellama serve`, and required models |
@@ -66,7 +69,7 @@ failures. MCP integration's `model-knowledge.test.ts` exercises the built server
 local HTTP fixtures for both Ollama API data and public HTML. It checks optional enrichment,
 assigned-backend inventory, tag pagination, validation, and preservation of benchmark/policy evidence.
 The fixture tests perform no inference, pulls, or deletions. See the
-[metadata reference](MODEL_METADATA.md) for the public states these tests protect.
+[metadata reference](MODEL_SELECTION.md#model-metadata) for the public states these tests protect.
 
 ## Run static Rust checks
 
@@ -133,11 +136,11 @@ Usage-ledger unit contracts cover ordered writes, bounded queue overflow, failed
 validation, replay, and flush behavior. `monitoring_contract` also checks that a blocked ledger path
 does not hold task completion. Persistence remains best effort, not a zero-loss promise.
 
-MCP `serve.test.ts` checks startup permission failure, retry, and child-lifecycle isolation. The
+MCP `serve.test.ts` checks startup permission failure, retry, child-lifecycle isolation, and the
+background start at launch. The
 protocol integration contracts verify that spawn failure leaves the MCP connection usable and that
 fixture-backed library requests fail on unexpected product errors. External catalog availability
-is separate from these deterministic checks. Runtime-view `app-windows.test.ts` checks legitimate
-Windows static paths alongside traversal refusal.
+is separate from these deterministic checks.
 
 Launcher fixtures use `process.execPath` to avoid an extra PATH-resolved shell wrapper. After the
 MCP fixture's real executable-permission refusal, a test-only transport launches that exact fixture
@@ -156,8 +159,8 @@ a resident routing preview to agree with execution under a low-memory-only hold.
 Run the isolated control timing experiments separately from the full suite:
 
 ```bash
-cargo test --test control_timing_contract -- --include-ignored --nocapture
-cargo test --test task_jobs_contract measure_deferred_control_latency -- --ignored --nocapture
+cargo test --test integration control_timing_contract -- --include-ignored --nocapture
+cargo test --test integration task_jobs_contract::measure_deferred_control_latency -- --ignored --nocapture
 ```
 
 These fixed-budget experiments use simulated backends to measure status, discovery, queue
@@ -188,32 +191,19 @@ After rebuilding the release, a fresh concurrent smoke check completed both mana
 Qwen, one embedding from Nomic, positive Qwen VRAM, zero Nomic VRAM, and primary Ollama 0.33.2
 through raw passthrough.
 
-For promotion, use the portable [hardware acceptance runner](../benchmark/hardware/README.md) and
-archive its JSON receipt with the release evidence. Compilation alone does not validate a driver,
-physical placement, shared-memory contention, or OCR quality.
-
-The runner measures separate sequential and parallel runs of identical workloads across at least
-three pairs, and checks every result before reporting successful-task throughput. It requires a
-configurable median gain (default 5%). Resident GPU bytes establish placement; they do not measure
-device busy percentage, bandwidth or energy efficiency. Sustained throughput and tail latency
-require a representative workload and separate device/thermal observations.
+The portable hardware acceptance, workload, comparison, and load runners were removed from the
+repository; they remain in git history. Compilation alone does not validate a driver, physical
+placement, shared-memory contention, or OCR quality.
 
 The historical CPU/GPU observations above predate the corrected paired-run acceptance gate.
-They remain records of those runs, and do not qualify a release under the current gate.
-
-For a GPU workload with independent expected answers, use
-`benchmark/hardware/run_workload.py --workload MANIFEST.json --output RECEIPT.json`. It accepts
-distinct generation or vision cases, runs at least three alternating pairs, retains every
-result, and rejects incomplete or incorrect workloads. Its client deadline also bounds a hung
-measurement worker. See the [hardware guide](../benchmark/hardware/README.md) for the manifest
-contract.
+They remain records of those runs and do not qualify a release.
 
 ## Verify release packages
 
 ```bash
 # Collect each target's `freellama` executable and `freellama.<target>.node`
 # under release-artifacts/<target>/, then stage all eight platform packages.
-yarn release:assemble release-artifacts release
+node scripts/assemble-release.mjs release-artifacts release
 yarn release:verify:publish
 ```
 
@@ -225,11 +215,8 @@ prove registry publication or another hardware class.
 
 ## Test one layer while iterating
 
-The October 2, 2026 fixture investigation retained failed launches, raw dynamic-loader samples,
-and passing reruns. After the executable-fixture correction described above, the normal
-`yarn test` command passed 152 tests in 1.83 seconds of suite time. Test and production deadlines
-remain unchanged. Preserve failed and passing receipts together; this result does not qualify
-production startup timing.
+When investigating a fixture failure, preserve failed and passing receipts together; a passing
+rerun does not qualify production startup timing.
 
 ```bash
 yarn build:native