@octocodeai/freellama-mcp-server 0.5.0 → 0.6.0

package/CONFIGURATION.md

@@ -0,0 +1,108 @@
+# FreeLlama MCP configuration
+
+Settings for the [FreeLlama MCP server](README.md).
+
+## Environment variables
+
+The following settings have environment-variable overrides. Set them in the MCP client's
+server-launch config (for example `.mcp.json`'s `env` block) or the launching shell.
+
+| Variable | Default | Affects |
+|---|---|---|
+| `FREELLAMA_OLLAMA_ENDPOINT` | `http://127.0.0.1:11434` | Default Ollama endpoint for `doctor`, the `models` detail/raw/library views, and all `ollama_*` tools |
+| `FREELLAMA_SERVE_ENDPOINT` | `http://127.0.0.1:11435` | Default `endpoint` for serve-backed tools and `delegate_research` |
+| `FREELLAMA_MCP_DEFAULT_MODEL` | unset (routes to an installed coding model) | `delegate_research`'s default `model` |
+| `FREELLAMA_MCP_DEFAULT_ADAPTER` | `bash` | `delegate_research`'s adapter (`octocode` to switch) |
+| `FREELLAMA_MCP_MAX_TURNS` | `8` | Max agent turns for `delegate_research` |
+| `FREELLAMA_MCP_DELEGATE_TIMEOUT_SECONDS` | `180` | Whole-subprocess timeout for `delegate_research` |
+| `FREELLAMA_MCP_PULL_TIMEOUT_SECONDS` | `1200` | Default `ollama_manage` `"pull"` timeout |
+| `FREELLAMA_MCP_FETCH_TIMEOUT_SECONDS` | `30` | Timeout for other direct Ollama HTTP calls |
+| `FREELLAMA_MCP_ALLOWED_ROOTS` | checkout: this repository; published: must be set | Platform-separated allowlist of directories `delegate_research` may read (`:` on macOS/Linux, `;` on Windows) |
+| `FREELLAMA_MCP_MODEL_EVIDENCE` | checkout: `packages/mcp/evidence/model-evidence.json`; published: unset | Path to per-model research grades (see "Runtime evidence") |
+| `FREELLAMA_EXTERNAL_COST_MODEL` | unset | External-model label in estimated avoided-cost telemetry; set all three cost variables or none |
+| `FREELLAMA_EXTERNAL_COST_INPUT_USD_PER_M` | unset | External input USD per million tokens |
+| `FREELLAMA_EXTERNAL_COST_OUTPUT_USD_PER_M` | unset | External output USD per million tokens |
+| `FREELLAMA_CONTROL_TIMEOUT_SECONDS` | `30` | Decision-only serve calls; read by `packages/rust-core/src/napi.rs` |
+| `FREELLAMA_TASK_TIMEOUT_SECONDS` | `900` | Generation calls; read by `packages/rust-core/src/napi.rs` |
+| `FREELLAMA_AUTH_TOKEN_FILE` | unset | Bearer-token file used by the native client and research adapters for all serve routes |
+| `FREELLAMA_AGENT_TOKEN_CALIBRATION_DIR` | platform data directory | Prompt-free, model-specific token-estimator calibration shared across adapter processes |
+| `FREELLAMA_MCP_AUTOSTART_SERVE` | `1` | When the default loopback serve endpoint does not answer, start `freellama serve` as a child of the MCP server (stopped with it). `0` disables |
+| `FREELLAMA_SERVE_BINARY` | checkout `target/release`/`debug`, then the platform package | Binary used for serve autostart |
+| `FREELLAMA_AGENT_OS_SANDBOX` | `auto` | Bash adapter OS confinement: `bwrap` on Linux, `sandbox-exec` on macOS, when present. `off` keeps only the restricted shell |
+| `FREELLAMA_AGENT_OCTOCODE_PACKAGE` | `octocode@19.1.0` | Pinned npm spec the Octocode adapter runs through `npx` |
+
+Serve-side placement settings (read by `freellama serve`, not the MCP process):
+
+| Variable | Default | Affects |
+|---|---|---|
+| `FREELLAMA_EVICT_IDLE_MODELS` | on | Unload idle, unpinned models using reuse and load costs when a cold load needs room. `0` turns eviction off |
+| `FREELLAMA_CPU_NUM_THREAD` | half the logical cores (not set on macOS) | `num_thread` for CPU-placed tasks only; GPU tasks leave it to Ollama |
+| `FREELLAMA_MAX_QUEUE_WAIT_SECONDS` | `120` | How long a managed task queues for a slot or memory |
+| `OLLAMA_NUM_PARALLEL`, `OLLAMA_KV_CACHE_TYPE`, `OLLAMA_FLASH_ATTENTION` | Ollama's | Read as hints for the memory estimate: parallel slots multiply the KV cache, and `q8_0`/`q4_0` shrink it only with flash attention. Set them for serve the same way as for Ollama |
+
+Research-adapter settings are deployment defaults. The optional `delegate_research.agent` object
+overrides the per-call budget (`maxTurns`, `contextTokens`, `outputTokens`); explicit per-call
+fields win. Sampling, keep-alive, timeouts, and placement preference for research come from the
+operator's `FREELLAMA_AGENT_*` variables or the adapter defaults.
+The research subprocess inherits only these server variables: `FREELLAMA_*`; `PATH`, `HOME`,
+`USER`, `LOGNAME`; `LANG`, `LC_ALL`, `LC_CTYPE`, `TZ`; `TMPDIR`, `TEMP`, `TMP`; Windows `SYSTEMROOT`,
+`SYSTEMDRIVE`, `WINDIR`, `COMSPEC`, `PATHEXT`, `APPDATA`, `LOCALAPPDATA`, `USERPROFILE`;
+`HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY` (and lowercase forms); `NODE_EXTRA_CA_CERTS`,
+`SSL_CERT_FILE`, `SSL_CERT_DIR`, `REQUESTS_CA_BUNDLE`; `XDG_CACHE_HOME`, `XDG_DATA_HOME`,
+`npm_config_cache`.
+
+| Variable | Default | Affects |
+|---|---:|---|
+| `FREELLAMA_AGENT_NUM_CTX` | `8192` | Ollama context window |
+| `FREELLAMA_AGENT_EXECUTION_PREFERENCE` | `auto` | Managed coding-agent backend hint |
+| `FREELLAMA_AGENT_MIN_PLACEMENT_EVIDENCE` | `configured` | Set `observed` to fail closed on cold/mismatched placement |
+| `FREELLAMA_AGENT_NUM_PREDICT` | `512` | Reserved output budget |
+| `FREELLAMA_AGENT_TEMPERATURE` / `FREELLAMA_AGENT_SEED` | `0` / `42` | Repeatable decoding |
+| `FREELLAMA_AGENT_THINK` / `FREELLAMA_AGENT_KEEP_ALIVE` | `false` / `5m` | Ollama reasoning and residency |
+| `FREELLAMA_AGENT_REQUEST_TIMEOUT_SECONDS` | `600` | Each Ollama chat attempt |
+| `FREELLAMA_AGENT_TOOL_TIMEOUT_SECONDS` | Bash `30`, Octocode `45` | Each local tool call |
+| `FREELLAMA_AGENT_RETRY_ATTEMPTS` / `FREELLAMA_AGENT_RETRY_BACKOFF_SECONDS` | `2` / `5` | Conversation-level retry |
+| `FREELLAMA_AGENT_MAX_PARSE_REPAIRS` / `FREELLAMA_AGENT_PARSE_REPAIR_ECHO_CHARS` | `2` / `500` | Invalid JSON correction limit and retained reply text |
+| `FREELLAMA_AGENT_CHARS_PER_TOKEN` | `4` | First-call token estimate; later calibrated from Ollama |
+| `FREELLAMA_AGENT_SAFETY_MARGIN_TOKENS` | `256` | Input headroom beyond `num_predict` |
+| `FREELLAMA_AGENT_IMAGE_TOKEN_ESTIMATE` | `1024` | Bounded charge per image |
+| `FREELLAMA_AGENT_KEEP_RECENT` | `2` | Recent observations preserved verbatim |
+| `FREELLAMA_AGENT_COMPACT_PREVIEW_CHARS` | `180` | Old-observation breadcrumb size |
+| `FREELLAMA_AGENT_COMPACT_RETAIN_RATIO` | `0.8` | Current payload retained per emergency pass |
+| `FREELLAMA_AGENT_CLIP_HEAD_RATIO` | `0.667` | Head share when an emergency clip preserves both ends |
+| `FREELLAMA_AGENT_OBSERVATION_PAGE_CHARS` | `3000` | Lossless observation page target |
+| `FREELLAMA_AGENT_PINNED_OVERFLOW` | `error` | Preserve system/task bytes; `clip` is explicit opt-in |
+
+Invalid values fail before an Ollama request. Each successful result returns
+`contextManagement` with the resolved policy, calibration source/scale, estimated input budget,
+and compaction count. Ollama has no stable preflight tokenizer endpoint, so “exact before the first
+call” is not claimed. Model-specific calibration persists after successful calls, so later
+processes start conservatively from prior `prompt_eval_count` evidence without sharing estimates
+between model templates.
+
+**Security:** `delegate_research` grants a local model read access to `workspacePath`. The path is
+confined to `FREELLAMA_MCP_ALLOWED_ROOTS` (default: this repository in a checkout; unset in a published
+install until you set it), resolved through symlinks so a link inside an allowed root can't escape
+it. The default `bash` adapter runs an allowlist of read-only tools in a restricted shell with a
+scrubbed environment and, where available, an OS sandbox; it also rejects command substitution,
+home-directory paths, `..`, and absolute paths outside that workspace. Production deployments should generate a permission-restricted token with
+`freellama auth-token`, start `serve` with `--auth-token-file`, and set
+`FREELLAMA_AUTH_TOKEN_FILE` for MCP. Authentication covers managed and raw passthrough routes; use
+an external TLS and authorization layer for untrusted or multi-tenant networks.
+
+## Runtime evidence
+
+Admission weights can be set by task kind in the server runtime file's `[task_costs]` table.
+Inspect `doctor {view:"full"}` at `platform_health.admission.costs.base_by_task` for effective
+weights and `run_task {preview:true}` for current cost and capacity advice. The table changes
+managed admission; it does not create Ollama
+slots or relax memory guards. Existing queued work keeps its captured cost. See
+[monitoring controls](https://github.com/bgauryy/FreeLlama/blob/main/docs/MONITORING.md#adaptive-concurrency).
+
+**No measurements are compiled in.** In a repository checkout, per-model research grades load from
+`packages/mcp/evidence/model-evidence.json`. A published package does not include that repository
+file, so its evidence table is empty unless you set `FREELLAMA_MCP_MODEL_EVIDENCE`. An unmeasured
+model yields a `verify` verdict, and an unreadable evidence file is reported on stderr and treated
+as empty. Tool descriptions state the rules; measurement details live in
+[`skills/freellama/references/`](https://github.com/bgauryy/FreeLlama/tree/main/skills/freellama/references/) and
+[`docs/dev/evidence/`](https://github.com/bgauryy/FreeLlama/tree/main/docs/dev/evidence/).

package/DEVELOPING.md

@@ -0,0 +1,109 @@
+# Developing the FreeLlama MCP server
+
+Build, test, and release notes for the [FreeLlama MCP server](README.md).
+
+## Build
+
+Build the native addon from the repository root and this package in place.
+
+```bash
+# From the repository root (yarn workspaces — one install for every package)
+corepack enable
+yarn install
+yarn build
+```
+
+`yarn build` writes **one** MCP file, `packages/mcp/dist/index.js` (shebang, executable). During
+checkout development it loads `native/freellama.<triple>.node`. Published installs resolve the
+matching `@octocodeai/freellama-native-<target>` optional package instead, so the base MCP package remains
+portable. Research adapters are copied into `adapters/` for packed installs.
+
+Piecewise, if you are iterating on one layer:
+
+```bash
+# 1. Native addon → packages/mcp/native/freellama.<triple>.node
+yarn build:native
+
+# 2. This server → dist/index.js via esbuild (then copies the research adapters into adapters/)
+yarn workspace @octocodeai/freellama-mcp-server build
+```
+
+`yarn start` needs `dist/index.js` and the compiled `.node` addon — `yarn build` from the repository
+root produces both in a checkout. `dist/index.js` loads `native/index.js`, which loads a local addon
+first and then a matching optional platform package. The `.node` binary is gitignored (a build
+artifact). `native/index.js` is a small hand-written loader; `native/index.d.ts` is generated from
+`napi.rs` by `yarn build:native`. Both are checked in; do not edit the declarations by hand —
+change the Rust doc comments or `ts_args_type`, rebuild, and commit. CI rebuilds it and fails
+on any diff.
+
+## Test
+
+Vitest, in three tiers (all TypeScript, under `test/`):
+
+| Tier | Command (from this package) | Checks | Needs |
+|---|---|---|---|
+| `test/unit/` | `yarn test` (watch: `yarn test:watch`) | pure functions straight from `src/*.ts` — no build step, this is the TDD loop | nothing, ~0.5s |
+| `test/integration/` | `yarn test:integration` | the built server over the real MCP protocol: tool contract, structured content, guardrails; rebuilds `dist/` itself | Ollama at `FREELLAMA_OLLAMA_ENDPOINT` (default :11434) |
+| `test/e2e/` | `yarn test:e2e` | every tool against the live system: real routing/execution, pull → delete round trip (net-zero), a real delegated research run | serve + Ollama + models, ~40s |
+
+The same commands work from the repository root (`yarn test` there also runs the CLI package's tests).
+The integration/e2e tiers fail fast with a readable message when Ollama is down, and the lifecycle
+test refuses to delete a model already installed on the test system.
+
+For default-endpoint autostart, a spawn failure, early exit, or readiness timeout rejects the
+request while leaving the MCP transport available for diagnosis or retry. Autostart does not apply
+to an explicit non-default endpoint; see `FREELLAMA_MCP_AUTOSTART_SERVE` in
+[configuration](CONFIGURATION.md#environment-variables).
+
+## Platform support
+
+The native addon is named by target triple (`freellama.<platform>-<arch>[-<abi>].node`). The base
+package has eight npm optional dependencies, one each for macOS arm64/x64, Linux arm64/x64 with
+glibc or musl, and Windows arm64/x64 with MSVC. `native/index.js` tries the checkout artifact first,
+then the host's optional package. A release is supported only when all eight artifact packages pass
+the publish verifier; an installation made with `--omit=optional` cannot load the addon. `engines`
+requires Node >= 20.
+
+Machine discovery distinguishes total host memory (`memory_bytes`) from known unified memory
+(`unified_memory_bytes`). Library-tag `fitsInMemory` is a conservative host-memory preflight and
+includes `fitScope: "host_memory_budget_only"`; inspect `/api/ps` for actual accelerator placement.
+
+Start the serve-backed tools' dependency from the repository root:
+
+```bash
+cargo run --release -- serve
+```
+
+See `skills/freellama/references/proxy-vs-serve.md` for the `proxy` vs `serve` distinction (only
+`serve` has the `/_freellama/v1/*` routes these tools call).
+
+## Publish
+
+`packages/mcp/` is the portable JavaScript package. Its `package.json` lists the platform packages
+as exact-version optional dependencies; it intentionally does not embed a `.node` binary. Publish
+the eight `packages/native/*` packages first, then publish `@octocodeai/freellama` (`npx @octocodeai/freellama`) and `@octocodeai/freellama-mcp-server` (`npx @octocodeai/freellama-mcp-server`)
+at the same version. `yarn release:verify:publish` refuses a release if any of the ten packages
+omits `assets/logo.jpg`, or if any platform package has a missing, empty, or unpacked
+executable/addon pair.
+
+```bash
+cd packages/mcp
+npm pack --dry-run   # confirm exactly what a publish would ship
+```
+
+Treat `npm publish` as a public release action. Follow the repository
+[release procedure](https://github.com/bgauryy/FreeLlama/blob/main/RELEASE.md) and obtain the repository owner's release authorization first.
+
+## Why native bindings
+
+- **No routing reimplementation.** `run_task`, `run_task {preview:true}`, and the installed-model list are HTTP
+  calls into a running `freellama serve`, matching the CLI tool map in `packages/cli/src/tools.rs`. `doctor` talks to Ollama
+  and the host's portable machine-discovery layer directly. `ollama_manage` / `ollama_delete` hit Ollama's HTTP API. `delegate_research`
+  is a subprocess. Routing decisions still live in exactly one place — the Rust core; see
+  [what the MCP layer computes](README.md#what-the-mcp-layer-computes).
+- **One connection pool.** `packages/rust-core/src/napi.rs` holds a single `reqwest::Client` in a
+  `OnceLock` and clones the handle per request, matching the server side.
+- **`unsafe_code` stays isolated.** The crate is `deny(unsafe_code)` except `napi.rs`, which
+  carries an explicit `#[allow(unsafe_code)]` for napi-derive's FFI glue.
+- **The napi build is off by default** (`cargo build` never touches it); the addon is built with
+  `napi build --features napi` from the repo root.