@octocodeai/freellama-mcp-server 0.1.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE-APACHE ADDED
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
package/LICENSE-MIT ADDED
@@ -0,0 +1,18 @@
1
+ MIT License
2
+
3
+ Copyright (c) FreeLlama contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
6
+ associated documentation files (the "Software"), to deal in the Software without restriction, including
7
+ without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
8
+ copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
9
+ following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be included in all copies or substantial
12
+ portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
15
+ LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
16
+ EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
17
+ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
18
+ USE OR OTHER DEALINGS IN THE SOFTWARE.
package/README.md CHANGED
@@ -1,23 +1,34 @@
1
1
  # FreeLlama MCP server
2
2
 
3
- Exposes FreeLlama's local-LLM control plane, and Ollama's lifecycle, as
4
- [MCP](https://modelcontextprotocol.io) tools, built on the official
5
- [TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk).
3
+ ![A brown cartoon llama stands on a pastel cloud background.](assets/logo.jpg)
6
4
 
7
- ## Understand the eight tools
5
+ This package exposes FreeLlama's management layer as Model Context Protocol (MCP) tools.
6
+ Agents offload bounded tasks to local Ollama models; FreeLlama handles model selection, resource
7
+ admission, memory checks, and routing to configured CPU/GPU backends. Results include execution
8
+ evidence for the calling agent to inspect.
8
9
 
9
- - `doctor`, `models`, `run_task`, `run_task_batch` — thin wrappers over the native NAPI bindings into the
10
+ The agent owns prompts, task dependencies, and answer verification. The operator configures
11
+ Ollama and model lifecycle permissions; Ollama performs inference. See
12
+ [architecture](../../docs/ARCHITECTURE.md) for the full ownership boundary.
13
+ The server uses the official [TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk).
14
+
15
+ ## Understand the tools
16
+
17
+ - `doctor`, `models`, `run_task`, `task_jobs`, `run_task_batch` — thin wrappers over the native NAPI bindings into the
10
18
  Rust core (`../rust-core/src/napi.rs`); no CLI subprocess, no reimplemented routing logic.
11
- `run_task { preview: true }` is the free decision-only form (the former `route` tool).
19
+ `run_task { preview: true }` is the decision-only form and does not run inference.
12
20
  Preview and execution are separate calls: preview accepts routing fields only and rejects task
13
21
  payloads or runtime controls instead of silently ignoring them.
14
22
  `models { view: "library" }` queries the public `ollama.com` library (the former `search_models` tool).
23
+ - `scope` — creates, inspects, forks, or deletes bounded process-local message history.
24
+ - `warm_model` — warms an exact installed model through ordinary managed admission and placement.
15
25
  - `session` — creates and releases a bounded, idle-expiring model-affinity handle for related
16
26
  route/task calls. It does not store messages, prompt history, or Ollama's runner KV cache.
17
27
  - `ollama_manage`, `ollama_delete` — Ollama's HTTP API for lifecycle operations the routing layer
18
28
  doesn't cover.
19
29
  - `delegate_research` — offloads a grounded code-research question to a local model and returns a
20
- verdict computed from what the run did.
30
+ verdict computed from model evidence, successful tool calls, and question-shape heuristics.
31
+ The verdict guides verification; it does not certify answer correctness.
21
32
 
22
33
  Use the smallest tool that owns the operation:
23
34
 
@@ -27,8 +38,11 @@ flowchart TD
27
38
  Q -->|"Diagnose the local runtime"| D["doctor"]
28
39
  Q -->|"Inspect installed, resident, detailed, raw, or online models"| M["models"]
29
40
  Q -->|"Choose or execute one task"| R["run_task"]
41
+ Q -->|"Inspect or cancel deferred work"| J["task_jobs"]
30
42
  Q -->|"Fan out independent tasks"| B["run_task_batch"]
31
43
  Q -->|"Keep/release related-task model affinity"| S["session"]
44
+ Q -->|"Retain or fork message history"| H["scope"]
45
+ Q -->|"Prepare runner residency"| W["warm_model"]
32
46
  Q -->|"Pull or stop a model"| O["ollama_manage"]
33
47
  Q -->|"Permanently remove a named model"| X["ollama_delete"]
34
48
  Q -->|"Ground an answer in workspace files"| G["delegate_research"]
@@ -39,15 +53,36 @@ flowchart TD
39
53
  | `doctor` | Makes runtime, version, memory, and configuration drift visible before work starts. |
40
54
  | `models` | Separates installed, resident, detailed, raw, and public-library questions instead of guessing from model names. |
41
55
  | `run_task` | Applies the same deterministic routing, confidence, admission, and execution contract as the Rust control plane. |
56
+ | `task_jobs` | Lists, retrieves, cancels, or removes a bounded process-local task submitted with `defer:true`. Removal stops active work first. |
42
57
  | `run_task_batch` | Executes only caller-declared independent tasks. Stable IDs, a bounded dispatcher, 3:2:1 fair priority classes, and per-item errors make fan-out inspectable rather than implicit. |
43
58
  | `session` | Lets an agent explicitly own the lifetime of a bounded affinity handle without misrepresenting it as conversation or KV storage. |
59
+ | `scope` | Retains bounded message history with expected revisions and explicit history reads. |
60
+ | `warm_model` | Loads an installed runner profile using the same memory, admission, placement, and job controls as task execution. |
44
61
  | `ollama_manage` | Exposes additive lifecycle operations that FreeLlama routing intentionally does not perform. |
45
62
  | `ollama_delete` | Keeps irreversible deletion separate and explicit so a client can guard it. |
46
63
  | `delegate_research` | Gives a local model bounded read-only tools and returns citations plus an independently computed verification verdict. |
47
64
 
48
65
  ## Follow the agent workflow
49
66
 
50
- The server sends this workflow to every MCP client in its initialization instructions:
67
+ The task lifecycle uses the UUID returned as `job.id` by `run_task {defer:true}`. A job is one
68
+ submitted task; pass that UUID as `jobId` for every subsequent operation.
69
+
70
+ | Operation | MCP call |
71
+ |---|---|
72
+ | List models | `models {view:"installed"}` |
73
+ | Submit a task and receive its ID | `run_task {prompt:"Explain this code", defer:true}` |
74
+ | List tasks with their states | `task_jobs {action:"list"}` |
75
+ | Get one task's state, result, or error | `task_jobs {action:"get", jobId:"<returned UUID>"}` |
76
+ | Stop one task and retain its record | `task_jobs {action:"cancel", jobId:"<returned UUID>"}` |
77
+ | Stop one task and remove its record | `task_jobs {action:"remove", jobId:"<returned UUID>"}` |
78
+
79
+ Removal waits for local permits to be released before deleting the record. It returns the ID and
80
+ terminal status; subsequent lookup returns `404`. It does not unload a shared Ollama model or
81
+ prove that physical computation has stopped. Receipts are bounded and lost on server restart;
82
+ see [retention and task states](../../docs/MONITORING.md#read-the-live-status).
83
+
84
+ Follow this workflow. The server's initialization instructions carry a compressed form of the
85
+ same contract (inventory first, preview semantics, evidence levels, ownership, and pull approval):
51
86
 
52
87
  1. Start with `models {view:"installed"}`, then `models {view:"resident"}`. This gives a routing
53
88
  agent the smallest current inventory first.
@@ -89,7 +124,7 @@ Use this economical sequence:
89
124
  For a delegated agent, estimated usable input is `contextTokens - outputTokens - safetyMarginTokens`
90
125
  (the default margin is 256). The adapter pins the system instruction and question, preserves the
91
126
  newest observations, compacts older observations into breadcrumbs, and refuses a pinned overflow
92
- by default. `pinnedOverflow:"clip"` is a deliberate quality-risk override. Initial token counting
127
+ by default. `FREELLAMA_AGENT_PINNED_OVERFLOW=clip` is a deliberate quality-risk override. Initial token counting
93
128
  is conservative; successful Ollama calls calibrate later estimates.
94
129
 
95
130
  Do not confuse operational evidence with answer quality. A receipt can prove admission, token
@@ -137,7 +172,7 @@ placement and admission. Ollama and the OS/driver own runner loading and physica
137
172
 
138
173
  ```mermaid
139
174
  flowchart TD
140
- C["MCP client"] -->|"stdio JSON-RPC"| B["dist/index.js: eight tool registrations"]
175
+ C["MCP client"] -->|"stdio JSON-RPC"| B["dist/index.js: tool registrations"]
141
176
  B --> N["native/index.js and platform addon"]
142
177
  N -->|"doctor"| L["Rust diagnostics"]
143
178
  N -->|"models installed/resident; run_task"| S["freellama serve"]
@@ -200,12 +235,34 @@ artifact); `native/index.js` and `native/index.d.ts` are hand-written and checke
200
235
 
201
236
  ## Run
202
237
 
238
+ **From npm (recommended):**
239
+
240
+ ```bash
241
+ npx @octocodeai/freellama-mcp-server
242
+ ```
243
+
244
+ **From the repository root after building:**
245
+
203
246
  ```bash
204
- node dist/index.js
247
+ node packages/mcp/dist/index.js
248
+ ```
249
+
250
+ The server speaks MCP over stdio — configure it in your MCP client, do not run it interactively.
251
+
252
+ **Claude Desktop (`claude_desktop_config.json`):**
253
+
254
+ ```json
255
+ {
256
+ "mcpServers": {
257
+ "freellama": {
258
+ "command": "npx",
259
+ "args": ["@octocodeai/freellama-mcp-server"]
260
+ }
261
+ }
262
+ }
205
263
  ```
206
264
 
207
- The server speaks MCP over stdio — launch it from an MCP client (for example, Claude Desktop's
208
- `claude_desktop_config.json`), not interactively.
265
+ **Cursor / other clients** — use the same `command`/`args` shape in their MCP server config.
209
266
 
210
267
  ## Test
211
268
 
@@ -214,28 +271,47 @@ Vitest, in three tiers (all TypeScript, under `test/`):
214
271
  | Tier | Command (from this package) | Checks | Needs |
215
272
  |---|---|---|---|
216
273
  | `test/unit/` | `yarn test` (watch: `yarn test:watch`) | pure functions straight from `src/*.ts` — no build step, this is the TDD loop | nothing, ~0.5s |
217
- | `test/integration/` | `yarn test:integration` | the built server over the real MCP protocol: tool contract, structured content, guardrails; rebuilds `dist/` itself | live Ollama on :11434, ~5s |
274
+ | `test/integration/` | `yarn test:integration` | the built server over the real MCP protocol: tool contract, structured content, guardrails; rebuilds `dist/` itself | Ollama at `FREELLAMA_OLLAMA_ENDPOINT` (default :11434) |
218
275
  | `test/e2e/` | `yarn test:e2e` | every tool against the live system: real routing/execution, pull → delete round trip (net-zero), a real delegated research run | serve + Ollama + models, ~40s |
219
276
 
220
277
  The same commands work from the repository root (`yarn test` there also runs the CLI package's tests).
221
278
  The integration/e2e tiers fail fast with a readable message when Ollama is down, and the lifecycle
222
279
  test refuses to delete a model already installed on the test system.
223
280
 
281
+ For default-endpoint autostart, a spawn failure, early exit, or readiness timeout rejects the
282
+ request while leaving the MCP transport available for diagnosis or retry. Autostart does not apply
283
+ to an explicit non-default endpoint; see `FREELLAMA_MCP_AUTOSTART_SERVE` below.
284
+
224
285
  ## Tools
225
286
 
226
287
  | Tool | Needs `freellama serve`? | What it does |
227
288
  |---|---|---|
228
- | `doctor` | No | Runtime/configuration diagnostic. `summary` (default) returns endpoint, version, resident count, host/profile signals, and next step; `scheduler` adds configured/snapshot admission data; `config` returns categorized settings; `full` adds all non-duplicated diagnostics. The report distinguishes an observed same-user macOS process from configuration hints and never claims remote process visibility. |
229
- | `models` | `installed` (default) and `resident` do; `detail`/`raw`/`library` don't | Views: `installed` (capabilities, derived `model_type`, VRAM, context, policy_rank), `resident` (loaded now + managed GPU/CPU split), `detail` (one model, real max context), paged `raw` (`GET /api/tags`), `library` (two-step ollama.com search: omit `model` for families, pass `model:"<family>"` for tags/`fitsInMemory`) |
289
+ | `doctor` | No | Runtime/configuration diagnostic. `summary` (default) returns endpoint, version, resident count, host/profile signals, and next step; `scheduler` adds configured/snapshot admission data; `config` returns categorized settings; `full` adds all non-duplicated diagnostics; `status` returns serve's live view (per-backend queues, current and adaptive limits, circuit breakers, loaded models, host memory, Ollama's effective settings, today's usage); `usage` returns task and token totals per day and model (`days`, default 7). The report distinguishes an observed same-user macOS process from configuration hints and never claims remote process visibility. |
290
+ | `models` | `installed` (default) and `resident` do; `detail`/`raw`/`library` don't | Views: `installed` (capabilities, use explanations, derived `model_type`, VRAM, context, policy_rank), `resident` (loaded now + managed GPU/CPU split), `detail` (one model, maximum context), paged `raw` (`GET /api/tags`), `library` (two-step ollama.com search: omit `model` for families, pass `model:"<family>"` for all tags/`fitsInMemory`). `includeLibrary:true` enriches installed/detail views with sourced public guidance; `includeReadme:true` returns bounded README text in library step 2 or enriched detail. |
230
291
  | `run_task` | Yes | **Routes or executes** a chat/generate/embed call. `preview: true` accepts routing fields only and returns a decision without generation. Execution omits preview and supplies the payload. Embedding results withhold raw vectors by default (`returnEmbeddings: true` to get them) |
292
+ | `scope` | Yes | Creates, inspects, forks, or deletes process-local message history; history inclusion requires `includeMessages:true`. |
293
+ | `warm_model` | Yes | Warms an exact installed model through managed admission; accepts residency/profile controls and optional `defer:true`. |
294
+ | `session` | Yes | Creates or releases model affinity without storing messages or KV. |
295
+ | `task_jobs` | Yes | Inspects, cancels, or removes a deferred task or warming operation. |
231
296
  | `run_task_batch` | Yes | Executes only typed `{id, independent:true, task}` items. The nested `task` exposes the same task, routing, payload, and runtime controls as `run_task`; `maxParallelism` bounds fan-out. It is not a dependency graph scheduler. |
232
297
  | `ollama_manage` | No (direct to Ollama) | `action: "pull"` downloads a model; `action: "stop"` force-unloads it. Both additive and idempotent |
233
298
  | `ollama_delete` | No (direct to Ollama) | **Destructive**: permanently removes a model. Call only on an explicit human instruction naming the exact model |
234
299
  | `delegate_research` | Yes (and spawns the adapter) | Answers a grounded question from files under `workspacePath`; managed coding turns return placement receipts, citations, and an independent verification verdict |
235
300
 
301
+ ### Inspect capabilities and intended uses
302
+
303
+ Use `models {view:"installed", includeLibrary:true}` to enrich each installed model, or
304
+ `models {view:"detail", model:"gemma3:1b", includeLibrary:true, includeReadme:true}` for one model.
305
+ Website access is optional for installed/detail views. Public guidance requires a matching exact tag
306
+ and digest prefix; local capabilities remain authoritative and benchmark/policy evidence stays separate.
307
+ Family claims have source excerpts and fetch dates, and lookup failures retain local inventory.
308
+ Search reports `cloudAvailable` and `cloudOnly:null`; inspect exact tags to establish local availability.
309
+ See [Ollama model metadata](../../docs/MODEL_METADATA.md) for response fields, lookup states, cache bounds,
310
+ custom models, and variant limitations.
311
+
236
312
  ### Preserve Ollama request controls
237
313
 
238
- All eight tools declare an MCP object `outputSchema`. Schemas type the stable fields FreeLlama owns
314
+ The tools declare an MCP object `outputSchema`. Schemas type the stable fields FreeLlama owns
239
315
  (such as route decision, page, answer, and session identifiers) while leaving Ollama-owned nested
240
316
  payloads forward-compatible. `structuredContent` is canonical; normal text is a concise cue. For
241
317
  the narrow `delegate_research` legacy case, set `legacyText:true` to receive serialized JSON text.
@@ -245,16 +321,29 @@ refuses continuation if the live model list changed. Library tag responses use t
245
321
  shape. Do not request `doctor {view:"full"}` in an agent loop: use `summary`, `scheduler`, or
246
322
  `config` for the smallest diagnostic that answers the question.
247
323
 
324
+ ### Preview and execution
325
+
326
+ For ordinary chat, `run_task {prompt:"Hello"}` is enough once the server and an eligible model
327
+ are available. `task` defaults to `completion`, including inside batch items. Task profiles are
328
+ optional routing presets, not restrictions on what you can ask the model.
329
+
330
+ Default `balanced` routing prefers policy-qualified candidates when available. Without an eligible
331
+ policy candidate, it falls back to capability/context-compatible models and reports low confidence,
332
+ `quality_evidence:"none"`, and `balanced_without_quality_policy`. Set `minConfidence:"medium"`
333
+ to require evidence, or `objective:"quality"` to require policy-based selection unless you pin a model.
334
+ Memory admission, authentication, and capability requirements apply in every mode.
335
+
248
336
  `run_task` has two deliberately exclusive request shapes:
249
337
 
250
338
  - Preview: set `preview: true`; pass routing fields such as `task`, `objective`, `model`,
251
339
  `contextTokens`, `requiredCapabilities`, and placement/confidence gates. To preview tool use,
252
340
  pass `requiredCapabilities: ["tools"]` rather than function definitions.
253
- - Execution: omit `preview` (or set it to `false`) and pass `prompt`/`messages` for generative
341
+ - Execution: omit `preview` (or set it to `false`) and pass `prompt`, `systemPrompt`, or `messages` for generative
254
342
  tasks or `input` for embeddings, plus any applicable Ollama runtime controls.
255
343
 
256
- Preview rejects `prompt`, `messages`, `input`, `images`, `tools`, `keepAlive`, `format`, `think`,
257
- `options`, `logprobs`, `topLogprobs`, and `returnEmbeddings`. This prevents a client from attaching
344
+ Preview rejects `prompt`, `systemPrompt`, `messages`, `input`, `images`, `tools`, `keepAlive`, `format`, `think`,
345
+ `options`, `logprobs`, `topLogprobs`, `returnEmbeddings`, `priority`, `maxWaitSeconds`, `timeoutSeconds`, and
346
+ `defer`, `scopeId`, and `scopeRevision`. This prevents a client from attaching
258
347
  work to a decision-only call and mistakenly assuming it ran.
259
348
 
260
349
  `run_task.messages` preserves Ollama message fields beyond `role` and `content`, including images,
@@ -263,11 +352,54 @@ thinking, tool calls, and tool names. Managed requests also accept `format`, `th
263
352
  keys are rejected inside `options`. The raw proxy remains available when a caller needs an Ollama
264
353
  endpoint or feature that the managed MCP tool does not expose, including streaming.
265
354
 
355
+ Managed execution waits for a slot and safe host capacity within the server's waiting budget
356
+ (120 seconds by default). Set `maxWaitSeconds` on `run_task` or each batch task to request a
357
+ shorter budget; the server caps it at its configured limit. Resource waiters remain counted in
358
+ the bounded queue after returning their execution slots. They resume when capacity becomes
359
+ available, and session cancellation removes pending work. A full queue or an expired deadline
360
+ returns a capacity error with retry guidance.
361
+
362
+ Set `defer:true` on `run_task` to return a job ID immediately. Use `task_jobs {action:"list"}` for
363
+ metadata, `{action:"get", jobId}` for a retained result, and `{action:"cancel", jobId}` to cancel
364
+ that task and wait for local permits to be released. `get` summarizes embedding vectors by default;
365
+ set `returnEmbeddings:true` to include them. Cancellation does not unload a shared model or confirm
366
+ physical runner shutdown. Jobs expire on restart; see [CLI controls](../../docs/CLI.md#execute-tasks)
367
+ for input, result, registry, and retention limits. Batch items cannot defer.
368
+
369
+ Set `timeoutSeconds` on a task or batch item for a total deadline covering discovery, queueing,
370
+ loading, and inference. The server caps it at its task timeout. The admission wait budget remains
371
+ separate; a queue refusal can happen before the total deadline. `priority` selects a fair scheduling
372
+ class without bypassing resource checks.
373
+
374
+ You own the task instructions. Pass system prompts as `messages` entries with `role:"system"`,
375
+ or use `systemPrompt` to prepend one caller-supplied system message. FreeLlama preserves existing
376
+ messages and adds no task instructions of its own. A plain `prompt` becomes one user message,
377
+ and a supplied `messages` array takes precedence over `prompt`. This also applies to
378
+ `task:"code_review"` (an alias for `coding`) and `run_task_batch`: no review wrapper, mandatory
379
+ JSON review format, or review-specific output-token minimum is added.
380
+
381
+ The separate `delegate_research` adapter and natural-language routing interpreter retain internal
382
+ protocol prompts for their own bounded jobs. Those prompts are not injected into `run_task`
383
+ conversations. Ollama can still apply the selected model's own template or Modelfile defaults.
384
+
266
385
  `model_type` is a display-oriented value derived from Ollama's additive capabilities:
267
386
  `generative`, `multimodal`, `embedding_only`, or `unknown`. Routing continues to use the original
268
387
  capability set, not this summary label. Unknown future Ollama capabilities are omitted from the
269
388
  typed routing set instead of being treated as a known capability.
270
389
 
390
+ ### Retain history and warm models
391
+
392
+ Use `scope {action:"create",messages}` to retain a caller-owned prefix. Pass its returned ID and
393
+ revision as `scopeId` and `scopeRevision` with only the new input to `run_task`; successful execution
394
+ returns the next scope revision. Fork a snapshot before parallel branches. `sessionId` continues to
395
+ mean model affinity only.
396
+
397
+ Use `warm_model {model,contextTokens,keepAlive?}` for a matching runner profile. It accepts no task
398
+ payload and does not pull a model. Add `defer:true` to inspect its progress through `task_jobs`.
399
+ Explicit `keepAlive` wins over finite adaptive retention. See the canonical
400
+ [Scope history and model warming reference](../../docs/SCOPES_AND_WARMING.md) for schemas, limits,
401
+ concurrency, privacy, and cache boundaries.
402
+
271
403
  ### Use CPU assignments through MCP
272
404
 
273
405
  `run_task` delegates to `freellama serve`, so it honors the server's `--cpu-upstream` and
@@ -277,14 +409,27 @@ managed tasks execute on the primary GPU-capable process.
277
409
  Set `executionPreference` on `run_task` to `auto` (default), `prefer_cpu`, or `prefer_gpu`. This is a
278
410
  guarded hint: only exact operator-assigned CPU models are eligible, explicit model/session pins win,
279
411
  and `preview: true` returns `execution.preference_satisfied`, placement, upstream, admission, and a
280
- reason without generating. Automatic feedback normalizes work by tokens, waits for three successful
281
- warm, physically verified samples per task on both backends, requires a 10% advantage, and never
282
- steers the `quality` objective. `minPlacementEvidence:"observed"` fails closed unless resident
412
+ reason without generating. Automatic feedback normalizes work by tokens and requires three
413
+ comparable warm serial samples on both backends with a 10% advantage. It matches model digest,
414
+ explicit observed context, effective controls, endpoint-attributed process settings, and managed
415
+ admission class. Unknown identity, parallel intervals, and decision-only payload-free previews
416
+ use capacity and policy signals. Stored legacy totals do not qualify speed hints. Feedback does
417
+ not steer the `quality` objective. See [automatic feedback](../../docs/CPU_GPU_ROUTING.md#understand-automatic-feedback).
418
+ `minPlacementEvidence:"observed"` fails closed unless resident
283
419
  `/api/ps` evidence matches the configured processor; warm cold models once with `"configured"`.
284
420
  `keepAlive:"0"` uses an observe-then-unload transaction: FreeLlama retains the runner long enough
285
421
  to inspect `/api/ps`, accepts feedback only for verified placement, requests an explicit unload,
286
422
  then reports the unload verification in `execution.lifecycle`.
287
423
 
424
+ Preview `execution.agent_plan` separates `queue_readiness`, `resource_readiness`, and combined
425
+ `dispatch_readiness`. Inspect `execution.resource_assessment` for required bytes and missing telemetry;
426
+ the preview shares execution's footprint estimate but never reserves memory. Local inference requires
427
+ RAM telemetry by default; configure an explicit alternative on `serve`, not in a model prompt.
428
+ Resource errors retain a human-readable `error` plus machine-readable `code` and `resource_admission`
429
+ fields in the HTTP response. MCP preserves JSON refusals in both error text and `structuredContent`,
430
+ including queue-full codes, `retry_after_seconds`, resource assessments, and unload receipts.
431
+ Agents can use those fields to back off or explain a capacity hold without parsing human prose.
432
+
288
433
  `models {view: "resident"}` uses the managed catalog, so it includes resident models from both
289
434
  Ollama processes and labels explicitly assigned CPU models instead of querying only the primary
290
435
  server.
@@ -302,14 +447,14 @@ inspect or modify that separate server. Read
302
447
 
303
448
  ## Configuration
304
449
 
305
- Every default is overridable through an environment variable — set them in the MCP client's
450
+ The following settings have environment-variable overrides. Set them in the MCP client's
306
451
  server-launch config (for example `.mcp.json`'s `env` block) or the launching shell.
307
452
 
308
453
  | Variable | Default | Affects |
309
454
  |---|---|---|
310
455
  | `FREELLAMA_OLLAMA_ENDPOINT` | `http://127.0.0.1:11434` | `doctor` and all `ollama_*` tools' default Ollama endpoint |
311
456
  | `FREELLAMA_SERVE_ENDPOINT` | `http://127.0.0.1:11435` | Default `endpoint` for serve-backed tools and `delegate_research` |
312
- | `FREELLAMA_MCP_DEFAULT_MODEL` | `qwen3.8:27b-mlx` | `delegate_research`'s default `model` |
457
+ | `FREELLAMA_MCP_DEFAULT_MODEL` | unset (routes to an installed coding model) | `delegate_research`'s default `model` |
313
458
  | `FREELLAMA_MCP_DEFAULT_ADAPTER` | `bash` | `delegate_research`'s adapter (`octocode` to switch) |
314
459
  | `FREELLAMA_MCP_MAX_TURNS` | `8` | Max agent turns for `delegate_research` |
315
460
  | `FREELLAMA_MCP_DELEGATE_TIMEOUT_SECONDS` | `180` | Whole-subprocess timeout for `delegate_research` |
@@ -324,9 +469,25 @@ server-launch config (for example `.mcp.json`'s `env` block) or the launching sh
324
469
  | `FREELLAMA_TASK_TIMEOUT_SECONDS` | `900` | Generation calls; read by `../rust-core/src/napi.rs` |
325
470
  | `FREELLAMA_AUTH_TOKEN_FILE` | unset | Bearer-token file used by the native client and research adapters for all serve routes |
326
471
  | `FREELLAMA_AGENT_TOKEN_CALIBRATION_DIR` | platform data directory | Prompt-free, model-specific token-estimator calibration shared across adapter processes |
472
+ | `FREELLAMA_MCP_AUTOSTART_SERVE` | `1` | When the default loopback serve endpoint does not answer, start `freellama serve` as a child of the MCP server (stopped with it). `0` disables |
473
+ | `FREELLAMA_SERVE_BINARY` | checkout `target/release`/`debug`, then the platform package | Binary used for serve autostart |
474
+ | `FREELLAMA_AGENT_OS_SANDBOX` | `auto` | Bash adapter OS confinement: `bwrap` on Linux, `sandbox-exec` on macOS, when present. `off` keeps only the restricted shell |
475
+ | `FREELLAMA_AGENT_OCTOCODE_PACKAGE` | `octocode@19.1.0` | Pinned npm spec the Octocode adapter runs through `npx` |
476
+
477
+ Serve-side placement settings (read by `freellama serve`, not the MCP process):
478
+
479
+ | Variable | Default | Affects |
480
+ |---|---|---|
481
+ | `FREELLAMA_EVICT_IDLE_MODELS` | on | Unload idle, unpinned models using reuse and load costs when a cold load needs room. `0` turns eviction off |
482
+ | `FREELLAMA_CPU_NUM_THREAD` | half the logical cores (not set on macOS) | `num_thread` for CPU-placed tasks only; GPU tasks leave it to Ollama |
483
+ | `FREELLAMA_MAX_QUEUE_WAIT_SECONDS` | `120` | How long a managed task queues for a slot or memory |
484
+ | `OLLAMA_NUM_PARALLEL`, `OLLAMA_KV_CACHE_TYPE`, `OLLAMA_FLASH_ATTENTION` | Ollama's | Read as hints for the memory estimate: parallel slots multiply the KV cache, and `q8_0`/`q4_0` shrink it only with flash attention. Set them for serve the same way as for Ollama |
327
485
 
328
486
  Research-adapter settings are deployment defaults. The optional `delegate_research.agent` object
329
- exposes the same values per call using camelCase; explicit per-call fields win.
487
+ overrides the per-call budget (`maxTurns`, `contextTokens`, `outputTokens`, `temperature`, `seed`,
488
+ `think`, `keepAlive`, `requestTimeoutSeconds`, `toolTimeoutSeconds`); explicit per-call fields win.
489
+ The research subprocess receives only `FREELLAMA_*`, locale, temp, and proxy/CA variables from the
490
+ server environment.
330
491
 
331
492
  | Variable | Default | Affects |
332
493
  |---|---:|---|
@@ -360,14 +521,22 @@ between model templates.
360
521
  **Security:** `delegate_research` grants a local model read access to `workspacePath`. The path is
361
522
  confined to `FREELLAMA_MCP_ALLOWED_ROOTS` (default: this repository in a checkout; unset in a published
362
523
  install until you set it), resolved through symlinks so a link inside an allowed root can't escape
363
- it. The default `bash` adapter also rejects home-directory paths, `..`, and absolute paths outside
364
- that workspace. Production deployments should generate a permission-restricted token with
524
+ it. The default `bash` adapter runs an allowlist of read-only tools in a restricted shell with a
525
+ scrubbed environment and, where available, an OS sandbox; it also rejects command substitution,
526
+ home-directory paths, `..`, and absolute paths outside that workspace. Production deployments should generate a permission-restricted token with
365
527
  `freellama auth-token`, start `serve` with `--auth-token-file`, and set
366
528
  `FREELLAMA_AUTH_TOKEN_FILE` for MCP. Authentication covers managed and raw passthrough routes; use
367
529
  an external TLS and authorization layer for untrusted or multi-tenant networks.
368
530
 
369
531
  ### Runtime evidence
370
532
 
533
+ Admission weights can be set by task kind in the server runtime file's `[task_costs]` table.
534
+ Inspect `doctor {view:"full"}` at `platform_health.admission.costs.base_by_task` for effective
535
+ weights and `run_task {preview:true}` for current cost and capacity advice. The table changes
536
+ managed admission; it does not create Ollama
537
+ slots or relax memory guards. Existing queued work keeps its captured cost. See
538
+ [monitoring controls](../../docs/MONITORING.md#adaptive-concurrency).
539
+
371
540
  **No measurements are compiled in.** In a repository checkout, per-model research grades load from
372
541
  `benchmark/evidence/model-evidence.json`. A published package does not include that repository
373
542
  file, so its evidence table is empty unless you set `FREELLAMA_MCP_MODEL_EVIDENCE`. An unmeasured
@@ -400,17 +569,18 @@ See `skills/freellama/references/proxy-vs-serve.md` for the `proxy` vs `serve` d
400
569
 
401
570
  `packages/mcp/` is the portable JavaScript package. Its `package.json` lists the platform packages
402
571
  as exact-version optional dependencies; it intentionally does not embed a `.node` binary. Publish
403
- the eight `packages/native/*` packages first, then publish `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server`
404
- at the same version. `yarn release:verify:publish` refuses a release if any platform package has a
405
- missing, empty, or unpacked executable/addon pair.
572
+ the eight `packages/native/*` packages first, then publish `@octocodeai/freellama` (`npx @octocodeai/freellama`) and `@octocodeai/freellama-mcp-server` (`npx @octocodeai/freellama-mcp-server`)
573
+ at the same version. `yarn release:verify:publish` refuses a release if any of the ten packages
574
+ omits `assets/logo.jpg`, or if any platform package has a missing, empty, or unpacked
575
+ executable/addon pair.
406
576
 
407
577
  ```bash
408
578
  cd packages/mcp
409
579
  npm pack --dry-run # confirm exactly what a publish would ship
410
580
  ```
411
581
 
412
- Version 0.1.0 is not published to a registry. Treat `npm publish` as a real, irreversible public
413
- action, and confirm with the repository owner first.
582
+ Treat `npm publish` as a public release action. Follow the repository
583
+ [release procedure](../../RELEASE.md) and obtain the repository owner's release authorization first.
414
584
 
415
585
  ## Why native bindings
416
586