@octocodeai/freellama-mcp-server 0.1.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE-APACHE +202 -0
- package/LICENSE-MIT +18 -0
- package/README.md +203 -33
- package/adapters/agent_context.py +216 -29
- package/adapters/agent_transport.py +131 -0
- package/adapters/bash_agent.py +67 -92
- package/adapters/octocode_agent.py +63 -53
- package/adapters/shell_sandbox.py +487 -0
- package/assets/logo.jpg +0 -0
- package/dist/index.js +1287 -412
- package/docs/ARCHITECTURE.md +169 -26
- package/docs/CLI.md +107 -17
- package/docs/CPU_GPU_ROUTING.md +65 -20
- package/docs/ECONOMICS.md +20 -15
- package/docs/INDEX.md +8 -2
- package/docs/MODEL_METADATA.md +106 -0
- package/docs/MODEL_SELECTION.md +15 -2
- package/docs/MONITORING.md +285 -0
- package/docs/OLLAMA_SIDECAR.md +8 -6
- package/docs/PRODUCTION.md +49 -18
- package/docs/PRODUCT_POSITIONING.md +86 -256
- package/docs/README.md +40 -0
- package/docs/SCOPES_AND_WARMING.md +226 -0
- package/docs/TESTING.md +118 -1
- package/docs/dev/FINDINGS_AND_POSITIONING.md +290 -0
- package/docs/dev/LOCAL_PERFORMANCE_TESTING.md +298 -0
- package/docs/{OLLAMA_SYSTEM_OPTIMIZATION.md → dev/OLLAMA_SYSTEM_OPTIMIZATION.md} +19 -18
- package/native/index.d.ts +31 -0
- package/package.json +13 -10
- /package/docs/{ADR_RESOURCE_AWARE_BACKEND_ROUTING.md → dev/ADR_RESOURCE_AWARE_BACKEND_ROUTING.md} +0 -0
package/LICENSE-APACHE
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
package/LICENSE-MIT
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) FreeLlama contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
|
|
6
|
+
associated documentation files (the "Software"), to deal in the Software without restriction, including
|
|
7
|
+
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
8
|
+
copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
|
|
9
|
+
following conditions:
|
|
10
|
+
|
|
11
|
+
The above copyright notice and this permission notice shall be included in all copies or substantial
|
|
12
|
+
portions of the Software.
|
|
13
|
+
|
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
|
|
15
|
+
LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
|
|
16
|
+
EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
|
17
|
+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
|
|
18
|
+
USE OR OTHER DEALINGS IN THE SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,23 +1,34 @@
|
|
|
1
1
|
# FreeLlama MCP server
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
[MCP](https://modelcontextprotocol.io) tools, built on the official
|
|
5
|
-
[TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk).
|
|
3
|
+

|
|
6
4
|
|
|
7
|
-
|
|
5
|
+
This package exposes FreeLlama's management layer as Model Context Protocol (MCP) tools.
|
|
6
|
+
Agents offload bounded tasks to local Ollama models; FreeLlama handles model selection, resource
|
|
7
|
+
admission, memory checks, and routing to configured CPU/GPU backends. Results include execution
|
|
8
|
+
evidence for the calling agent to inspect.
|
|
8
9
|
|
|
9
|
-
|
|
10
|
+
The agent owns prompts, task dependencies, and answer verification. The operator configures
|
|
11
|
+
Ollama and model lifecycle permissions; Ollama performs inference. See
|
|
12
|
+
[architecture](../../docs/ARCHITECTURE.md) for the full ownership boundary.
|
|
13
|
+
The server uses the official [TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk).
|
|
14
|
+
|
|
15
|
+
## Understand the tools
|
|
16
|
+
|
|
17
|
+
- `doctor`, `models`, `run_task`, `task_jobs`, `run_task_batch` — thin wrappers over the native NAPI bindings into the
|
|
10
18
|
Rust core (`../rust-core/src/napi.rs`); no CLI subprocess, no reimplemented routing logic.
|
|
11
|
-
`run_task { preview: true }` is the
|
|
19
|
+
`run_task { preview: true }` is the decision-only form and does not run inference.
|
|
12
20
|
Preview and execution are separate calls: preview accepts routing fields only and rejects task
|
|
13
21
|
payloads or runtime controls instead of silently ignoring them.
|
|
14
22
|
`models { view: "library" }` queries the public `ollama.com` library (the former `search_models` tool).
|
|
23
|
+
- `scope` — creates, inspects, forks, or deletes bounded process-local message history.
|
|
24
|
+
- `warm_model` — warms an exact installed model through ordinary managed admission and placement.
|
|
15
25
|
- `session` — creates and releases a bounded, idle-expiring model-affinity handle for related
|
|
16
26
|
route/task calls. It does not store messages, prompt history, or Ollama's runner KV cache.
|
|
17
27
|
- `ollama_manage`, `ollama_delete` — Ollama's HTTP API for lifecycle operations the routing layer
|
|
18
28
|
doesn't cover.
|
|
19
29
|
- `delegate_research` — offloads a grounded code-research question to a local model and returns a
|
|
20
|
-
verdict computed from
|
|
30
|
+
verdict computed from model evidence, successful tool calls, and question-shape heuristics.
|
|
31
|
+
The verdict guides verification; it does not certify answer correctness.
|
|
21
32
|
|
|
22
33
|
Use the smallest tool that owns the operation:
|
|
23
34
|
|
|
@@ -27,8 +38,11 @@ flowchart TD
|
|
|
27
38
|
Q -->|"Diagnose the local runtime"| D["doctor"]
|
|
28
39
|
Q -->|"Inspect installed, resident, detailed, raw, or online models"| M["models"]
|
|
29
40
|
Q -->|"Choose or execute one task"| R["run_task"]
|
|
41
|
+
Q -->|"Inspect or cancel deferred work"| J["task_jobs"]
|
|
30
42
|
Q -->|"Fan out independent tasks"| B["run_task_batch"]
|
|
31
43
|
Q -->|"Keep/release related-task model affinity"| S["session"]
|
|
44
|
+
Q -->|"Retain or fork message history"| H["scope"]
|
|
45
|
+
Q -->|"Prepare runner residency"| W["warm_model"]
|
|
32
46
|
Q -->|"Pull or stop a model"| O["ollama_manage"]
|
|
33
47
|
Q -->|"Permanently remove a named model"| X["ollama_delete"]
|
|
34
48
|
Q -->|"Ground an answer in workspace files"| G["delegate_research"]
|
|
@@ -39,15 +53,36 @@ flowchart TD
|
|
|
39
53
|
| `doctor` | Makes runtime, version, memory, and configuration drift visible before work starts. |
|
|
40
54
|
| `models` | Separates installed, resident, detailed, raw, and public-library questions instead of guessing from model names. |
|
|
41
55
|
| `run_task` | Applies the same deterministic routing, confidence, admission, and execution contract as the Rust control plane. |
|
|
56
|
+
| `task_jobs` | Lists, retrieves, cancels, or removes a bounded process-local task submitted with `defer:true`. Removal stops active work first. |
|
|
42
57
|
| `run_task_batch` | Executes only caller-declared independent tasks. Stable IDs, a bounded dispatcher, 3:2:1 fair priority classes, and per-item errors make fan-out inspectable rather than implicit. |
|
|
43
58
|
| `session` | Lets an agent explicitly own the lifetime of a bounded affinity handle without misrepresenting it as conversation or KV storage. |
|
|
59
|
+
| `scope` | Retains bounded message history with expected revisions and explicit history reads. |
|
|
60
|
+
| `warm_model` | Loads an installed runner profile using the same memory, admission, placement, and job controls as task execution. |
|
|
44
61
|
| `ollama_manage` | Exposes additive lifecycle operations that FreeLlama routing intentionally does not perform. |
|
|
45
62
|
| `ollama_delete` | Keeps irreversible deletion separate and explicit so a client can guard it. |
|
|
46
63
|
| `delegate_research` | Gives a local model bounded read-only tools and returns citations plus an independently computed verification verdict. |
|
|
47
64
|
|
|
48
65
|
## Follow the agent workflow
|
|
49
66
|
|
|
50
|
-
The
|
|
67
|
+
The task lifecycle uses the UUID returned as `job.id` by `run_task {defer:true}`. A job is one
|
|
68
|
+
submitted task; pass that UUID as `jobId` for every subsequent operation.
|
|
69
|
+
|
|
70
|
+
| Operation | MCP call |
|
|
71
|
+
|---|---|
|
|
72
|
+
| List models | `models {view:"installed"}` |
|
|
73
|
+
| Submit a task and receive its ID | `run_task {prompt:"Explain this code", defer:true}` |
|
|
74
|
+
| List tasks with their states | `task_jobs {action:"list"}` |
|
|
75
|
+
| Get one task's state, result, or error | `task_jobs {action:"get", jobId:"<returned UUID>"}` |
|
|
76
|
+
| Stop one task and retain its record | `task_jobs {action:"cancel", jobId:"<returned UUID>"}` |
|
|
77
|
+
| Stop one task and remove its record | `task_jobs {action:"remove", jobId:"<returned UUID>"}` |
|
|
78
|
+
|
|
79
|
+
Removal waits for local permits to be released before deleting the record. It returns the ID and
|
|
80
|
+
terminal status; subsequent lookup returns `404`. It does not unload a shared Ollama model or
|
|
81
|
+
prove that physical computation has stopped. Receipts are bounded and lost on server restart;
|
|
82
|
+
see [retention and task states](../../docs/MONITORING.md#read-the-live-status).
|
|
83
|
+
|
|
84
|
+
Follow this workflow. The server's initialization instructions carry a compressed form of the
|
|
85
|
+
same contract (inventory first, preview semantics, evidence levels, ownership, and pull approval):
|
|
51
86
|
|
|
52
87
|
1. Start with `models {view:"installed"}`, then `models {view:"resident"}`. This gives a routing
|
|
53
88
|
agent the smallest current inventory first.
|
|
@@ -89,7 +124,7 @@ Use this economical sequence:
|
|
|
89
124
|
For a delegated agent, estimated usable input is `contextTokens - outputTokens - safetyMarginTokens`
|
|
90
125
|
(the default margin is 256). The adapter pins the system instruction and question, preserves the
|
|
91
126
|
newest observations, compacts older observations into breadcrumbs, and refuses a pinned overflow
|
|
92
|
-
by default. `
|
|
127
|
+
by default. `FREELLAMA_AGENT_PINNED_OVERFLOW=clip` is a deliberate quality-risk override. Initial token counting
|
|
93
128
|
is conservative; successful Ollama calls calibrate later estimates.
|
|
94
129
|
|
|
95
130
|
Do not confuse operational evidence with answer quality. A receipt can prove admission, token
|
|
@@ -137,7 +172,7 @@ placement and admission. Ollama and the OS/driver own runner loading and physica
|
|
|
137
172
|
|
|
138
173
|
```mermaid
|
|
139
174
|
flowchart TD
|
|
140
|
-
C["MCP client"] -->|"stdio JSON-RPC"| B["dist/index.js:
|
|
175
|
+
C["MCP client"] -->|"stdio JSON-RPC"| B["dist/index.js: tool registrations"]
|
|
141
176
|
B --> N["native/index.js and platform addon"]
|
|
142
177
|
N -->|"doctor"| L["Rust diagnostics"]
|
|
143
178
|
N -->|"models installed/resident; run_task"| S["freellama serve"]
|
|
@@ -200,12 +235,34 @@ artifact); `native/index.js` and `native/index.d.ts` are hand-written and checke
|
|
|
200
235
|
|
|
201
236
|
## Run
|
|
202
237
|
|
|
238
|
+
**From npm (recommended):**
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
npx @octocodeai/freellama-mcp-server
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
**From the repository root after building:**
|
|
245
|
+
|
|
203
246
|
```bash
|
|
204
|
-
node dist/index.js
|
|
247
|
+
node packages/mcp/dist/index.js
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
The server speaks MCP over stdio — configure it in your MCP client, do not run it interactively.
|
|
251
|
+
|
|
252
|
+
**Claude Desktop (`claude_desktop_config.json`):**
|
|
253
|
+
|
|
254
|
+
```json
|
|
255
|
+
{
|
|
256
|
+
"mcpServers": {
|
|
257
|
+
"freellama": {
|
|
258
|
+
"command": "npx",
|
|
259
|
+
"args": ["@octocodeai/freellama-mcp-server"]
|
|
260
|
+
}
|
|
261
|
+
}
|
|
262
|
+
}
|
|
205
263
|
```
|
|
206
264
|
|
|
207
|
-
|
|
208
|
-
`claude_desktop_config.json`), not interactively.
|
|
265
|
+
**Cursor / other clients** — use the same `command`/`args` shape in their MCP server config.
|
|
209
266
|
|
|
210
267
|
## Test
|
|
211
268
|
|
|
@@ -214,28 +271,47 @@ Vitest, in three tiers (all TypeScript, under `test/`):
|
|
|
214
271
|
| Tier | Command (from this package) | Checks | Needs |
|
|
215
272
|
|---|---|---|---|
|
|
216
273
|
| `test/unit/` | `yarn test` (watch: `yarn test:watch`) | pure functions straight from `src/*.ts` — no build step, this is the TDD loop | nothing, ~0.5s |
|
|
217
|
-
| `test/integration/` | `yarn test:integration` | the built server over the real MCP protocol: tool contract, structured content, guardrails; rebuilds `dist/` itself |
|
|
274
|
+
| `test/integration/` | `yarn test:integration` | the built server over the real MCP protocol: tool contract, structured content, guardrails; rebuilds `dist/` itself | Ollama at `FREELLAMA_OLLAMA_ENDPOINT` (default :11434) |
|
|
218
275
|
| `test/e2e/` | `yarn test:e2e` | every tool against the live system: real routing/execution, pull → delete round trip (net-zero), a real delegated research run | serve + Ollama + models, ~40s |
|
|
219
276
|
|
|
220
277
|
The same commands work from the repository root (`yarn test` there also runs the CLI package's tests).
|
|
221
278
|
The integration/e2e tiers fail fast with a readable message when Ollama is down, and the lifecycle
|
|
222
279
|
test refuses to delete a model already installed on the test system.
|
|
223
280
|
|
|
281
|
+
For default-endpoint autostart, a spawn failure, early exit, or readiness timeout rejects the
|
|
282
|
+
request while leaving the MCP transport available for diagnosis or retry. Autostart does not apply
|
|
283
|
+
to an explicit non-default endpoint; see `FREELLAMA_MCP_AUTOSTART_SERVE` below.
|
|
284
|
+
|
|
224
285
|
## Tools
|
|
225
286
|
|
|
226
287
|
| Tool | Needs `freellama serve`? | What it does |
|
|
227
288
|
|---|---|---|
|
|
228
|
-
| `doctor` | No | Runtime/configuration diagnostic. `summary` (default) returns endpoint, version, resident count, host/profile signals, and next step; `scheduler` adds configured/snapshot admission data; `config` returns categorized settings; `full` adds all non-duplicated diagnostics. The report distinguishes an observed same-user macOS process from configuration hints and never claims remote process visibility. |
|
|
229
|
-
| `models` | `installed` (default) and `resident` do; `detail`/`raw`/`library` don't | Views: `installed` (capabilities, derived `model_type`, VRAM, context, policy_rank), `resident` (loaded now + managed GPU/CPU split), `detail` (one model,
|
|
289
|
+
| `doctor` | No | Runtime/configuration diagnostic. `summary` (default) returns endpoint, version, resident count, host/profile signals, and next step; `scheduler` adds configured/snapshot admission data; `config` returns categorized settings; `full` adds all non-duplicated diagnostics; `status` returns serve's live view (per-backend queues, current and adaptive limits, circuit breakers, loaded models, host memory, Ollama's effective settings, today's usage); `usage` returns task and token totals per day and model (`days`, default 7). The report distinguishes an observed same-user macOS process from configuration hints and never claims remote process visibility. |
|
|
290
|
+
| `models` | `installed` (default) and `resident` do; `detail`/`raw`/`library` don't | Views: `installed` (capabilities, use explanations, derived `model_type`, VRAM, context, policy_rank), `resident` (loaded now + managed GPU/CPU split), `detail` (one model, maximum context), paged `raw` (`GET /api/tags`), `library` (two-step ollama.com search: omit `model` for families, pass `model:"<family>"` for all tags/`fitsInMemory`). `includeLibrary:true` enriches installed/detail views with sourced public guidance; `includeReadme:true` returns bounded README text in library step 2 or enriched detail. |
|
|
230
291
|
| `run_task` | Yes | **Routes or executes** a chat/generate/embed call. `preview: true` accepts routing fields only and returns a decision without generation. Execution omits preview and supplies the payload. Embedding results withhold raw vectors by default (`returnEmbeddings: true` to get them) |
|
|
292
|
+
| `scope` | Yes | Creates, inspects, forks, or deletes process-local message history; history inclusion requires `includeMessages:true`. |
|
|
293
|
+
| `warm_model` | Yes | Warms an exact installed model through managed admission; accepts residency/profile controls and optional `defer:true`. |
|
|
294
|
+
| `session` | Yes | Creates or releases model affinity without storing messages or KV. |
|
|
295
|
+
| `task_jobs` | Yes | Inspects, cancels, or removes a deferred task or warming operation. |
|
|
231
296
|
| `run_task_batch` | Yes | Executes only typed `{id, independent:true, task}` items. The nested `task` exposes the same task, routing, payload, and runtime controls as `run_task`; `maxParallelism` bounds fan-out. It is not a dependency graph scheduler. |
|
|
232
297
|
| `ollama_manage` | No (direct to Ollama) | `action: "pull"` downloads a model; `action: "stop"` force-unloads it. Both additive and idempotent |
|
|
233
298
|
| `ollama_delete` | No (direct to Ollama) | **Destructive**: permanently removes a model. Call only on an explicit human instruction naming the exact model |
|
|
234
299
|
| `delegate_research` | Yes (and spawns the adapter) | Answers a grounded question from files under `workspacePath`; managed coding turns return placement receipts, citations, and an independent verification verdict |
|
|
235
300
|
|
|
301
|
+
### Inspect capabilities and intended uses
|
|
302
|
+
|
|
303
|
+
Use `models {view:"installed", includeLibrary:true}` to enrich each installed model, or
|
|
304
|
+
`models {view:"detail", model:"gemma3:1b", includeLibrary:true, includeReadme:true}` for one model.
|
|
305
|
+
Website access is optional for installed/detail views. Public guidance requires a matching exact tag
|
|
306
|
+
and digest prefix; local capabilities remain authoritative and benchmark/policy evidence stays separate.
|
|
307
|
+
Family claims have source excerpts and fetch dates, and lookup failures retain local inventory.
|
|
308
|
+
Search reports `cloudAvailable` and `cloudOnly:null`; inspect exact tags to establish local availability.
|
|
309
|
+
See [Ollama model metadata](../../docs/MODEL_METADATA.md) for response fields, lookup states, cache bounds,
|
|
310
|
+
custom models, and variant limitations.
|
|
311
|
+
|
|
236
312
|
### Preserve Ollama request controls
|
|
237
313
|
|
|
238
|
-
|
|
314
|
+
The tools declare an MCP object `outputSchema`. Schemas type the stable fields FreeLlama owns
|
|
239
315
|
(such as route decision, page, answer, and session identifiers) while leaving Ollama-owned nested
|
|
240
316
|
payloads forward-compatible. `structuredContent` is canonical; normal text is a concise cue. For
|
|
241
317
|
the narrow `delegate_research` legacy case, set `legacyText:true` to receive serialized JSON text.
|
|
@@ -245,16 +321,29 @@ refuses continuation if the live model list changed. Library tag responses use t
|
|
|
245
321
|
shape. Do not request `doctor {view:"full"}` in an agent loop: use `summary`, `scheduler`, or
|
|
246
322
|
`config` for the smallest diagnostic that answers the question.
|
|
247
323
|
|
|
324
|
+
### Preview and execution
|
|
325
|
+
|
|
326
|
+
For ordinary chat, `run_task {prompt:"Hello"}` is enough once the server and an eligible model
|
|
327
|
+
are available. `task` defaults to `completion`, including inside batch items. Task profiles are
|
|
328
|
+
optional routing presets, not restrictions on what you can ask the model.
|
|
329
|
+
|
|
330
|
+
Default `balanced` routing prefers policy-qualified candidates when available. Without an eligible
|
|
331
|
+
policy candidate, it falls back to capability/context-compatible models and reports low confidence,
|
|
332
|
+
`quality_evidence:"none"`, and `balanced_without_quality_policy`. Set `minConfidence:"medium"`
|
|
333
|
+
to require evidence, or `objective:"quality"` to require policy-based selection unless you pin a model.
|
|
334
|
+
Memory admission, authentication, and capability requirements apply in every mode.
|
|
335
|
+
|
|
248
336
|
`run_task` has two deliberately exclusive request shapes:
|
|
249
337
|
|
|
250
338
|
- Preview: set `preview: true`; pass routing fields such as `task`, `objective`, `model`,
|
|
251
339
|
`contextTokens`, `requiredCapabilities`, and placement/confidence gates. To preview tool use,
|
|
252
340
|
pass `requiredCapabilities: ["tools"]` rather than function definitions.
|
|
253
|
-
- Execution: omit `preview` (or set it to `false`) and pass `prompt
|
|
341
|
+
- Execution: omit `preview` (or set it to `false`) and pass `prompt`, `systemPrompt`, or `messages` for generative
|
|
254
342
|
tasks or `input` for embeddings, plus any applicable Ollama runtime controls.
|
|
255
343
|
|
|
256
|
-
Preview rejects `prompt`, `messages`, `input`, `images`, `tools`, `keepAlive`, `format`, `think`,
|
|
257
|
-
`options`, `logprobs`, `topLogprobs`,
|
|
344
|
+
Preview rejects `prompt`, `systemPrompt`, `messages`, `input`, `images`, `tools`, `keepAlive`, `format`, `think`,
|
|
345
|
+
`options`, `logprobs`, `topLogprobs`, `returnEmbeddings`, `priority`, `maxWaitSeconds`, `timeoutSeconds`, and
|
|
346
|
+
`defer`, `scopeId`, and `scopeRevision`. This prevents a client from attaching
|
|
258
347
|
work to a decision-only call and mistakenly assuming it ran.
|
|
259
348
|
|
|
260
349
|
`run_task.messages` preserves Ollama message fields beyond `role` and `content`, including images,
|
|
@@ -263,11 +352,54 @@ thinking, tool calls, and tool names. Managed requests also accept `format`, `th
|
|
|
263
352
|
keys are rejected inside `options`. The raw proxy remains available when a caller needs an Ollama
|
|
264
353
|
endpoint or feature that the managed MCP tool does not expose, including streaming.
|
|
265
354
|
|
|
355
|
+
Managed execution waits for a slot and safe host capacity within the server's waiting budget
|
|
356
|
+
(120 seconds by default). Set `maxWaitSeconds` on `run_task` or each batch task to request a
|
|
357
|
+
shorter budget; the server caps it at its configured limit. Resource waiters remain counted in
|
|
358
|
+
the bounded queue after returning their execution slots. They resume when capacity becomes
|
|
359
|
+
available, and session cancellation removes pending work. A full queue or an expired deadline
|
|
360
|
+
returns a capacity error with retry guidance.
|
|
361
|
+
|
|
362
|
+
Set `defer:true` on `run_task` to return a job ID immediately. Use `task_jobs {action:"list"}` for
|
|
363
|
+
metadata, `{action:"get", jobId}` for a retained result, and `{action:"cancel", jobId}` to cancel
|
|
364
|
+
that task and wait for local permits to be released. `get` summarizes embedding vectors by default;
|
|
365
|
+
set `returnEmbeddings:true` to include them. Cancellation does not unload a shared model or confirm
|
|
366
|
+
physical runner shutdown. Jobs expire on restart; see [CLI controls](../../docs/CLI.md#execute-tasks)
|
|
367
|
+
for input, result, registry, and retention limits. Batch items cannot defer.
|
|
368
|
+
|
|
369
|
+
Set `timeoutSeconds` on a task or batch item for a total deadline covering discovery, queueing,
|
|
370
|
+
loading, and inference. The server caps it at its task timeout. The admission wait budget remains
|
|
371
|
+
separate; a queue refusal can happen before the total deadline. `priority` selects a fair scheduling
|
|
372
|
+
class without bypassing resource checks.
|
|
373
|
+
|
|
374
|
+
You own the task instructions. Pass system prompts as `messages` entries with `role:"system"`,
|
|
375
|
+
or use `systemPrompt` to prepend one caller-supplied system message. FreeLlama preserves existing
|
|
376
|
+
messages and adds no task instructions of its own. A plain `prompt` becomes one user message,
|
|
377
|
+
and a supplied `messages` array takes precedence over `prompt`. This also applies to
|
|
378
|
+
`task:"code_review"` (an alias for `coding`) and `run_task_batch`: no review wrapper, mandatory
|
|
379
|
+
JSON review format, or review-specific output-token minimum is added.
|
|
380
|
+
|
|
381
|
+
The separate `delegate_research` adapter and natural-language routing interpreter retain internal
|
|
382
|
+
protocol prompts for their own bounded jobs. Those prompts are not injected into `run_task`
|
|
383
|
+
conversations. Ollama can still apply the selected model's own template or Modelfile defaults.
|
|
384
|
+
|
|
266
385
|
`model_type` is a display-oriented value derived from Ollama's additive capabilities:
|
|
267
386
|
`generative`, `multimodal`, `embedding_only`, or `unknown`. Routing continues to use the original
|
|
268
387
|
capability set, not this summary label. Unknown future Ollama capabilities are omitted from the
|
|
269
388
|
typed routing set instead of being treated as a known capability.
|
|
270
389
|
|
|
390
|
+
### Retain history and warm models
|
|
391
|
+
|
|
392
|
+
Use `scope {action:"create",messages}` to retain a caller-owned prefix. Pass its returned ID and
|
|
393
|
+
revision as `scopeId` and `scopeRevision` with only the new input to `run_task`; successful execution
|
|
394
|
+
returns the next scope revision. Fork a snapshot before parallel branches. `sessionId` continues to
|
|
395
|
+
mean model affinity only.
|
|
396
|
+
|
|
397
|
+
Use `warm_model {model,contextTokens,keepAlive?}` for a matching runner profile. It accepts no task
|
|
398
|
+
payload and does not pull a model. Add `defer:true` to inspect its progress through `task_jobs`.
|
|
399
|
+
Explicit `keepAlive` wins over finite adaptive retention. See the canonical
|
|
400
|
+
[Scope history and model warming reference](../../docs/SCOPES_AND_WARMING.md) for schemas, limits,
|
|
401
|
+
concurrency, privacy, and cache boundaries.
|
|
402
|
+
|
|
271
403
|
### Use CPU assignments through MCP
|
|
272
404
|
|
|
273
405
|
`run_task` delegates to `freellama serve`, so it honors the server's `--cpu-upstream` and
|
|
@@ -277,14 +409,27 @@ managed tasks execute on the primary GPU-capable process.
|
|
|
277
409
|
Set `executionPreference` on `run_task` to `auto` (default), `prefer_cpu`, or `prefer_gpu`. This is a
|
|
278
410
|
guarded hint: only exact operator-assigned CPU models are eligible, explicit model/session pins win,
|
|
279
411
|
and `preview: true` returns `execution.preference_satisfied`, placement, upstream, admission, and a
|
|
280
|
-
reason without generating. Automatic feedback normalizes work by tokens
|
|
281
|
-
warm
|
|
282
|
-
|
|
412
|
+
reason without generating. Automatic feedback normalizes work by tokens and requires three
|
|
413
|
+
comparable warm serial samples on both backends with a 10% advantage. It matches model digest,
|
|
414
|
+
explicit observed context, effective controls, endpoint-attributed process settings, and managed
|
|
415
|
+
admission class. Unknown identity, parallel intervals, and decision-only payload-free previews
|
|
416
|
+
use capacity and policy signals. Stored legacy totals do not qualify speed hints. Feedback does
|
|
417
|
+
not steer the `quality` objective. See [automatic feedback](../../docs/CPU_GPU_ROUTING.md#understand-automatic-feedback).
|
|
418
|
+
`minPlacementEvidence:"observed"` fails closed unless resident
|
|
283
419
|
`/api/ps` evidence matches the configured processor; warm cold models once with `"configured"`.
|
|
284
420
|
`keepAlive:"0"` uses an observe-then-unload transaction: FreeLlama retains the runner long enough
|
|
285
421
|
to inspect `/api/ps`, accepts feedback only for verified placement, requests an explicit unload,
|
|
286
422
|
then reports the unload verification in `execution.lifecycle`.
|
|
287
423
|
|
|
424
|
+
Preview `execution.agent_plan` separates `queue_readiness`, `resource_readiness`, and combined
|
|
425
|
+
`dispatch_readiness`. Inspect `execution.resource_assessment` for required bytes and missing telemetry;
|
|
426
|
+
the preview shares execution's footprint estimate but never reserves memory. Local inference requires
|
|
427
|
+
RAM telemetry by default; configure an explicit alternative on `serve`, not in a model prompt.
|
|
428
|
+
Resource errors retain a human-readable `error` plus machine-readable `code` and `resource_admission`
|
|
429
|
+
fields in the HTTP response. MCP preserves JSON refusals in both error text and `structuredContent`,
|
|
430
|
+
including queue-full codes, `retry_after_seconds`, resource assessments, and unload receipts.
|
|
431
|
+
Agents can use those fields to back off or explain a capacity hold without parsing human prose.
|
|
432
|
+
|
|
288
433
|
`models {view: "resident"}` uses the managed catalog, so it includes resident models from both
|
|
289
434
|
Ollama processes and labels explicitly assigned CPU models instead of querying only the primary
|
|
290
435
|
server.
|
|
@@ -302,14 +447,14 @@ inspect or modify that separate server. Read
|
|
|
302
447
|
|
|
303
448
|
## Configuration
|
|
304
449
|
|
|
305
|
-
|
|
450
|
+
The following settings have environment-variable overrides. Set them in the MCP client's
|
|
306
451
|
server-launch config (for example `.mcp.json`'s `env` block) or the launching shell.
|
|
307
452
|
|
|
308
453
|
| Variable | Default | Affects |
|
|
309
454
|
|---|---|---|
|
|
310
455
|
| `FREELLAMA_OLLAMA_ENDPOINT` | `http://127.0.0.1:11434` | `doctor` and all `ollama_*` tools' default Ollama endpoint |
|
|
311
456
|
| `FREELLAMA_SERVE_ENDPOINT` | `http://127.0.0.1:11435` | Default `endpoint` for serve-backed tools and `delegate_research` |
|
|
312
|
-
| `FREELLAMA_MCP_DEFAULT_MODEL` |
|
|
457
|
+
| `FREELLAMA_MCP_DEFAULT_MODEL` | unset (routes to an installed coding model) | `delegate_research`'s default `model` |
|
|
313
458
|
| `FREELLAMA_MCP_DEFAULT_ADAPTER` | `bash` | `delegate_research`'s adapter (`octocode` to switch) |
|
|
314
459
|
| `FREELLAMA_MCP_MAX_TURNS` | `8` | Max agent turns for `delegate_research` |
|
|
315
460
|
| `FREELLAMA_MCP_DELEGATE_TIMEOUT_SECONDS` | `180` | Whole-subprocess timeout for `delegate_research` |
|
|
@@ -324,9 +469,25 @@ server-launch config (for example `.mcp.json`'s `env` block) or the launching sh
|
|
|
324
469
|
| `FREELLAMA_TASK_TIMEOUT_SECONDS` | `900` | Generation calls; read by `../rust-core/src/napi.rs` |
|
|
325
470
|
| `FREELLAMA_AUTH_TOKEN_FILE` | unset | Bearer-token file used by the native client and research adapters for all serve routes |
|
|
326
471
|
| `FREELLAMA_AGENT_TOKEN_CALIBRATION_DIR` | platform data directory | Prompt-free, model-specific token-estimator calibration shared across adapter processes |
|
|
472
|
+
| `FREELLAMA_MCP_AUTOSTART_SERVE` | `1` | When the default loopback serve endpoint does not answer, start `freellama serve` as a child of the MCP server (stopped with it). `0` disables |
|
|
473
|
+
| `FREELLAMA_SERVE_BINARY` | checkout `target/release`/`debug`, then the platform package | Binary used for serve autostart |
|
|
474
|
+
| `FREELLAMA_AGENT_OS_SANDBOX` | `auto` | Bash adapter OS confinement: `bwrap` on Linux, `sandbox-exec` on macOS, when present. `off` keeps only the restricted shell |
|
|
475
|
+
| `FREELLAMA_AGENT_OCTOCODE_PACKAGE` | `octocode@19.1.0` | Pinned npm spec the Octocode adapter runs through `npx` |
|
|
476
|
+
|
|
477
|
+
Serve-side placement settings (read by `freellama serve`, not the MCP process):
|
|
478
|
+
|
|
479
|
+
| Variable | Default | Affects |
|
|
480
|
+
|---|---|---|
|
|
481
|
+
| `FREELLAMA_EVICT_IDLE_MODELS` | on | Unload idle, unpinned models using reuse and load costs when a cold load needs room. `0` turns eviction off |
|
|
482
|
+
| `FREELLAMA_CPU_NUM_THREAD` | half the logical cores (not set on macOS) | `num_thread` for CPU-placed tasks only; GPU tasks leave it to Ollama |
|
|
483
|
+
| `FREELLAMA_MAX_QUEUE_WAIT_SECONDS` | `120` | How long a managed task queues for a slot or memory |
|
|
484
|
+
| `OLLAMA_NUM_PARALLEL`, `OLLAMA_KV_CACHE_TYPE`, `OLLAMA_FLASH_ATTENTION` | Ollama's | Read as hints for the memory estimate: parallel slots multiply the KV cache, and `q8_0`/`q4_0` shrink it only with flash attention. Set them for serve the same way as for Ollama |
|
|
327
485
|
|
|
328
486
|
Research-adapter settings are deployment defaults. The optional `delegate_research.agent` object
|
|
329
|
-
|
|
487
|
+
overrides the per-call budget (`maxTurns`, `contextTokens`, `outputTokens`, `temperature`, `seed`,
|
|
488
|
+
`think`, `keepAlive`, `requestTimeoutSeconds`, `toolTimeoutSeconds`); explicit per-call fields win.
|
|
489
|
+
The research subprocess receives only `FREELLAMA_*`, locale, temp, and proxy/CA variables from the
|
|
490
|
+
server environment.
|
|
330
491
|
|
|
331
492
|
| Variable | Default | Affects |
|
|
332
493
|
|---|---:|---|
|
|
@@ -360,14 +521,22 @@ between model templates.
|
|
|
360
521
|
**Security:** `delegate_research` grants a local model read access to `workspacePath`. The path is
|
|
361
522
|
confined to `FREELLAMA_MCP_ALLOWED_ROOTS` (default: this repository in a checkout; unset in a published
|
|
362
523
|
install until you set it), resolved through symlinks so a link inside an allowed root can't escape
|
|
363
|
-
it. The default `bash` adapter
|
|
364
|
-
|
|
524
|
+
it. The default `bash` adapter runs an allowlist of read-only tools in a restricted shell with a
|
|
525
|
+
scrubbed environment and, where available, an OS sandbox; it also rejects command substitution,
|
|
526
|
+
home-directory paths, `..`, and absolute paths outside that workspace. Production deployments should generate a permission-restricted token with
|
|
365
527
|
`freellama auth-token`, start `serve` with `--auth-token-file`, and set
|
|
366
528
|
`FREELLAMA_AUTH_TOKEN_FILE` for MCP. Authentication covers managed and raw passthrough routes; use
|
|
367
529
|
an external TLS and authorization layer for untrusted or multi-tenant networks.
|
|
368
530
|
|
|
369
531
|
### Runtime evidence
|
|
370
532
|
|
|
533
|
+
Admission weights can be set by task kind in the server runtime file's `[task_costs]` table.
|
|
534
|
+
Inspect `doctor {view:"full"}` at `platform_health.admission.costs.base_by_task` for effective
|
|
535
|
+
weights and `run_task {preview:true}` for current cost and capacity advice. The table changes
|
|
536
|
+
managed admission; it does not create Ollama
|
|
537
|
+
slots or relax memory guards. Existing queued work keeps its captured cost. See
|
|
538
|
+
[monitoring controls](../../docs/MONITORING.md#adaptive-concurrency).
|
|
539
|
+
|
|
371
540
|
**No measurements are compiled in.** In a repository checkout, per-model research grades load from
|
|
372
541
|
`benchmark/evidence/model-evidence.json`. A published package does not include that repository
|
|
373
542
|
file, so its evidence table is empty unless you set `FREELLAMA_MCP_MODEL_EVIDENCE`. An unmeasured
|
|
@@ -400,17 +569,18 @@ See `skills/freellama/references/proxy-vs-serve.md` for the `proxy` vs `serve` d
|
|
|
400
569
|
|
|
401
570
|
`packages/mcp/` is the portable JavaScript package. Its `package.json` lists the platform packages
|
|
402
571
|
as exact-version optional dependencies; it intentionally does not embed a `.node` binary. Publish
|
|
403
|
-
the eight `packages/native/*` packages first, then publish `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server`
|
|
404
|
-
at the same version. `yarn release:verify:publish` refuses a release if any
|
|
405
|
-
missing, empty, or unpacked
|
|
572
|
+
the eight `packages/native/*` packages first, then publish `@octocodeai/freellama` (`npx @octocodeai/freellama`) and `@octocodeai/freellama-mcp-server` (`npx @octocodeai/freellama-mcp-server`)
|
|
573
|
+
at the same version. `yarn release:verify:publish` refuses a release if any of the ten packages
|
|
574
|
+
omits `assets/logo.jpg`, or if any platform package has a missing, empty, or unpacked
|
|
575
|
+
executable/addon pair.
|
|
406
576
|
|
|
407
577
|
```bash
|
|
408
578
|
cd packages/mcp
|
|
409
579
|
npm pack --dry-run # confirm exactly what a publish would ship
|
|
410
580
|
```
|
|
411
581
|
|
|
412
|
-
|
|
413
|
-
|
|
582
|
+
Treat `npm publish` as a public release action. Follow the repository
|
|
583
|
+
[release procedure](../../RELEASE.md) and obtain the repository owner's release authorization first.
|
|
414
584
|
|
|
415
585
|
## Why native bindings
|
|
416
586
|
|