@octocodeai/freellama 0.5.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE-MIT +1 -1
- package/README.md +10 -15
- package/package.json +10 -9
package/LICENSE-MIT
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
MIT License
|
|
2
2
|
|
|
3
|
-
Copyright (c)
|
|
3
|
+
Copyright (c) 2026 Guy Bary
|
|
4
4
|
|
|
5
5
|
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
|
|
6
6
|
associated documentation files (the "Software"), to deal in the Software without restriction, including
|
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@ submit managed tasks. Ollama supplies model loading and inference.
|
|
|
9
9
|
The JavaScript launcher locates the
|
|
10
10
|
compiled Rust CLI in the matching optional native package and forwards arguments, signals, and exit
|
|
11
11
|
status. In a source checkout, it uses `target/release/freellama` after a build. Routing and execution
|
|
12
|
-
logic remain in [`freellama-core`](
|
|
12
|
+
logic remain in [`freellama-core`](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/README.md).
|
|
13
13
|
|
|
14
14
|
```mermaid
|
|
15
15
|
flowchart LR
|
|
@@ -27,7 +27,6 @@ flowchart LR
|
|
|
27
27
|
npx @octocodeai/freellama doctor
|
|
28
28
|
npx @octocodeai/freellama auth-token --out ~/.local/share/freellama/auth.token
|
|
29
29
|
npx @octocodeai/freellama serve \
|
|
30
|
-
--recommendation-catalog recommendations.example.toml \
|
|
31
30
|
--auth-token-file ~/.local/share/freellama/auth.token \
|
|
32
31
|
--feedback-file ~/.local/share/freellama/feedback.json
|
|
33
32
|
```
|
|
@@ -44,9 +43,9 @@ npx @octocodeai/freellama route --task coding --objective fastest
|
|
|
44
43
|
npx @octocodeai/freellama task --task completion --objective fastest "Reply with exactly OK."
|
|
45
44
|
```
|
|
46
45
|
|
|
47
|
-
Read the repository [CLI reference](
|
|
46
|
+
Read the repository [CLI reference](https://github.com/bgauryy/FreeLlama/blob/main/docs/CLI.md) for the complete command map, task kinds, routing
|
|
48
47
|
confidence, policy generation, and admission controls. Use the
|
|
49
|
-
[Scope history and model warming reference](
|
|
48
|
+
[Scope history and model warming reference](https://github.com/bgauryy/FreeLlama/blob/main/docs/SCOPES_AND_WARMING.md) for `scope`, `warm`,
|
|
50
49
|
and the scoped task flags.
|
|
51
50
|
|
|
52
51
|
## Run separate CPU and GPU backends
|
|
@@ -61,12 +60,12 @@ npx @octocodeai/freellama serve \
|
|
|
61
60
|
```
|
|
62
61
|
|
|
63
62
|
Managed tasks for assigned models use the CPU backend; other managed models and raw Ollama
|
|
64
|
-
passthrough use the primary backend. See [CPU and GPU model routing](
|
|
63
|
+
passthrough use the primary backend. See [CPU and GPU model routing](https://github.com/bgauryy/FreeLlama/blob/main/docs/CPU_GPU_ROUTING.md)
|
|
65
64
|
for the process setup, verification steps, and measured concurrency result.
|
|
66
65
|
|
|
67
66
|
The primary admission pool derives its default weighted budget from observed Ollama parallelism;
|
|
68
67
|
the independent CPU pool defaults to one unit. Inspect the effective values with `config`. Override
|
|
69
|
-
them with `--max-concurrent-tasks` and `--cpu-max-concurrent-tasks`. `route
|
|
68
|
+
them with `--max-concurrent-tasks` and `--cpu-max-concurrent-tasks`. `route` and
|
|
70
69
|
`task` also accept `--execution-preference auto|prefer-cpu|prefer-gpu`; this only chooses among
|
|
71
70
|
models already eligible on the requested backend and reports any fallback in the execution receipt.
|
|
72
71
|
`--min-placement-evidence observed` fails closed unless resident `/api/ps` evidence matches; use
|
|
@@ -76,13 +75,13 @@ The health endpoint advertises backend, guarded-preference, runtime-feedback, an
|
|
|
76
75
|
persistence contracts, plus per-backend admission capacity. Comparable speed samples require matching
|
|
77
76
|
model digest, observed context, controls, process settings, and serial admission identity. A successful
|
|
78
77
|
parallel request can report verified placement without qualifying a speed sample. See
|
|
79
|
-
[feedback and adaptive profiles](
|
|
78
|
+
[feedback and adaptive profiles](https://github.com/bgauryy/FreeLlama/blob/main/docs/MONITORING.md#adaptive-concurrency) for sample eligibility.
|
|
80
79
|
Treat a missing contract as a stale running binary, then rebuild and restart before testing placement.
|
|
81
80
|
|
|
82
81
|
Authentication covers both control routes and raw Ollama passthrough. A nonloopback listener also
|
|
83
82
|
requires `--allow-remote`; terminate TLS and add tenant authorization outside FreeLlama. The CLI
|
|
84
83
|
persists bounded aggregate feedback by default. Use `--ephemeral-feedback` only for disposable
|
|
85
|
-
runs. See the [production runbook](
|
|
84
|
+
runs. See the [production runbook](https://github.com/bgauryy/FreeLlama/blob/main/docs/PRODUCTION.md).
|
|
86
85
|
|
|
87
86
|
## Build the launcher and binary
|
|
88
87
|
|
|
@@ -118,13 +117,9 @@ errors, and the files included in the package. Core CLI contracts live in
|
|
|
118
117
|
|
|
119
118
|
## Understand CLI and MCP differences
|
|
120
119
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
The output is contract-tested against the MCP server source. `delegate_research` and the online
|
|
126
|
-
model-library view are MCP-only. Control-plane startup, proxy startup, recommendation,
|
|
127
|
-
benchmarks, policy generation, and frozen-suite comparison are CLI-only.
|
|
120
|
+
`delegate_research` and the online model-library view are MCP-only. Control-plane startup, proxy
|
|
121
|
+
startup, auth-token creation, and benchmarks are CLI-only. Generate a routing policy with
|
|
122
|
+
`freellama policy from-eval` (see [CLI reference](https://github.com/bgauryy/FreeLlama/blob/main/docs/CLI.md#earn-medium-routing-confidence)).
|
|
128
123
|
|
|
129
124
|
Use the surfaces together: the CLI starts and verifies the local services; MCP lets an agent inspect
|
|
130
125
|
that state and submit bounded work. An installed MCP client can read `freellama://docs/index` and
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@octocodeai/freellama",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.6.1",
|
|
4
4
|
"description": "Local Ollama model gateway, router, and benchmarker — CLI. Wraps the compiled Rust binary from packages/rust-core.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -33,16 +33,17 @@
|
|
|
33
33
|
"test": "vitest run",
|
|
34
34
|
"test:watch": "vitest"
|
|
35
35
|
},
|
|
36
|
+
"author": "Guy Bary",
|
|
36
37
|
"license": "Apache-2.0 OR MIT",
|
|
37
38
|
"optionalDependencies": {
|
|
38
|
-
"@octocodeai/freellama-native-darwin-arm64": "0.
|
|
39
|
-
"@octocodeai/freellama-native-darwin-x64": "0.
|
|
40
|
-
"@octocodeai/freellama-native-linux-arm64-gnu": "0.
|
|
41
|
-
"@octocodeai/freellama-native-linux-arm64-musl": "0.
|
|
42
|
-
"@octocodeai/freellama-native-linux-x64-gnu": "0.
|
|
43
|
-
"@octocodeai/freellama-native-linux-x64-musl": "0.
|
|
44
|
-
"@octocodeai/freellama-native-win32-arm64-msvc": "0.
|
|
45
|
-
"@octocodeai/freellama-native-win32-x64-msvc": "0.
|
|
39
|
+
"@octocodeai/freellama-native-darwin-arm64": "0.6.1",
|
|
40
|
+
"@octocodeai/freellama-native-darwin-x64": "0.6.1",
|
|
41
|
+
"@octocodeai/freellama-native-linux-arm64-gnu": "0.6.1",
|
|
42
|
+
"@octocodeai/freellama-native-linux-arm64-musl": "0.6.1",
|
|
43
|
+
"@octocodeai/freellama-native-linux-x64-gnu": "0.6.1",
|
|
44
|
+
"@octocodeai/freellama-native-linux-x64-musl": "0.6.1",
|
|
45
|
+
"@octocodeai/freellama-native-win32-arm64-msvc": "0.6.1",
|
|
46
|
+
"@octocodeai/freellama-native-win32-x64-msvc": "0.6.1"
|
|
46
47
|
},
|
|
47
48
|
"publishConfig": {
|
|
48
49
|
"access": "public",
|