@octocodeai/freellama 0.6.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE-MIT +1 -1
- package/README.md +7 -7
- package/package.json +10 -9
package/LICENSE-MIT
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
MIT License
|
|
2
2
|
|
|
3
|
-
Copyright (c)
|
|
3
|
+
Copyright (c) 2026 Guy Bary
|
|
4
4
|
|
|
5
5
|
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
|
|
6
6
|
associated documentation files (the "Software"), to deal in the Software without restriction, including
|
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@ submit managed tasks. Ollama supplies model loading and inference.
|
|
|
9
9
|
The JavaScript launcher locates the
|
|
10
10
|
compiled Rust CLI in the matching optional native package and forwards arguments, signals, and exit
|
|
11
11
|
status. In a source checkout, it uses `target/release/freellama` after a build. Routing and execution
|
|
12
|
-
logic remain in [`freellama-core`](
|
|
12
|
+
logic remain in [`freellama-core`](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/README.md).
|
|
13
13
|
|
|
14
14
|
```mermaid
|
|
15
15
|
flowchart LR
|
|
@@ -43,9 +43,9 @@ npx @octocodeai/freellama route --task coding --objective fastest
|
|
|
43
43
|
npx @octocodeai/freellama task --task completion --objective fastest "Reply with exactly OK."
|
|
44
44
|
```
|
|
45
45
|
|
|
46
|
-
Read the repository [CLI reference](
|
|
46
|
+
Read the repository [CLI reference](https://github.com/bgauryy/FreeLlama/blob/main/docs/CLI.md) for the complete command map, task kinds, routing
|
|
47
47
|
confidence, policy generation, and admission controls. Use the
|
|
48
|
-
[Scope history and model warming reference](
|
|
48
|
+
[Scope history and model warming reference](https://github.com/bgauryy/FreeLlama/blob/main/docs/SCOPES_AND_WARMING.md) for `scope`, `warm`,
|
|
49
49
|
and the scoped task flags.
|
|
50
50
|
|
|
51
51
|
## Run separate CPU and GPU backends
|
|
@@ -60,7 +60,7 @@ npx @octocodeai/freellama serve \
|
|
|
60
60
|
```
|
|
61
61
|
|
|
62
62
|
Managed tasks for assigned models use the CPU backend; other managed models and raw Ollama
|
|
63
|
-
passthrough use the primary backend. See [CPU and GPU model routing](
|
|
63
|
+
passthrough use the primary backend. See [CPU and GPU model routing](https://github.com/bgauryy/FreeLlama/blob/main/docs/CPU_GPU_ROUTING.md)
|
|
64
64
|
for the process setup, verification steps, and measured concurrency result.
|
|
65
65
|
|
|
66
66
|
The primary admission pool derives its default weighted budget from observed Ollama parallelism;
|
|
@@ -75,13 +75,13 @@ The health endpoint advertises backend, guarded-preference, runtime-feedback, an
|
|
|
75
75
|
persistence contracts, plus per-backend admission capacity. Comparable speed samples require matching
|
|
76
76
|
model digest, observed context, controls, process settings, and serial admission identity. A successful
|
|
77
77
|
parallel request can report verified placement without qualifying a speed sample. See
|
|
78
|
-
[feedback and adaptive profiles](
|
|
78
|
+
[feedback and adaptive profiles](https://github.com/bgauryy/FreeLlama/blob/main/docs/MONITORING.md#adaptive-concurrency) for sample eligibility.
|
|
79
79
|
Treat a missing contract as a stale running binary, then rebuild and restart before testing placement.
|
|
80
80
|
|
|
81
81
|
Authentication covers both control routes and raw Ollama passthrough. A nonloopback listener also
|
|
82
82
|
requires `--allow-remote`; terminate TLS and add tenant authorization outside FreeLlama. The CLI
|
|
83
83
|
persists bounded aggregate feedback by default. Use `--ephemeral-feedback` only for disposable
|
|
84
|
-
runs. See the [production runbook](
|
|
84
|
+
runs. See the [production runbook](https://github.com/bgauryy/FreeLlama/blob/main/docs/PRODUCTION.md).
|
|
85
85
|
|
|
86
86
|
## Build the launcher and binary
|
|
87
87
|
|
|
@@ -119,7 +119,7 @@ errors, and the files included in the package. Core CLI contracts live in
|
|
|
119
119
|
|
|
120
120
|
`delegate_research` and the online model-library view are MCP-only. Control-plane startup, proxy
|
|
121
121
|
startup, auth-token creation, and benchmarks are CLI-only. Generate a routing policy with
|
|
122
|
-
`freellama policy from-eval` (see [CLI reference](
|
|
122
|
+
`freellama policy from-eval` (see [CLI reference](https://github.com/bgauryy/FreeLlama/blob/main/docs/CLI.md#earn-medium-routing-confidence)).
|
|
123
123
|
|
|
124
124
|
Use the surfaces together: the CLI starts and verifies the local services; MCP lets an agent inspect
|
|
125
125
|
that state and submit bounded work. An installed MCP client can read `freellama://docs/index` and
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@octocodeai/freellama",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.1",
|
|
4
4
|
"description": "Local Ollama model gateway, router, and benchmarker — CLI. Wraps the compiled Rust binary from packages/rust-core.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -33,16 +33,17 @@
|
|
|
33
33
|
"test": "vitest run",
|
|
34
34
|
"test:watch": "vitest"
|
|
35
35
|
},
|
|
36
|
+
"author": "Guy Bary",
|
|
36
37
|
"license": "Apache-2.0 OR MIT",
|
|
37
38
|
"optionalDependencies": {
|
|
38
|
-
"@octocodeai/freellama-native-darwin-arm64": "0.6.
|
|
39
|
-
"@octocodeai/freellama-native-darwin-x64": "0.6.
|
|
40
|
-
"@octocodeai/freellama-native-linux-arm64-gnu": "0.6.
|
|
41
|
-
"@octocodeai/freellama-native-linux-arm64-musl": "0.6.
|
|
42
|
-
"@octocodeai/freellama-native-linux-x64-gnu": "0.6.
|
|
43
|
-
"@octocodeai/freellama-native-linux-x64-musl": "0.6.
|
|
44
|
-
"@octocodeai/freellama-native-win32-arm64-msvc": "0.6.
|
|
45
|
-
"@octocodeai/freellama-native-win32-x64-msvc": "0.6.
|
|
39
|
+
"@octocodeai/freellama-native-darwin-arm64": "0.6.1",
|
|
40
|
+
"@octocodeai/freellama-native-darwin-x64": "0.6.1",
|
|
41
|
+
"@octocodeai/freellama-native-linux-arm64-gnu": "0.6.1",
|
|
42
|
+
"@octocodeai/freellama-native-linux-arm64-musl": "0.6.1",
|
|
43
|
+
"@octocodeai/freellama-native-linux-x64-gnu": "0.6.1",
|
|
44
|
+
"@octocodeai/freellama-native-linux-x64-musl": "0.6.1",
|
|
45
|
+
"@octocodeai/freellama-native-win32-arm64-msvc": "0.6.1",
|
|
46
|
+
"@octocodeai/freellama-native-win32-x64-msvc": "0.6.1"
|
|
46
47
|
},
|
|
47
48
|
"publishConfig": {
|
|
48
49
|
"access": "public",
|