@gapi/gcli 1.8.263 → 1.8.265

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: local-ai-runners
3
- description: Work on the gcli local AI runner (runner:start/list/models/delete) - the subscriptions-transport-ws client that bridges a user's Ollama to the Lambforge chat. USE when the runner won't connect, drops requests, or before changing src/commands/runner/**.
3
+ description: Work on the gcli local AI runner (runner:start/list/models/delete) - the subscriptions-transport-ws client that bridges a user's local model servers (Ollama, Strata / OpenAI-compatible, via adapters) to the Lambforge chat. USE when the runner won't connect, drops requests, or before changing src/commands/runner/**.
4
4
  ---
5
5
 
6
6
  # gcli local AI runner
@@ -22,6 +22,16 @@ security, symptom table): `graphql-server/docs/features/local-ai-runners.md`.
22
22
  idle timeout kills it (spawned agent workers queue on one GPU).
23
23
  - Socket close aborts every in-flight Ollama request; a refused token is
24
24
  fatal (`RunnerAuthError`), everything else reconnects with backoff.
25
+ - Model servers are adapters (`src/commands/runner/adapters/`): the chat
26
+ format stays OpenAI `/v1/chat/completions`; adapters only discover models
27
+ and forward. The heartbeat's model objects must keep the
28
+ `LocalRunnerModelInput` shape (name, parameterSize, quantization, family,
29
+ capabilities, contextLength, runtimeContextLength) - adding a field there
30
+ breaks the GraphQL input unless graphql-server adds it too. The primary
31
+ (first) backend's model names must stay unprefixed: saved `runner:models`
32
+ selections depend on them.
33
+ - API keys of backends live only in the 0600 `runner.json` or env
34
+ (`$<NAME>_API_KEY`), never in the systemd unit or logs.
25
35
  - Global `WebSocket` -> Node >= 22.
26
36
  - Daemon (`runner:daemon:*`, `daemon.ts`): systemd USER unit; never put the
27
37
  token or other secrets in the unit (it's world-readable-ish config) - it
package/README-CLI.md CHANGED
@@ -5,7 +5,7 @@ Options:
5
5
  -h, --help display help for command
6
6
 
7
7
  Commands:
8
- login [options] Login to graphql-server
8
+ login [options] Log in. Without --token: prints a link, opens your browser and waits for you to approve it on lambforge.com
9
9
  lambda:list [options] List of all lambdas for project
10
10
  lambda:get [options] Get lambda by id
11
11
  lambda:create [options]
@@ -43,6 +43,7 @@ Commands:
43
43
  cluster:nodes [options] Every node of a cluster: its Hetzner control planes and workers and the machines attached to it
44
44
  cluster:pause [options] Pause an attached machine (cordon): nothing new runs there, running work stays
45
45
  cluster:resume [options] Let new work run on a paused machine again (uncordon)
46
+ cluster:node-storage [options] Let an attached machine carry volumes on its own disk (it restarts once), or stop with --off
46
47
  cluster:reset-machine [options] Bring a detached machine back to maintenance mode to attach it again (wipes its config and data, keeps the boot loader)
47
48
  cluster:talosconfig [options] A short-lived talosconfig for one of your Talos clusters (admin of its machines with talosctl) - printed, or saved with -o
48
49
  cluster:detach [options] Drain and remove an attached machine from a cluster (reset the machine afterwards)
@@ -52,6 +53,7 @@ Commands:
52
53
  cluster:scale [options] Set the number of worker nodes (1-10) of a multi-node cluster - billed per node-hour
53
54
  cluster:scale:control-planes [options] Set the number of control-plane nodes (1, 3 or 5)
54
55
  cluster:autoscale [options] Change the worker autoscaling policy - only the flags you pass change
56
+ cluster:plan [options] Why pods are pending and what the pods-mode autoscaler would do: a verdict per pod, the nodes it would create, consolidation
55
57
  cluster:multi-node [options] Convert a single-node cluster to dedicated worker nodes (required for autoscaling)
56
58
  cluster:single-node [options] Convert a multi-node cluster back to a single node (removes the worker nodes)
57
59
  cluster:load-balancer [options] Add or remove the cluster's load balancer (billed per hour while enabled). On a virtual cluster this is its dedicated LB - its own IP; the platform subdomain keeps working either way
@@ -62,7 +64,7 @@ Commands:
62
64
  plugin:installed [options] List plugins installed for a project (may include the same plugin on several clusters)
63
65
  plugin:install [options] Install a marketplace plugin for a project
64
66
  plugin:uninstall [options] Uninstall a plugin from a project (--cluster picks which installation, if it's on several)
65
- runner:start [options] Connect this machine's Ollama to the Lambforge chat as a local AI runner (outbound connection only - works behind NAT, Ollama is never exposed)
67
+ runner:start [options] Connect this machine's model servers (Ollama, Strata or any OpenAI-compatible server) to the Lambforge chat as a local AI runner (outbound connection only - works behind NAT, nothing is exposed)
66
68
  runner:daemon:install|runner:start:daemon [options] Run the runner in the background as a systemd user service: starts with your session, restarts on failure (Linux)
67
69
  runner:daemon:status [options] Show the runner service status (systemctl --user status)
68
70
  runner:daemon:start [options] Start the installed runner service, and report whether it stays up
@@ -94,9 +96,10 @@ Commands:
94
96
  --- Subcommand: login ---
95
97
  Usage: gcli login [options]
96
98
 
97
- Login to graphql-server
99
+ Log in. Without --token: prints a link, opens your browser and waits for you to approve it on lambforge.com
98
100
 
99
101
  Options:
102
+ --no-browser Only print the login link, do not open a browser
100
103
  -k, --key <key> Specify api key
101
104
  -uu, --uploadUrl <key> Specify upload server
102
105
  -ci, --ci Initialize directories for CI/CD purposes
@@ -371,12 +374,17 @@ Usage: gcli build [options]
371
374
  Build bundle using esbuild https://esbuild.github.io
372
375
 
373
376
  Options:
374
- -f, --files <files...> File or files to bundle defaults to index.ts
375
- -b, --bundle Bundle code (default: true)
376
- -m, --minify Minify code (default: false)
377
+ -f, --files <files...> File or files to bundle (default: function.file
378
+ from lambforge.yaml, else index.ts)
379
+ -b, --bundle Bundle code (default: options.bundler.bundle,
380
+ else true)
381
+ -m, --minify Minify code (default: options.bundler.minify,
382
+ else false)
377
383
  -p, --platform <char> Platform (default: "node")
378
- -t, --target <char> Target (default: "node14.4")
379
- -o, --outfile <char> Outfile name
384
+ -t, --target <char> Target (default: options.bundler.target, else
385
+ node24)
386
+ -o, --outfile <char> Outfile name (default: options.bundler.outfile,
387
+ else index.js)
380
388
  -e, --external <external...> External libraries
381
389
  -h, --help display help for command
382
390
  
@@ -693,6 +701,8 @@ Options:
693
701
  --reserved Only workloads that tolerate the pool run there
694
702
  (default: plain extra capacity)
695
703
  --disk <disk> Install disk, e.g. /dev/nvme0n1 (WIPED)
704
+ --storage The machine also carries volumes on its own disk
705
+ (default: lambdas only)
696
706
  --yes Do not ask before wiping the disk
697
707
  --no-wait Do not wait for the machine to join
698
708
  --timeout <minutes> How long to wait for the join (default 10)
@@ -733,6 +743,22 @@ Options:
733
743
  -n, --name <name> Machine name (see cluster:nodes)
734
744
  -h, --help display help for command
735
745
  
746
+ --- Subcommand: cluster:node-storage ---
747
+ Usage: gcli cluster:node-storage [options]
748
+
749
+ Let an attached machine carry volumes on its own disk (it restarts once), or stop with --off
750
+
751
+ Options:
752
+ -p, --project <project>
753
+ -c, --cluster <cluster> Cluster id (see cluster:list)
754
+ --clusterName <clusterName> Cluster name instead of --cluster
755
+ -n, --name <name> Machine name (see cluster:nodes)
756
+ --off Stop carrying volumes (refused while it holds
757
+ one)
758
+ --no-wait Do not wait until the machine is ready
759
+ --timeout <minutes> How long to wait (default 40)
760
+ -h, --help display help for command
761
+ 
736
762
  --- Subcommand: cluster:reset-machine ---
737
763
  Usage: gcli cluster:reset-machine [options]
738
764
 
@@ -851,6 +877,31 @@ Options:
851
877
  --memDown <percent> Scale down below this memory % (1-100)
852
878
  --samples <n> Consecutive readings before acting (1-20)
853
879
  --cooldown <minutes> Minutes between scale actions (1-1440)
880
+ --profile <profile> conservative | balanced | aggressive presets the
881
+ thresholds above; custom uses them as given
882
+ --serverTypes <types> Comma-separated worker server types in order of
883
+ preference, e.g. cx33,cpx22 (the next is used
884
+ when one is out of stock); "" = the cluster's
885
+ own type
886
+ --step <n> Workers added per scale-up (1-3, custom profile)
887
+ --mode <mode> thresholds (CPU/memory %) | pods (create the
888
+ cheapest nodes that fit unschedulable pods)
889
+ --consolidation pods mode: remove the newest worker when its
890
+ pods fit elsewhere
891
+ --no-consolidation pods mode: never remove workers by itself
892
+ --minNodeAge <minutes> pods mode: never remove a worker younger than
893
+ this (5-1440)
894
+ -h, --help display help for command
895
+ 
896
+ --- Subcommand: cluster:plan ---
897
+ Usage: gcli cluster:plan [options]
898
+
899
+ Why pods are pending and what the pods-mode autoscaler would do: a verdict per pod, the nodes it would create, consolidation
900
+
901
+ Options:
902
+ -p, --project <project>
903
+ -c, --cluster <cluster> Cluster id (see cluster:list)
904
+ --clusterName <clusterName> Cluster name instead of --cluster
854
905
  -h, --help display help for command
855
906
  
856
907
  --- Subcommand: cluster:multi-node ---
@@ -972,18 +1023,25 @@ Options:
972
1023
  --- Subcommand: runner:start ---
973
1024
  Usage: gcli runner:start [options]
974
1025
 
975
- Connect this machine's Ollama to the Lambforge chat as a local AI runner (outbound connection only - works behind NAT, Ollama is never exposed)
976
-
977
- Options:
978
- -n, --name <name> Runner name shown in the chat model picker (default:
979
- hostname)
980
- -t, --token <token> Runner token from Profile -> Local AI runners (instead
981
- of `gcli login`)
982
- --ollama <url> Ollama URL (default: $OLLAMA_HOST or
983
- http://127.0.0.1:11434)
984
- --wsUrl <url> Lambforge subscriptions endpoint override
985
- --reset Forget the saved runner and register a new one
986
- -h, --help display help for command
1026
+ Connect this machine's model servers (Ollama, Strata or any OpenAI-compatible server) to the Lambforge chat as a local AI runner (outbound connection only - works behind NAT, nothing is exposed)
1027
+
1028
+ Options:
1029
+ -n, --name <name> Runner name shown in the chat model picker
1030
+ (default: hostname)
1031
+ -t, --token <token> Runner token from Profile -> Local AI runners
1032
+ (instead of `gcli login`)
1033
+ --ollama <url> Ollama URL (default: $OLLAMA_HOST or
1034
+ http://127.0.0.1:11434)
1035
+ --backend <name=kind:url> Model server to serve from, repeatable; kinds:
1036
+ ollama, openai (Strata, llama-server, LM Studio,
1037
+ vLLM). E.g. --backend
1038
+ strata=openai:http://127.0.0.1:8080. With
1039
+ --ollama, Ollama comes first
1040
+ --backend-key <name=key> API key for a backend, repeatable (default:
1041
+ $<NAME>_API_KEY, e.g. $STRATA_API_KEY)
1042
+ --wsUrl <url> Lambforge subscriptions endpoint override
1043
+ --reset Forget the saved runner and register a new one
1044
+ -h, --help display help for command
987
1045
  
988
1046
  --- Subcommand: runner:daemon:install ---
989
1047
  Usage: gcli runner:daemon:install|runner:start:daemon [options]
@@ -991,15 +1049,22 @@ Usage: gcli runner:daemon:install|runner:start:daemon [options]
991
1049
  Run the runner in the background as a systemd user service: starts with your session, restarts on failure (Linux)
992
1050
 
993
1051
  Options:
994
- -n, --name <name> Runner name when registering (default: hostname)
995
- -t, --token <token> Runner token for a machine without `gcli login`
996
- --ollama <url> Ollama URL (default: $OLLAMA_HOST or
997
- http://127.0.0.1:11434)
998
- --wsUrl <url> Lambforge subscriptions endpoint override
999
- --reset Forget the saved runner and register a new one
1000
- --service <name> systemd unit name (default: lambforge-runner)
1001
- --dry-run Print the unit file instead of installing it
1002
- -h, --help display help for command
1052
+ -n, --name <name> Runner name when registering (default: hostname)
1053
+ -t, --token <token> Runner token for a machine without `gcli login`
1054
+ --ollama <url> Ollama URL (default: $OLLAMA_HOST or
1055
+ http://127.0.0.1:11434)
1056
+ --backend <name=kind:url> Model server to serve from, repeatable; kinds:
1057
+ ollama, openai (Strata, llama-server, LM Studio,
1058
+ vLLM). E.g. --backend
1059
+ strata=openai:http://127.0.0.1:8080. With
1060
+ --ollama, Ollama comes first
1061
+ --backend-key <name=key> API key for a backend, repeatable (default:
1062
+ $<NAME>_API_KEY, e.g. $STRATA_API_KEY)
1063
+ --wsUrl <url> Lambforge subscriptions endpoint override
1064
+ --reset Forget the saved runner and register a new one
1065
+ --service <name> systemd unit name (default: lambforge-runner)
1066
+ --dry-run Print the unit file instead of installing it
1067
+ -h, --help display help for command
1003
1068
  
1004
1069
  --- Subcommand: runner:daemon:status ---
1005
1070
  Usage: gcli runner:daemon:status [options]
package/README.md CHANGED
@@ -409,6 +409,8 @@ function:
409
409
  concurrency: 1
410
410
  functionTimeout: 60
411
411
  specializationTimeout: 120
412
+ spread: machines # replicas on different machines ('none' = the scheduler decides)
413
+ spreadStrict: false # true = a replica waits rather than share a machine
412
414
 
413
415
  environment:
414
416
  name: nodejs
@@ -432,6 +434,14 @@ options:
432
434
  external: []
433
435
  ```
434
436
 
437
+ **Spread across machines** (newdeploy, plan 33): `spread: machines` puts the
438
+ replicas on different machines of the cluster, as evenly as possible, so a
439
+ machine going down takes one replica, not several (use it with
440
+ `minScale >= 2`). Soft by default: it never blocks a scale-up.
441
+ `spreadStrict: true` keeps a replica waiting rather than sharing a machine.
442
+ Flags: `--spread machines|none`, `--spreadStrict true|false`. Left out = the
443
+ lambda keeps what it has. Changing it restarts the replicas once.
444
+
435
445
  The same file can also carry `cluster:`, `rabbitmq:`, `plugin:` and `triggers:` sections, read by `cluster:apply` / `cluster:create`, `rabbitmq:create`, `plugin:install` and `cron:create` respectively (each also accepts its own dedicated `--spec` file, e.g. `cluster.yaml`, if you'd rather keep them separate):
436
446
 
437
447
  ```yaml
@@ -1038,9 +1048,39 @@ running after logout and start it at boot, enable lingering once:
1038
1048
  `sudo loginctl enable-linger $USER`. Preview the unit with `--dry-run`;
1039
1049
  `--service <name>` installs several runners side by side.
1040
1050
 
1051
+ ### Several model servers in one runner (Ollama + Strata, LM Studio, ...)
1052
+
1053
+ One runner can serve models from several local servers at once. Each one is a
1054
+ `--backend <name>=<kind>:<url>` (repeatable). Built-in kinds:
1055
+
1056
+ | Kind | Servers | Models from |
1057
+ |---|---|---|
1058
+ | `ollama` | Ollama | `/api/tags` + `/api/show` + `/api/ps` |
1059
+ | `openai` | any OpenAI-compatible server: Strata, llama.cpp `llama-server`, LM Studio, vLLM | `/v1/models` (+ `/props` for the running context window, when the server has it) |
1060
+
1061
+ ```bash
1062
+ # Ollama + Strata in one runner
1063
+ gcli runner:start --ollama http://127.0.0.1:11434 \
1064
+ --backend strata=openai:http://127.0.0.1:8080
1065
+ # API key: --backend-key strata=<key>, or $STRATA_API_KEY ($<NAME>_API_KEY)
1066
+
1067
+ # the same as a service: backends + keys are saved in ~/.graphql-server/runner.json (0600)
1068
+ gcli runner:daemon:install --ollama http://127.0.0.1:11434 \
1069
+ --backend strata=openai:http://127.0.0.1:8080
1070
+ ```
1071
+
1072
+ The first backend is the primary: its models keep their names. The others
1073
+ appear as `<name>/<model>` (e.g. `strata/Qwen3.8-Flash-Next`) in the chat
1074
+ picker and in `runner:models --show`. A backend that is down only removes its
1075
+ own models; the rest keep working. Without `--backend` the runner serves one
1076
+ Ollama, exactly as before. Re-running `runner:daemon:install` without
1077
+ `--backend` goes back to Ollama only.
1078
+
1041
1079
  Notes:
1042
1080
 
1043
- - Models are only offered if Ollama reports them as chat-capable; a model
1081
+ - Ollama models are only offered if Ollama reports them as chat-capable;
1082
+ OpenAI-compatible servers don't report capabilities, so their models are
1083
+ offered with tools; a model
1044
1084
  without the `tools` capability is used without platform tools.
1045
1085
  - Local models don't count against your plan's AI token quota.
1046
1086
  - Requires Node.js >= 22.
@@ -0,0 +1,43 @@
1
+ ## 2026-11-15 - local-ai-runners: one runner serves several model servers through adapters (Ollama + OpenAI-compatible, e.g. Strata)
2
+
3
+ **Feature:** local-ai-runners · **Type:** feat
4
+
5
+ Why: a runner could only reach one Ollama. Models served by Strata (a llama.cpp-based engine) or any other OpenAI-compatible server (llama-server, LM Studio, vLLM) could not reach the Lambforge chat. Pointing `--ollama` at them did not work either: they have no `/api/tags`, `/api/show` or `/api/ps`.
6
+
7
+ - New `src/commands/runner/adapters/`:
8
+ - `ModelAdapter` (`types.ts`): an adapter owns `listModels()` and `forward()`. The format between Lambforge and the runner stays the OpenAI chat API (`POST /v1/chat/completions`). The allow-list is unchanged, so this is a gcli-only change.
9
+ - `HttpModelAdapter` (`base.ts`) handles the URL and the optional bearer key. Its default `forward` passes the request straight through.
10
+ - Built-in adapters:
11
+ - `OllamaAdapter` (`ollama.ts`) wraps the existing `OllamaModels`.
12
+ - `OpenAIAdapter` (`openai.ts`) reads `/v1/models`. The running context window comes from `meta.n_ctx`, llama.cpp's `/props` `n_ctx`, or vLLM's `max_model_len`.
13
+ - Capabilities use Ollama's names. When `/props` answers: `completion`, plus `tools` (`chat_template_caps.supports_tools` / `supports_tool_calls`), plus `vision`. Without `/props`: `[]`. An empty list would work in the backend, but the web app's runner page shows "chat only - no agent tools" for it; that label is what showed up first with Strata.
14
+ - llama.cpp lists a model again under each alias (Strata: `qwen3.8-flash-next-iq3_s` and `iq3_s`). The adapter keeps the first entry only.
15
+ - `registry.ts`: `registerAdapter(kind, factory)`. A new kind of server needs one class and one registration line.
16
+ - `ModelRouter` (`router.ts`):
17
+ - Merges every backend's models into one heartbeat. The first backend is the primary and keeps its model names; the others are shown as `<backend>/<model>`.
18
+ - Routes each request by its `model` and renames the model back before forwarding. The request body is rewritten only when the name differs.
19
+ - Tracks reachability per backend: a backend that is down only removes its own models.
20
+ - `spec.ts` handles `--backend <name>=<kind>:<url>` parsing and `resolveBackends`. Resolution order: `--backend` (`--ollama` is prepended), then `runner.json.backends`, then one Ollama. API keys come from `--backend-key`, then `$<NAME>_API_KEY`, then the saved key.
21
+ - `runner:start` and `runner:daemon:install` gain the repeatable `--backend` and `--backend-key` flags (`index.ts`):
22
+ - `start.ts` logs `Backends: ...`; a set key shows as `[key]`.
23
+ - `daemon-commands.ts` saves the resolved backends, keys included, in the 0600 `runner.json`, and the unit then runs plain `runner:start`. Keys never go into the unit. Re-installing without `--backend` drops the saved backends.
24
+ - `RunnerConnection` (`connection.ts`) takes `backends` or ready-made `adapters`; the old `ollamaUrl` still works. Errors name the backend, e.g. "Could not reach strata (openai) at ...".
25
+ - Backward compatible: without `--backend`, the heartbeat, the requests and the request bodies are byte-for-byte what they were. Existing units (`runner:start --ollama <url>`) behave the same.
26
+ - No graphql-server change is needed: the model objects keep the `LocalRunnerModelInput` fields, and `/` in model names was already valid (e.g. `x/z-image-turbo:fp8`).
27
+ - Docs updated: README ("Several model servers in one runner"), `docs/features/local-ai-runners.md` ("Backends (model adapters)"), and the `local-ai-runners` skill (invariants).
28
+ - **Test:**
29
+ - `npx jest --coverage=false test/runner-adapters.spec.ts test/runner.spec.ts`. The adapter spec covers `--backend` parsing and errors, key precedence, the OpenAI adapter with and without `/props`, the registry with a custom kind, and the router (naming, renaming, prefix fallback, a down backend, duplicate names). `runner.spec.ts` adds end to end through the socket: one heartbeat carrying both backends' models; a `strata/` model sent to the second backend under its own name with the bearer key; Ollama kept serving while the second backend is down.
30
+ - Live: `gcli runner:start --ollama http://127.0.0.1:11434 --backend strata=openai:http://127.0.0.1:8080`, then pick `strata/<model>` in the chat.
31
+ - **Debug:**
32
+ - Startup line `Backends: ...` shows what was resolved.
33
+ - `Cannot reach <backend> (<kind>) at <url> (...)` is logged once per outage per backend; `<backend> at <url>: N model(s)` when it comes back.
34
+ - `-> chat request (<model>) via <backend>` shows the routing.
35
+ - A model missing from the picker: check that its server answers `/v1/models`. A 401 there means the key is missing or wrong (`--backend-key` / `$<NAME>_API_KEY`).
36
+ - A `strata/...` model that reaches the wrong server: the backend name must match the prefix.
37
+ - **Verification:**
38
+ - jest: 20 suites, 169 tests pass. `tsc --noEmit` and eslint are clean.
39
+ - `npm run build`: `runner:start --help` lists the flags. `runner:daemon:install --dry-run --backend strata=openai:http://127.0.0.1:8080` gives `ExecStart=... runner:start` (no key); without `--backend` it is unchanged (`--ollama http://127.0.0.1:11434`).
40
+ - Live, against the real Ollama with the router and both adapters: 3 models listed with capabilities and context, and a chat request through `forward` returned 200. Strata on :8080 was not running (its model was still downloading); it was reported as unreachable while Ollama kept working.
41
+ - Live with Strata (port 18500, with key) through the `lambforge-runner` service: `gcli runner:list` shows `qwen3.8:27b, gpt-oss-128k:latest, gpt-oss:20b, strata/qwen3.8-flash-next-iq3_s`. The Strata model reports `completion, tools, vision` and runtime context 131072.
42
+ - Not run: a chat request to the Strata model through the Lambforge chat.
43
+ - jest after the capabilities and alias fix: 20 suites, 171 tests pass.
@@ -0,0 +1,16 @@
1
+ ## 2026-11-17 - lambda-spread: lambda:create/update --spread and --spreadStrict, lambforge.yaml spread keys
2
+
3
+ **Feature:** lambda-spread · **Type:** feat
4
+
5
+ - `lambda:create` / `lambda:update` take `--spread machines|none` and
6
+ `--spreadStrict true|false`; `lambforge.yaml` takes
7
+ `function.scaleOptions.spread` / `spreadStrict`. Flags win over the yaml.
8
+ Sent only when set (`helpers/spread.ts` `spreadScaleOptions`), so gcli
9
+ works against an API without the fields and an update without them keeps
10
+ the lambda's current choice. Canonical doc: graphql-server-lambdas
11
+ `docs/features/lambda-spread.md`.
12
+ - **Test:** `npx jest --coverage=false test/lambda-spread.spec.ts`.
13
+ - **Debug:** "--spread must be 'machines' or 'none'" = a typo in the flag or
14
+ yaml. An API older than the feature answers `Field "spread" is not defined
15
+ by type "LambdaScaleInputOptions"`.
16
+ - **Verification:** jest 174/174, tsc clean.
@@ -42,3 +42,4 @@ Outline (drop sections that don't apply):
42
42
  | `mcp-proxy` | [mcp-proxy.md](mcp-proxy.md) | `gcli mcp:start` - local stdio MCP server forwarding to the platform MCP endpoint; token refreshed per request |
43
43
  | `cli-browser-login` | [cli-browser-login.md](cli-browser-login.md) | Slice (canonical: graphql-server): `gcli login` with no token through the browser; `mcp:start` LambforgeLogin |
44
44
  | `lambda-site-url` | [lambda-site-url.md](lambda-site-url.md) | Slice (canonical: graphql-server-lambdas): `Site:` line after create/update |
45
+ | `lambda-spread` | [lambda-spread.md](lambda-spread.md) | Slice (canonical: graphql-server-lambdas): replicas across machines (plan 33): `--spread` / `--spreadStrict`, yaml keys |
@@ -0,0 +1,9 @@
1
+ # lambda-spread (gcli slice)
2
+
3
+ Canonical: graphql-server-lambdas `docs/features/lambda-spread.md` (plan 33).
4
+
5
+ - Flags: `--spread machines|none`, `--spreadStrict true|false` on
6
+ `lambda:create` / `lambda:update`.
7
+ - yaml: `function.scaleOptions.spread`, `function.scaleOptions.spreadStrict`.
8
+ - `src/commands/lambda/helpers/spread.ts` - only set values are sent; flags
9
+ win over the yaml; unknown values refused before the call.
@@ -9,11 +9,11 @@ what lives in gcli.
9
9
 
10
10
  | Command | What it does |
11
11
  |---|---|
12
- | `gcli runner:start [--name] [--token] [--ollama] [--wsUrl] [--reset]` | Connects this machine's Ollama to the chat. First run registers a runner with the `gcli login` session and saves it to `~/.graphql-server/runner.json` (0600); `--token` instead uses a token from Profile -> Local AI runners (headless machines); `--reset` forgets the saved runner and registers a new one. Runs until Ctrl+C, reconnecting with backoff (1s -> 30s). |
12
+ | `gcli runner:start [--name] [--token] [--ollama] [--backend name=kind:url]... [--backend-key name=key]... [--wsUrl] [--reset]` | Connects this machine's model servers (Ollama by default; see "Backends") to the chat. First run registers a runner with the `gcli login` session and saves it to `~/.graphql-server/runner.json` (0600); `--token` instead uses a token from Profile -> Local AI runners (headless machines); `--reset` forgets the saved runner and registers a new one. Runs until Ctrl+C, reconnecting with backoff (1s -> 30s). |
13
13
  | `gcli runner:list` | Your runners: online, last seen, models |
14
14
  | `gcli runner:models [--id] [--show a,b \| --all \| --none]` | Which models appear in the chat picker; no flag = list |
15
15
  | `gcli runner:delete [--id]` | Delete a runner and revoke its token (default: the saved one) |
16
- | `gcli runner:daemon:install` (alias `runner:start:daemon`) `[runner:start options] [--service] [--dry-run]` | Resolve/register the runner now (saves `runner.json`), then install + enable + restart a systemd USER unit running `runner:start --ollama <resolved>` (Linux) |
16
+ | `gcli runner:daemon:install` (alias `runner:start:daemon`) `[runner:start options] [--service] [--dry-run]` | Resolve/register the runner now (saves `runner.json`), then install + enable + restart a systemd USER unit (Linux). One Ollama: the unit runs `runner:start --ollama <resolved>`. With `--backend`: the resolved backends (keys included) are saved in `runner.json.backends` and the unit runs plain `runner:start` - keys never go in the unit. Re-installing without `--backend` drops saved backends |
17
17
  | `gcli runner:daemon:start` / `:restart` `[--service]` | `systemctl --user start/restart`, then after ~2 s reads `ActiveState`/`ExecMainStatus` (`SystemdUserService.outcome`): running, or an error - exit 78 = token refused (deleted, or registered on another server than `~/.graphql-server/url`) with the `runner:daemon:install --reset` hint. Refuses when the unit is not installed |
18
18
  | `gcli runner:daemon:stop` `[--service]` | `systemctl --user stop` (stays enabled: starts again at the next login) |
19
19
  | `gcli runner:daemon:status` / `:logs` / `:uninstall` | `systemctl --user status`, `journalctl --user -f`, disable --now + remove the unit |
@@ -24,6 +24,46 @@ Endpoints: `--wsUrl` > `runner.json` > derived from the `gcli login` URL
24
24
  Ollama: `--ollama` > `$OLLAMA_HOST` (scheme optional, `0.0.0.0` -> `127.0.0.1`)
25
25
  > `http://127.0.0.1:11434`. Requires Node >= 22 (global `WebSocket`).
26
26
 
27
+ ## Backends (model adapters)
28
+
29
+ The wire format between Lambforge and the runner is always the OpenAI chat
30
+ API (`POST /v1/chat/completions`, the only allowed path - unchanged). A
31
+ **backend** is one local model server; its **adapter** owns only what differs
32
+ per server: discovering models (`listModels`) and forwarding a chat request
33
+ (`forward`, a pass-through for both built-ins). The backend (graphql-server)
34
+ is unaware of adapters: it still sees one runner and one model list, with the
35
+ same `LocalRunnerModelInput` fields.
36
+
37
+ - Built-in kinds: `ollama` (`/api/tags`, `/api/show`, `/api/ps`) and `openai`
38
+ (`/v1/models`; `runtimeContextLength` from llama.cpp's `/props` `n_ctx`,
39
+ else vLLM's `max_model_len`; `contextLength` from `meta.n_ctx_train` /
40
+ `context_length` / `max_model_len`; capabilities from `/props`:
41
+ `completion` + `tools` when `chat_template_caps.supports_tools` /
42
+ `supports_tool_calls`, + `vision` when `modalities.vision` or an image
43
+ input modality - no `/props` = `[]`, which the backend treats as
44
+ tool-capable but the web app's runner page labels "chat only"; alias
45
+ entries of one model are dropped, the first wins). Strata's engine is
46
+ llama.cpp-based: the port and the required bearer key are per instance in
47
+ `~/.strata/data/native-instances/<uuid>.json` (`port`, `api_key`) - e.g.
48
+ 18500, not the 8080 of its install file.
49
+ - Resolution (`adapters/spec.ts` `resolveBackends`): `--backend` (with
50
+ `--ollama` prepended as `ollama`) > `runner.json.backends` (ignored when
51
+ `--ollama` is given - that's what single-Ollama units run) > one Ollama.
52
+ Keys: `--backend-key` > `$<NAME>_API_KEY` > saved key. Logs show `[key]`,
53
+ never the key.
54
+ - Naming (`adapters/router.ts` `ModelRouter`): the first backend is primary
55
+ and keeps its model names (a plain Ollama runner reports exactly what it
56
+ always did, so saved `runner:models` selections keep working); the others
57
+ are `<backend>/<model>`. `route()` maps a request's `model` back to the
58
+ backend and its own name (body rewritten only then); unknown models go by
59
+ prefix, else to the primary, unchanged. Each backend's reachability is
60
+ tracked and logged separately; a down backend only drops its own models.
61
+ - New kind: implement `ModelAdapter` (or extend `HttpModelAdapter`) in
62
+ `adapters/` and `registerAdapter('<kind>', factory)` in
63
+ `adapters/registry.ts`. An adapter for a server with a different chat API
64
+ overrides `forward` and translates request and stream to/from the OpenAI
65
+ format.
66
+
27
67
  ## Code
28
68
 
29
69
  `src/commands/runner/`:
@@ -36,6 +76,13 @@ Ollama: `--ollama` > `$OLLAMA_HOST` (scheme optional, `0.0.0.0` -> `127.0.0.1`)
36
76
  30s while Ollama keeps a request queued. `FrameSender` numbers frames
37
77
  (`seq`), merges consecutive chunks (<= 32KB) and sends batches every 50ms,
38
78
  sequentially. `RunnerAuthError` = token refused -> exit 1 with a `--reset` hint.
79
+ - `adapters/` - `types.ts` (`ModelAdapter`, `AdapterSpec`), `base.ts`
80
+ (`HttpModelAdapter`: URL, bearer key, pass-through `forward`),
81
+ `ollama.ts` (`OllamaAdapter` over `OllamaModels`), `openai.ts`
82
+ (`OpenAIAdapter`), `registry.ts`, `router.ts` (`ModelRouter`), `spec.ts`
83
+ (`--backend` parsing, `resolveBackends`). `RunnerConnection` takes
84
+ `backends` (or ready `adapters`; the old `ollamaUrl` still works) and
85
+ delegates heartbeat models and request routing to the router.
39
86
  - `ollama.ts` - `ALLOWED_REQUESTS` (only `POST /v1/chat/completions`; the
40
87
  backend has its own copy), `OllamaModels` (`/api/tags` + cached
41
88
  `/api/show` per digest; drops models without the `completion` capability
@@ -64,7 +111,8 @@ Ollama: `--ollama` > `$OLLAMA_HOST` (scheme optional, `0.0.0.0` -> `127.0.0.1`)
64
111
  ## How to test
65
112
 
66
113
  ```bash
67
- npx jest test/runner.spec.ts --coverage=false # fake server socket + fake streaming Ollama
114
+ npx jest test/runner.spec.ts --coverage=false # fake server socket + fake streaming Ollama (+ a Strata-like 2nd backend)
115
+ npx jest test/runner-adapters.spec.ts --coverage=false # --backend parsing, keys, OpenAI adapter, registry, router
68
116
  npm run build && node release/index.js runner:start --help
69
117
  ```
70
118
 
@@ -78,7 +126,9 @@ that from a feature branch unless you are releasing.
78
126
  ## How to debug
79
127
 
80
128
  Runner log lines: `-> chat request (<model>)`, `<- <status> (<model>) in Ns`,
81
- `x cancelled`, `x <error>`, `Heartbeat failed: ...`, `Cannot reach Ollama
82
- at ...`. `Disconnected from Lambforge; reconnecting in Ns` = socket closed
129
+ `x cancelled`, `x <error>`, `Heartbeat failed: ...`, `Cannot reach <backend>
130
+ (<kind>) at <url> (...)` (once per outage, per backend), `<backend> at <url>: N
131
+ model(s)`. With several backends, `-> chat request (<model>) via <backend>`.
132
+ `Backends: ...` at startup shows what was resolved (`[key]` = a key is set). `Disconnected from Lambforge; reconnecting in Ns` = socket closed
83
133
  (backend restart, network). For the backend side and the symptom table see
84
134
  the canonical doc.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@gapi/gcli",
3
- "version": "1.8.263",
3
+ "version": "1.8.265",
4
4
  "repository": {
5
5
  "type": "git",
6
6
  "url": "https://github.com/Stradivario/gapi.git"