pi-lemonade-link 1.0.2 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +68 -21
- package/docs/README.md +23 -0
- package/docs/anthropic.md +183 -6
- package/docs/internal.md +1080 -0
- package/docs/lemonade.md +2453 -1665
- package/docs/llamacpp.md +385 -252
- package/docs/mcp.md +232 -128
- package/docs/ollama.md +967 -15
- package/docs/openai.md +1734 -634
- package/docs/router.md +201 -0
- package/index.ts +2038 -306
- package/lemonade.example.json +24 -2
- package/package.json +3 -3
package/docs/internal.md
ADDED
|
@@ -0,0 +1,1080 @@
|
|
|
1
|
+
# Internal API
|
|
2
|
+
|
|
3
|
+
Internal endpoints control and configure `lemond`. The clients bundled with the server, such as the desktop app, the tray and the `lemonade` CLI, use them, and they accept connections from any address, so a client on another machine can manage a shared `lemond`.
|
|
4
|
+
|
|
5
|
+
Apps that talk to a system-wide Lemonade Server install must not use these endpoints: that server belongs to its bundled clients, and these endpoints may change without notice. An app that embeds its own `lemond` owns that instance and should use them to manage it; see [Embeddable Lemonade: Runtime](../embeddable/runtime.md).
|
|
6
|
+
|
|
7
|
+
> **Warning:** When `lemond` is bound to a non-loopback host such as `0.0.0.0`, these endpoints, including shutdown and config, are reachable from the network. `LEMONADE_ADMIN_API_KEY` on its own secures only `/internal/*` and leaves every other endpoint open, including inference, model management, the Ollama and Anthropic routes and `/mcp`, so set `LEMONADE_API_KEY` to protect those too. `lemond` logs a startup warning when bound to a non-loopback host without `LEMONADE_API_KEY`.
|
|
8
|
+
|
|
9
|
+
<!-- BEGIN GENERATED: internal.summary -->
|
|
10
|
+
| Method | Endpoint | Description |
|
|
11
|
+
|--------|----------|-------------|
|
|
12
|
+
| `POST` | [`/internal/shutdown`](#post-internalshutdown) | Unload every model and stop `lemond` |
|
|
13
|
+
| `POST` | [`/internal/telemetry/flush`](#post-internaltelemetryflush) | Force-flush all queued telemetry trace spans |
|
|
14
|
+
| `POST` | [`/internal/pin`](#post-internalpin) | Pin or unpin a loaded model |
|
|
15
|
+
| `POST` | [`/internal/set`](#post-internalset) | Change config settings at runtime |
|
|
16
|
+
| `GET` | [`/internal/config`](#get-internalconfig) | Read the runtime config |
|
|
17
|
+
| `GET` | [`/internal/config/defaults`](#get-internalconfigdefaults) | Read the factory default config |
|
|
18
|
+
| `POST` | [`/internal/cleanup-cache`](#post-internalcleanup-cache) | Find or remove orphaned model files |
|
|
19
|
+
| `POST` | [`/internal/simulate-vram-pressure`](#post-internalsimulate-vram-pressure) | Run the eviction engine at a simulated VRAM usage |
|
|
20
|
+
| `POST` | [`/internal/models/sync`](#post-internalmodelssync) | Download updates for downloaded models |
|
|
21
|
+
| `GET` | [`/internal/models/sync/status`](#get-internalmodelssyncstatus) | Read the progress of a model sync |
|
|
22
|
+
| `GET` | [`/internal/aliases`](#get-internalaliases) | List all active model aliases |
|
|
23
|
+
| `POST` | [`/internal/aliases`](#post-internalaliases) | Create or update a model alias |
|
|
24
|
+
| `DELETE` | [`/internal/aliases/{alias}`](#delete-internalaliasesalias) | Remove a model alias |
|
|
25
|
+
| `GET` | [`/internal/routes`](#get-internalroutes) | List every route's specification |
|
|
26
|
+
| `GET` | [`/internal/model-sources`](#get-internalmodel-sources) | List every model source's descriptor |
|
|
27
|
+
<!-- END GENERATED: internal.summary -->
|
|
28
|
+
|
|
29
|
+
<!-- BEGIN GENERATED: internal.shutdown -->
|
|
30
|
+
## `POST /internal/shutdown`
|
|
31
|
+
<sub></sub>
|
|
32
|
+
|
|
33
|
+
Unloads every model, answers, and then stops `lemond`.
|
|
34
|
+
|
|
35
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
36
|
+
|
|
37
|
+
### Parameters
|
|
38
|
+
|
|
39
|
+
This endpoint does not take any parameters.
|
|
40
|
+
|
|
41
|
+
Backend processes such as `llama-server` have exited by the time the caller reads the answer. `lemond` then cancels its downloads and exits about 100 ms later.
|
|
42
|
+
|
|
43
|
+
### Example
|
|
44
|
+
|
|
45
|
+
=== "PowerShell"
|
|
46
|
+
|
|
47
|
+
```powershell
|
|
48
|
+
Invoke-WebRequest `
|
|
49
|
+
-Uri "http://localhost:13305/internal/shutdown" `
|
|
50
|
+
-Method POST
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
=== "Bash"
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
curl -X POST http://localhost:13305/internal/shutdown
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
**Response:** `200 OK`
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{"status": "shutting down"}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
### Response format
|
|
66
|
+
|
|
67
|
+
- `status` - Always `shutting down`.
|
|
68
|
+
<!-- END GENERATED: internal.shutdown -->
|
|
69
|
+
|
|
70
|
+
<!-- BEGIN GENERATED: internal.telemetry_flush -->
|
|
71
|
+
## `POST /internal/telemetry/flush`
|
|
72
|
+
<sub></sub>
|
|
73
|
+
|
|
74
|
+
Sends every queued telemetry trace span to the configured OTLP collector, and answers once the queue is empty.
|
|
75
|
+
|
|
76
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
77
|
+
|
|
78
|
+
### Parameters
|
|
79
|
+
|
|
80
|
+
This endpoint does not take any parameters.
|
|
81
|
+
|
|
82
|
+
A batch the collector rejects or does not receive is dropped without further retries, and the answer is still `flushed`.
|
|
83
|
+
|
|
84
|
+
### Example
|
|
85
|
+
|
|
86
|
+
=== "PowerShell"
|
|
87
|
+
|
|
88
|
+
```powershell
|
|
89
|
+
Invoke-WebRequest `
|
|
90
|
+
-Uri "http://localhost:13305/internal/telemetry/flush" `
|
|
91
|
+
-Method POST
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
=== "Bash"
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
curl -X POST http://localhost:13305/internal/telemetry/flush
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
**Response:** `200 OK`
|
|
101
|
+
|
|
102
|
+
```json
|
|
103
|
+
{"status": "flushed"}
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### Response format
|
|
107
|
+
|
|
108
|
+
- `status` - Always `flushed`.
|
|
109
|
+
<!-- END GENERATED: internal.telemetry_flush -->
|
|
110
|
+
|
|
111
|
+
<!-- BEGIN GENERATED: internal.pin -->
|
|
112
|
+
## `POST /internal/pin`
|
|
113
|
+
<sub></sub>
|
|
114
|
+
|
|
115
|
+
Pins or unpins a loaded model without reloading it. Least-recently-used eviction skips pinned models.
|
|
116
|
+
|
|
117
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
118
|
+
|
|
119
|
+
### Parameters
|
|
120
|
+
|
|
121
|
+
| Parameter | Required | Description | Status |
|
|
122
|
+
|-----------|----------|-------------|--------|
|
|
123
|
+
| `model_name` or `model` | Yes | [Lemonade Server model name](https://lemonade-server.ai/models.html). | <sub></sub> |
|
|
124
|
+
| `pinned` | Yes | `true` pins the model; `false` unpins it. | <sub></sub> |
|
|
125
|
+
|
|
126
|
+
A [`POST /v1/load`](./lemonade.md#post-v1load) or inference request that needs a slot held only by pinned models fails with a `slots_pinned_error` code; see [Model Pinning](../guide/configuration/multi-model.md#model-pinning).
|
|
127
|
+
|
|
128
|
+
### Example
|
|
129
|
+
|
|
130
|
+
=== "PowerShell"
|
|
131
|
+
|
|
132
|
+
```powershell
|
|
133
|
+
Invoke-WebRequest `
|
|
134
|
+
-Uri "http://localhost:13305/internal/pin" `
|
|
135
|
+
-Method POST `
|
|
136
|
+
-Headers @{ "Content-Type" = "application/json" } `
|
|
137
|
+
-Body '{"model_name": "Qwen3-0.6B-GGUF", "pinned": true}'
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
=== "Bash"
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
curl http://localhost:13305/internal/pin \
|
|
144
|
+
-H "Content-Type: application/json" \
|
|
145
|
+
-d '{"model_name": "Qwen3-0.6B-GGUF", "pinned": true}'
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
**Response:** `200 OK`
|
|
149
|
+
|
|
150
|
+
```json
|
|
151
|
+
{"status": "success", "model_name": "Qwen3-0.6B-GGUF", "pinned": true}
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
### Response format
|
|
155
|
+
|
|
156
|
+
- `status` - Always `success`.
|
|
157
|
+
- `model_name` - The model name as sent.
|
|
158
|
+
- `pinned` - The model's pinned state now.
|
|
159
|
+
|
|
160
|
+
### Errors
|
|
161
|
+
|
|
162
|
+
| Status | Code | When |
|
|
163
|
+
|--------|------|------|
|
|
164
|
+
| `400` | `invalid_json` | The body is not a JSON object |
|
|
165
|
+
| `400` | `missing_argument` | A required argument is missing |
|
|
166
|
+
| `400` | `invalid_argument` | An argument or a value nested in it has the wrong JSON type, or a value its schema does not list |
|
|
167
|
+
| `400` | | The model is not loaded |
|
|
168
|
+
<!-- END GENERATED: internal.pin -->
|
|
169
|
+
|
|
170
|
+
<!-- BEGIN GENERATED: internal.set -->
|
|
171
|
+
## `POST /internal/set`
|
|
172
|
+
<sub></sub>
|
|
173
|
+
|
|
174
|
+
Changes one or more `config.json` settings without restarting `lemond`, and saves them. The `lemonade config set` command uses it.
|
|
175
|
+
|
|
176
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
177
|
+
|
|
178
|
+
### Parameters
|
|
179
|
+
|
|
180
|
+
| Parameter | Required | Description | Status |
|
|
181
|
+
|-----------|----------|-------------|--------|
|
|
182
|
+
| `acestep` | No | `acestep` settings; send only the keys to change. | <sub></sub> |
|
|
183
|
+
| `allowed_origins` | No | Defaults to `""`. | <sub></sub> |
|
|
184
|
+
| `auto_check_model_updates` | No | Defaults to `true`. | <sub></sub> |
|
|
185
|
+
| `auto_evict` | No | Defaults to `false`. | <sub></sub> |
|
|
186
|
+
| `auto_evict_threshold_pct` | No | Defaults to `0.9`. | <sub></sub> |
|
|
187
|
+
| `auto_update_models` | No | Defaults to `false`. | <sub></sub> |
|
|
188
|
+
| `broadcast` | No | Defaults to `true`. | <sub></sub> |
|
|
189
|
+
| `config_version` | No | Defaults to `2`. | <sub></sub> |
|
|
190
|
+
| `ctx_size` | No | Defaults to `-1`. | <sub></sub> |
|
|
191
|
+
| `default_model_source` | No | Defaults to `"huggingface"`. | <sub></sub> |
|
|
192
|
+
| `disable_model_filtering` | No | Defaults to `false`. | <sub></sub> |
|
|
193
|
+
| `download_rate_limit` | No | Defaults to `""`. | <sub></sub> |
|
|
194
|
+
| `ds4` | No | `ds4` settings; send only the keys to change. | <sub></sub> |
|
|
195
|
+
| `enable_dgpu_gtt` | No | Defaults to `false`. | <sub></sub> |
|
|
196
|
+
| `extra_models_dir` | No | Defaults to `""`. | <sub></sub> |
|
|
197
|
+
| `flm` | No | `flm` settings; send only the keys to change. | <sub></sub> |
|
|
198
|
+
| `global_timeout` | No | Defaults to `600`. | <sub></sub> |
|
|
199
|
+
| `gufo` | No | `gufo` settings; send only the keys to change. | <sub></sub> |
|
|
200
|
+
| `host` | No | Defaults to `"localhost"`. | <sub></sub> |
|
|
201
|
+
| `hrx` | No | `hrx` settings; send only the keys to change. | <sub></sub> |
|
|
202
|
+
| `inhibit_suspend` | No | Defaults to `true`. | <sub></sub> |
|
|
203
|
+
| `kokoro` | No | `kokoro` settings; send only the keys to change. | <sub></sub> |
|
|
204
|
+
| `llamacpp` | No | `llamacpp` settings; send only the keys to change. | <sub></sub> |
|
|
205
|
+
| `log_file` | No | Defaults to `"auto"`. | <sub></sub> |
|
|
206
|
+
| `log_level` | No | Defaults to `"info"`. | <sub></sub> |
|
|
207
|
+
| `log_max_file_size_mb` | No | Defaults to `10`. | <sub></sub> |
|
|
208
|
+
| `log_max_files` | No | Defaults to `5`. | <sub></sub> |
|
|
209
|
+
| `max_loaded_models` | No | Defaults to `1`. | <sub></sub> |
|
|
210
|
+
| `model_catalogs` | No | Defaults to `[]`. | <sub></sub> |
|
|
211
|
+
| `models_dir` | No | Defaults to `"auto"`. | <sub></sub> |
|
|
212
|
+
| `moonshine` | No | `moonshine` settings; send only the keys to change. | <sub></sub> |
|
|
213
|
+
| `no_fetch_executables` | No | Defaults to `false`. | <sub></sub> |
|
|
214
|
+
| `offline` | No | Defaults to `false`. | <sub></sub> |
|
|
215
|
+
| `onnxruntime` | No | `onnxruntime` settings; send only the keys to change. | <sub></sub> |
|
|
216
|
+
| `openmoss` | No | `openmoss` settings; send only the keys to change. | <sub></sub> |
|
|
217
|
+
| `port` | No | Defaults to `13305`. | <sub></sub> |
|
|
218
|
+
| `rocm_channel` | No | Defaults to `"stable"`. | <sub></sub> |
|
|
219
|
+
| `rocm_install_method` | No | Defaults to `"auto"`. | <sub></sub> |
|
|
220
|
+
| `rocmfpx` | No | `rocmfpx` settings; send only the keys to change. | <sub></sub> |
|
|
221
|
+
| `ryzenai` | No | `ryzenai` settings; send only the keys to change. | <sub></sub> |
|
|
222
|
+
| `sdcpp` | No | `sdcpp` settings; send only the keys to change. | <sub></sub> |
|
|
223
|
+
| `telemetry` | No | `telemetry` settings; send only the keys to change. | <sub></sub> |
|
|
224
|
+
| `thenoise` | No | `thenoise` settings; send only the keys to change. | <sub></sub> |
|
|
225
|
+
| `thinksound` | No | `thinksound` settings; send only the keys to change. | <sub></sub> |
|
|
226
|
+
| `trellis` | No | `trellis` settings; send only the keys to change. | <sub></sub> |
|
|
227
|
+
| `vllm` | No | `vllm` settings; send only the keys to change. | <sub></sub> |
|
|
228
|
+
| `websocket_port` | No | Defaults to `"auto"`. | <sub></sub> |
|
|
229
|
+
| `whispercpp` | No | `whispercpp` settings; send only the keys to change. | <sub></sub> |
|
|
230
|
+
|
|
231
|
+
Every key is validated before any is applied, so an unknown key or one invalid value rejects the whole request and changes nothing.
|
|
232
|
+
|
|
233
|
+
Settings with a runtime effect apply immediately, such as `port` and `host`, which rebind the listeners, the logging settings, and a backend section's `*_bin`, which reinstalls that backend and reloads its models. Other settings are read the next time they are needed, such as at the next model load. See the [Settings Reference](../guide/configuration/README.md#settings-reference) for every key.
|
|
234
|
+
|
|
235
|
+
`config.json` keeps only values that differ from the defaults, so setting a key to its default removes it from the file.
|
|
236
|
+
|
|
237
|
+
### Example
|
|
238
|
+
|
|
239
|
+
=== "PowerShell"
|
|
240
|
+
|
|
241
|
+
```powershell
|
|
242
|
+
Invoke-WebRequest `
|
|
243
|
+
-Uri "http://localhost:13305/internal/set" `
|
|
244
|
+
-Method POST `
|
|
245
|
+
-Headers @{ "Content-Type" = "application/json" } `
|
|
246
|
+
-Body '{"log_level": "info"}'
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
=== "Bash"
|
|
250
|
+
|
|
251
|
+
```bash
|
|
252
|
+
curl http://localhost:13305/internal/set \
|
|
253
|
+
-H "Content-Type: application/json" \
|
|
254
|
+
-d '{"log_level": "info"}'
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
**Response:** `200 OK`
|
|
258
|
+
|
|
259
|
+
```json
|
|
260
|
+
{"status": "success", "updated": {"log_level": "info"}}
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
### Response format
|
|
264
|
+
|
|
265
|
+
- `status` - Always `success`.
|
|
266
|
+
- `updated` - The keys the request set, with their new values, in `config.json`'s nested form: a flat key such as `llamacpp_backend` appears under `llamacpp`.
|
|
267
|
+
- `message` - Present when a deprecated value was replaced, such as `rocm_channel=preview`.
|
|
268
|
+
|
|
269
|
+
### Errors
|
|
270
|
+
|
|
271
|
+
| Status | Code | When |
|
|
272
|
+
|--------|------|------|
|
|
273
|
+
| `400` | `invalid_json` | The body is not a JSON object |
|
|
274
|
+
| `400` | `invalid_argument` | An argument or a value nested in it has the wrong JSON type, or a value its schema does not list |
|
|
275
|
+
| `400` | | The body is `{}`, names an unknown setting, or gives a setting a value outside its range or options |
|
|
276
|
+
| `500` | | The request fails unexpectedly |
|
|
277
|
+
<!-- END GENERATED: internal.set -->
|
|
278
|
+
|
|
279
|
+
<!-- BEGIN GENERATED: internal.config -->
|
|
280
|
+
## `GET /internal/config`
|
|
281
|
+
<sub></sub>
|
|
282
|
+
|
|
283
|
+
Returns the full runtime configuration: every `config.json` key, server-level and per-backend, with its current value. The `lemonade config` command reads it.
|
|
284
|
+
|
|
285
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
286
|
+
|
|
287
|
+
### Parameters
|
|
288
|
+
|
|
289
|
+
This endpoint does not take any parameters.
|
|
290
|
+
|
|
291
|
+
See the [Settings Reference](../guide/configuration/README.md#settings-reference) for each key.
|
|
292
|
+
|
|
293
|
+
### Example
|
|
294
|
+
|
|
295
|
+
=== "PowerShell"
|
|
296
|
+
|
|
297
|
+
```powershell
|
|
298
|
+
Invoke-WebRequest -Uri "http://localhost:13305/internal/config"
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
=== "Bash"
|
|
302
|
+
|
|
303
|
+
```bash
|
|
304
|
+
curl http://localhost:13305/internal/config
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
**Response:** `200 OK`
|
|
308
|
+
|
|
309
|
+
```json
|
|
310
|
+
{
|
|
311
|
+
"_generated": "GENERATED by docs/tools/gen_backend_boilerplate.py -- do not hand-edit per-recipe sections (they come from each backend's descriptor config_defaults()). Global keys are hand-maintained in this file. Regenerate and verify with that script; CI --check fails on drift.",
|
|
312
|
+
... 46 more
|
|
313
|
+
}
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
### Response format
|
|
317
|
+
|
|
318
|
+
Every `config.json` key with its current value.
|
|
319
|
+
|
|
320
|
+
- A setting; see the Settings Reference.
|
|
321
|
+
|
|
322
|
+
### Errors
|
|
323
|
+
|
|
324
|
+
| Status | When |
|
|
325
|
+
|--------|------|
|
|
326
|
+
| `500` | The request fails unexpectedly |
|
|
327
|
+
<!-- END GENERATED: internal.config -->
|
|
328
|
+
|
|
329
|
+
<!-- BEGIN GENERATED: internal.config_defaults -->
|
|
330
|
+
## `GET /internal/config/defaults`
|
|
331
|
+
<sub></sub>
|
|
332
|
+
|
|
333
|
+
Returns the default configuration built into this release, independent of this instance's `config.json` and of any deployment override. `docs/tools/gen_backend_boilerplate.py` reads it to regenerate `src/cpp/resources/defaults.json`.
|
|
334
|
+
|
|
335
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
336
|
+
|
|
337
|
+
### Parameters
|
|
338
|
+
|
|
339
|
+
This endpoint does not take any parameters.
|
|
340
|
+
|
|
341
|
+
The per-backend sections come from the backend descriptors, so this is the authoritative list of factory defaults.
|
|
342
|
+
|
|
343
|
+
### Example
|
|
344
|
+
|
|
345
|
+
=== "PowerShell"
|
|
346
|
+
|
|
347
|
+
```powershell
|
|
348
|
+
Invoke-WebRequest -Uri "http://localhost:13305/internal/config/defaults"
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
=== "Bash"
|
|
352
|
+
|
|
353
|
+
```bash
|
|
354
|
+
curl http://localhost:13305/internal/config/defaults
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
**Response:** `200 OK`
|
|
358
|
+
|
|
359
|
+
```json
|
|
360
|
+
{
|
|
361
|
+
"_generated": "GENERATED by docs/tools/gen_backend_boilerplate.py -- do not hand-edit per-recipe sections (they come from each backend's descriptor config_defaults()). Global keys are hand-maintained in this file. Regenerate and verify with that script; CI --check fails on drift.",
|
|
362
|
+
... 46 more
|
|
363
|
+
}
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
### Response format
|
|
367
|
+
|
|
368
|
+
Every `config.json` key with its factory default.
|
|
369
|
+
|
|
370
|
+
- A setting; see the Settings Reference.
|
|
371
|
+
|
|
372
|
+
### Errors
|
|
373
|
+
|
|
374
|
+
| Status | When |
|
|
375
|
+
|--------|------|
|
|
376
|
+
| `500` | `resources/defaults.json` is missing or is not valid JSON |
|
|
377
|
+
<!-- END GENERATED: internal.config_defaults -->
|
|
378
|
+
|
|
379
|
+
<!-- BEGIN GENERATED: internal.cleanup_cache -->
|
|
380
|
+
## `POST /internal/cleanup-cache`
|
|
381
|
+
<sub></sub>
|
|
382
|
+
|
|
383
|
+
Finds, and optionally removes, model files that a multi-repository model left in another repository's Hugging Face cache folder.
|
|
384
|
+
|
|
385
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
386
|
+
|
|
387
|
+
### Parameters
|
|
388
|
+
|
|
389
|
+
| Parameter | Required | Description | Status |
|
|
390
|
+
|-----------|----------|-------------|--------|
|
|
391
|
+
| `dry_run` | No | List the orphaned files without removing them. Defaults to `true`. | <sub></sub> |
|
|
392
|
+
|
|
393
|
+
An orphan is a non-`main` checkpoint file, such as an `mmproj` from a different repository, found inside the `main` checkpoint's cache folder.
|
|
394
|
+
|
|
395
|
+
### Example
|
|
396
|
+
|
|
397
|
+
=== "PowerShell"
|
|
398
|
+
|
|
399
|
+
```powershell
|
|
400
|
+
Invoke-WebRequest `
|
|
401
|
+
-Uri "http://localhost:13305/internal/cleanup-cache" `
|
|
402
|
+
-Method POST `
|
|
403
|
+
-Headers @{ "Content-Type" = "application/json" } `
|
|
404
|
+
-Body '{"dry_run": true}'
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
=== "Bash"
|
|
408
|
+
|
|
409
|
+
```bash
|
|
410
|
+
curl http://localhost:13305/internal/cleanup-cache \
|
|
411
|
+
-H "Content-Type: application/json" \
|
|
412
|
+
-d '{"dry_run": true}'
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
**Response:** `200 OK`
|
|
416
|
+
|
|
417
|
+
```json
|
|
418
|
+
{"orphaned_files": [], "total_bytes": 0, "dry_run": true}
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
### Response format
|
|
422
|
+
|
|
423
|
+
- `orphaned_files` - Each orphaned file found.
|
|
424
|
+
- `path` - Absolute path of the file.
|
|
425
|
+
- `size` - Size in bytes.
|
|
426
|
+
- `model` - Model whose checkpoint the file is.
|
|
427
|
+
- `type` - Checkpoint role, such as `mmproj`.
|
|
428
|
+
- `belongs_to` - Repository the file belongs to.
|
|
429
|
+
- `total_bytes` - Combined size of the orphaned files.
|
|
430
|
+
- `dry_run` - Whether the files were left in place.
|
|
431
|
+
|
|
432
|
+
### Errors
|
|
433
|
+
|
|
434
|
+
| Status | Code | When |
|
|
435
|
+
|--------|------|------|
|
|
436
|
+
| `400` | `invalid_json` | The body is not a JSON object |
|
|
437
|
+
| `400` | `invalid_argument` | An argument or a value nested in it has the wrong JSON type, or a value its schema does not list |
|
|
438
|
+
| `500` | `model_not_found` | Scanning or removing cache files fails |
|
|
439
|
+
<!-- END GENERATED: internal.cleanup_cache -->
|
|
440
|
+
|
|
441
|
+
<!-- BEGIN GENERATED: internal.simulate_vram_pressure -->
|
|
442
|
+
## `POST /internal/simulate-vram-pressure`
|
|
443
|
+
<sub></sub>
|
|
444
|
+
|
|
445
|
+
Hands the eviction engine a simulated global VRAM usage, so tests can exercise pressure eviction without filling a GPU.
|
|
446
|
+
|
|
447
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
448
|
+
|
|
449
|
+
### Parameters
|
|
450
|
+
|
|
451
|
+
| Parameter | Required | Description | Status |
|
|
452
|
+
|-----------|----------|-------------|--------|
|
|
453
|
+
| `pct` | No | Simulated fraction of VRAM in use, such as `0.95`. Defaults to `0`. | <sub></sub> |
|
|
454
|
+
|
|
455
|
+
Only unpinned models with `auto_evict` enabled are candidates, and a model in use is never chosen. One evaluation unloads at most one model: the first, in load order, past its `evict_idle_timeout`, or when there is none and `pct` is at or above `auto_evict_threshold_pct`, the one with the highest [eviction score](../guide/configuration/multi-model.md#dynamic-vram-management-opt-in). It also downsizes ready models past their `downsize_idle_timeout`. `-1` runs only the idle-timeout checks, and any other value below the threshold does nothing.
|
|
456
|
+
|
|
457
|
+
The answer comes after the evaluation finishes, so a test can check the outcome right away. While an exclusive job session is active no evaluation runs, and the answer is still `200`.
|
|
458
|
+
|
|
459
|
+
### Example
|
|
460
|
+
|
|
461
|
+
=== "PowerShell"
|
|
462
|
+
|
|
463
|
+
```powershell
|
|
464
|
+
Invoke-WebRequest `
|
|
465
|
+
-Uri "http://localhost:13305/internal/simulate-vram-pressure" `
|
|
466
|
+
-Method POST `
|
|
467
|
+
-Headers @{ "Content-Type" = "application/json" } `
|
|
468
|
+
-Body '{"pct": 0.5}'
|
|
469
|
+
```
|
|
470
|
+
|
|
471
|
+
=== "Bash"
|
|
472
|
+
|
|
473
|
+
```bash
|
|
474
|
+
curl http://localhost:13305/internal/simulate-vram-pressure \
|
|
475
|
+
-H "Content-Type: application/json" \
|
|
476
|
+
-d '{"pct": 0.5}'
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
**Response:** `200 OK`
|
|
480
|
+
|
|
481
|
+
```json
|
|
482
|
+
{"status": "ok"}
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
### Response format
|
|
486
|
+
|
|
487
|
+
- `status` - Always `ok`.
|
|
488
|
+
|
|
489
|
+
### Errors
|
|
490
|
+
|
|
491
|
+
| Status | Code | When |
|
|
492
|
+
|--------|------|------|
|
|
493
|
+
| `400` | `invalid_json` | The body is not a JSON object |
|
|
494
|
+
| `400` | `invalid_argument` | An argument or a value nested in it has the wrong JSON type, or a value its schema does not list |
|
|
495
|
+
<!-- END GENERATED: internal.simulate_vram_pressure -->
|
|
496
|
+
|
|
497
|
+
<!-- BEGIN GENERATED: internal.models_sync -->
|
|
498
|
+
## `POST /internal/models/sync`
|
|
499
|
+
<sub></sub>
|
|
500
|
+
|
|
501
|
+
Checks downloaded models for newer upstream commits and downloads the updates. The `lemonade update-models` command uses it.
|
|
502
|
+
|
|
503
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
504
|
+
|
|
505
|
+
### Parameters
|
|
506
|
+
|
|
507
|
+
| Parameter | Required | Description | Status |
|
|
508
|
+
|-----------|----------|-------------|--------|
|
|
509
|
+
| `models` | No | Models to sync. Defaults to every downloaded model. | <sub></sub> |
|
|
510
|
+
| `dry_run` | No | `true` reports what the request would do without doing it. Defaults to `false`. | <sub></sub> |
|
|
511
|
+
| `async` | No | Run in the background and answer immediately. Defaults to `true`. | <sub></sub> |
|
|
512
|
+
| `attach_if_running` | No | `true` reports a running sync, or with `async: false` waits for it, without adding this request's models to it. Defaults to `false`, which queues the models into the running sync. | <sub></sub> |
|
|
513
|
+
|
|
514
|
+
A background sync answers `202` with its status; poll [`GET /internal/models/sync/status`](#get-internalmodelssyncstatus) with its `sync_id`.
|
|
515
|
+
|
|
516
|
+
A `dry_run` finishes before answering, whatever `async` says, and answers `200` with what a sync would update.
|
|
517
|
+
|
|
518
|
+
While a sync is running, a new request answers `200` with that sync's status instead of starting another.
|
|
519
|
+
|
|
520
|
+
See Model Synchronization & Auto-Updates in [Server Configuration](../guide/configuration/README.md).
|
|
521
|
+
|
|
522
|
+
### Example
|
|
523
|
+
|
|
524
|
+
=== "PowerShell"
|
|
525
|
+
|
|
526
|
+
```powershell
|
|
527
|
+
Invoke-WebRequest `
|
|
528
|
+
-Uri "http://localhost:13305/internal/models/sync" `
|
|
529
|
+
-Method POST `
|
|
530
|
+
-Headers @{ "Content-Type" = "application/json" } `
|
|
531
|
+
-Body '{"models": ["Qwen3-0.6B-GGUF"], "dry_run": true}'
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
=== "Bash"
|
|
535
|
+
|
|
536
|
+
```bash
|
|
537
|
+
curl http://localhost:13305/internal/models/sync \
|
|
538
|
+
-H "Content-Type: application/json" \
|
|
539
|
+
-d '{"models": ["Qwen3-0.6B-GGUF"], "dry_run": true}'
|
|
540
|
+
```
|
|
541
|
+
|
|
542
|
+
**Response:** `200 OK`
|
|
543
|
+
|
|
544
|
+
```json
|
|
545
|
+
{
|
|
546
|
+
"status": "success",
|
|
547
|
+
"sync_id": 0,
|
|
548
|
+
"completed_sync_id": 0,
|
|
549
|
+
"already_in_progress": false,
|
|
550
|
+
"dry_run": true,
|
|
551
|
+
"models_updated": [],
|
|
552
|
+
"models_up_to_date": ["Qwen3-0.6B-GGUF"],
|
|
553
|
+
"failed_models": {},
|
|
554
|
+
"checked_count": 1,
|
|
555
|
+
"updated_count": 0
|
|
556
|
+
}
|
|
557
|
+
```
|
|
558
|
+
|
|
559
|
+
### Response format
|
|
560
|
+
|
|
561
|
+
- `status` - `idle` before any sync, and after a sync other than a dry run that found no models to check; `in_progress` while one runs; `success` or `failed` once it finishes; and `not_found` for an unknown `sync_id`.
|
|
562
|
+
- `sync_id` - The sync this status describes; `0` before any sync.
|
|
563
|
+
- `completed_sync_id` - The most recent sync that finished.
|
|
564
|
+
- `already_in_progress` - Whether a sync this request did not start is running.
|
|
565
|
+
- `dry_run` - `true` when the request was a dry run.
|
|
566
|
+
- `async` - `true` when the sync was started in the background.
|
|
567
|
+
- `message` - Present when the sync was started in the background.
|
|
568
|
+
- `is_full_sync` - Whether the sync covers every downloaded model. Absent in a dry run, and when `status` is `not_found`.
|
|
569
|
+
- `active_targets` - Models being updated now. Absent in a dry run, and when `status` is `not_found`.
|
|
570
|
+
- `pending_targets` - Models queued for the running sync and not yet started. Absent in a dry run, and when `status` is `not_found`.
|
|
571
|
+
- `completed_targets` - Models the sync has updated. Absent in a dry run, and when `status` is `not_found`.
|
|
572
|
+
- `models_updated` - Models with an update; downloaded unless `dry_run`. Absent when `status` is `not_found`.
|
|
573
|
+
- `models_up_to_date` - Models already at the latest upstream commit; absent when `status` is `not_found`.
|
|
574
|
+
- `failed_models` - Error message per model that could not be checked or updated; absent when `status` is `not_found`.
|
|
575
|
+
- `terminal_error` - Error that stopped the sync; empty when none did. Absent in a dry run, and when `status` is `not_found`.
|
|
576
|
+
- `checked_count` - Number of models checked for updates; absent when `status` is `not_found`.
|
|
577
|
+
- `updated_count` - Number of models in `models_updated`; absent when `status` is `not_found`.
|
|
578
|
+
- `progress` - While a sync runs: the file downloading now.
|
|
579
|
+
- `current_model` - The model being updated.
|
|
580
|
+
- `file` - The file downloading now.
|
|
581
|
+
- `file_index` - Its position among the model's files, from `1`.
|
|
582
|
+
- `total_files` - How many files the model has.
|
|
583
|
+
- `bytes_downloaded` - Bytes of the current file downloaded so far.
|
|
584
|
+
- `bytes_total` - Size of the current file in bytes.
|
|
585
|
+
- `percent` - Share of the current file downloaded, from `0` to `100`.
|
|
586
|
+
|
|
587
|
+
### Errors
|
|
588
|
+
|
|
589
|
+
| Status | Code | When |
|
|
590
|
+
|--------|------|------|
|
|
591
|
+
| `400` | `invalid_json` | The body is not a JSON object |
|
|
592
|
+
| `400` | `invalid_argument` | An argument or a value nested in it has the wrong JSON type, or a value its schema does not list |
|
|
593
|
+
| `409` | | `offline` is set to `true`; no network request is made |
|
|
594
|
+
| `500` | | The request fails unexpectedly |
|
|
595
|
+
<!-- END GENERATED: internal.models_sync -->
|
|
596
|
+
|
|
597
|
+
<!-- BEGIN GENERATED: internal.models_sync_status -->
|
|
598
|
+
## `GET /internal/models/sync/status`
|
|
599
|
+
<sub></sub>
|
|
600
|
+
|
|
601
|
+
Returns the status of a model sync started by [`POST /internal/models/sync`](#post-internalmodelssync).
|
|
602
|
+
|
|
603
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
604
|
+
|
|
605
|
+
### Parameters
|
|
606
|
+
|
|
607
|
+
| Parameter | Required | Description | Status |
|
|
608
|
+
|-----------|----------|-------------|--------|
|
|
609
|
+
| `sync_id` (query) | No | Sync to report. Defaults to the latest. For a sync that has finished, the answer is the status recorded when it finished, `completed_sync_id` and `already_in_progress` included. | <sub></sub> |
|
|
610
|
+
|
|
611
|
+
A `sync_id` that is neither running nor recorded answers `200` with `status: "not_found"`.
|
|
612
|
+
|
|
613
|
+
### Example
|
|
614
|
+
|
|
615
|
+
=== "PowerShell"
|
|
616
|
+
|
|
617
|
+
```powershell
|
|
618
|
+
Invoke-WebRequest -Uri "http://localhost:13305/internal/models/sync/status"
|
|
619
|
+
```
|
|
620
|
+
|
|
621
|
+
=== "Bash"
|
|
622
|
+
|
|
623
|
+
```bash
|
|
624
|
+
curl http://localhost:13305/internal/models/sync/status
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
**Response:** `200 OK`
|
|
628
|
+
|
|
629
|
+
```json
|
|
630
|
+
{
|
|
631
|
+
"status": "idle",
|
|
632
|
+
"sync_id": 0,
|
|
633
|
+
"completed_sync_id": 0,
|
|
634
|
+
"already_in_progress": false,
|
|
635
|
+
"is_full_sync": false,
|
|
636
|
+
"active_targets": [],
|
|
637
|
+
"pending_targets": [],
|
|
638
|
+
"completed_targets": [],
|
|
639
|
+
"models_updated": [],
|
|
640
|
+
"models_up_to_date": [],
|
|
641
|
+
"failed_models": {},
|
|
642
|
+
"terminal_error": "",
|
|
643
|
+
"checked_count": 0,
|
|
644
|
+
"updated_count": 0
|
|
645
|
+
}
|
|
646
|
+
```
|
|
647
|
+
|
|
648
|
+
### Response format
|
|
649
|
+
|
|
650
|
+
- `status` - `idle` before any sync, and after a sync other than a dry run that found no models to check; `in_progress` while one runs; `success` or `failed` once it finishes; and `not_found` for an unknown `sync_id`.
|
|
651
|
+
- `sync_id` - The sync this status describes; `0` before any sync.
|
|
652
|
+
- `completed_sync_id` - The most recent sync that finished.
|
|
653
|
+
- `already_in_progress` - Whether a sync this request did not start is running.
|
|
654
|
+
- `dry_run` - `true` when the request was a dry run.
|
|
655
|
+
- `async` - `true` when the sync was started in the background.
|
|
656
|
+
- `message` - Present when the sync was started in the background.
|
|
657
|
+
- `is_full_sync` - Whether the sync covers every downloaded model. Absent in a dry run, and when `status` is `not_found`.
|
|
658
|
+
- `active_targets` - Models being updated now. Absent in a dry run, and when `status` is `not_found`.
|
|
659
|
+
- `pending_targets` - Models queued for the running sync and not yet started. Absent in a dry run, and when `status` is `not_found`.
|
|
660
|
+
- `completed_targets` - Models the sync has updated. Absent in a dry run, and when `status` is `not_found`.
|
|
661
|
+
- `models_updated` - Models with an update; downloaded unless `dry_run`. Absent when `status` is `not_found`.
|
|
662
|
+
- `models_up_to_date` - Models already at the latest upstream commit; absent when `status` is `not_found`.
|
|
663
|
+
- `failed_models` - Error message per model that could not be checked or updated; absent when `status` is `not_found`.
|
|
664
|
+
- `terminal_error` - Error that stopped the sync; empty when none did. Absent in a dry run, and when `status` is `not_found`.
|
|
665
|
+
- `checked_count` - Number of models checked for updates; absent when `status` is `not_found`.
|
|
666
|
+
- `updated_count` - Number of models in `models_updated`; absent when `status` is `not_found`.
|
|
667
|
+
- `progress` - While a sync runs: the file downloading now.
|
|
668
|
+
- `current_model` - The model being updated.
|
|
669
|
+
- `file` - The file downloading now.
|
|
670
|
+
- `file_index` - Its position among the model's files, from `1`.
|
|
671
|
+
- `total_files` - How many files the model has.
|
|
672
|
+
- `bytes_downloaded` - Bytes of the current file downloaded so far.
|
|
673
|
+
- `bytes_total` - Size of the current file in bytes.
|
|
674
|
+
- `percent` - Share of the current file downloaded, from `0` to `100`.
|
|
675
|
+
|
|
676
|
+
### Errors
|
|
677
|
+
|
|
678
|
+
| Status | When |
|
|
679
|
+
|--------|------|
|
|
680
|
+
| `400` | `sync_id` is too large for 64 bits |
|
|
681
|
+
| `500` | The request fails unexpectedly |
|
|
682
|
+
<!-- END GENERATED: internal.models_sync_status -->
|
|
683
|
+
|
|
684
|
+
<!-- BEGIN GENERATED: internal.aliases_list -->
|
|
685
|
+
## `GET /internal/aliases`
|
|
686
|
+
<sub></sub>
|
|
687
|
+
|
|
688
|
+
Lists every model alias with its target.
|
|
689
|
+
|
|
690
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
691
|
+
|
|
692
|
+
### Parameters
|
|
693
|
+
|
|
694
|
+
This endpoint does not take any parameters.
|
|
695
|
+
|
|
696
|
+
An alias stands in for its target in the `model` field of inference requests and of [`POST /v1/load`](./lemonade.md#post-v1load) and [`POST /v1/unload`](./lemonade.md#post-v1unload); the Ollama and Anthropic gateways do not resolve aliases. Aliases live in `<cache_dir>/aliases.json`; see [Model Aliases](../guide/configuration/custom-models.md).
|
|
697
|
+
|
|
698
|
+
### Example
|
|
699
|
+
|
|
700
|
+
=== "PowerShell"
|
|
701
|
+
|
|
702
|
+
```powershell
|
|
703
|
+
Invoke-WebRequest -Uri "http://localhost:13305/internal/aliases"
|
|
704
|
+
```
|
|
705
|
+
|
|
706
|
+
=== "Bash"
|
|
707
|
+
|
|
708
|
+
```bash
|
|
709
|
+
curl http://localhost:13305/internal/aliases
|
|
710
|
+
```
|
|
711
|
+
|
|
712
|
+
**Response:** `200 OK`
|
|
713
|
+
|
|
714
|
+
```json
|
|
715
|
+
{
|
|
716
|
+
"aliases": [
|
|
717
|
+
{
|
|
718
|
+
"alias": "my-alias-1",
|
|
719
|
+
"target": "user.custom-llama",
|
|
720
|
+
"downloaded": false,
|
|
721
|
+
"recipe": "llamacpp"
|
|
722
|
+
}
|
|
723
|
+
]
|
|
724
|
+
}
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
### Response format
|
|
728
|
+
|
|
729
|
+
- `aliases` - Each alias with its target.
|
|
730
|
+
- `alias` - The alias name.
|
|
731
|
+
- `target` - Model or alias the alias points to.
|
|
732
|
+
- `downloaded` - Whether the target model is downloaded.
|
|
733
|
+
- `recipe` - The target model's recipe; `llamacpp` when the target is not in the registry.
|
|
734
|
+
|
|
735
|
+
### Errors
|
|
736
|
+
|
|
737
|
+
| Status | Code | When |
|
|
738
|
+
|--------|------|------|
|
|
739
|
+
| `500` | `internal_error` | The alias manager is not initialized, or reading or checking the alias file fails |
|
|
740
|
+
<!-- END GENERATED: internal.aliases_list -->
|
|
741
|
+
|
|
742
|
+
<!-- BEGIN GENERATED: internal.aliases_create -->
|
|
743
|
+
## `POST /internal/aliases`
|
|
744
|
+
<sub></sub>
|
|
745
|
+
|
|
746
|
+
Points an alias at a target model, replacing any target it had. The `lemonade alias add` command uses it.
|
|
747
|
+
|
|
748
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
749
|
+
|
|
750
|
+
### Parameters
|
|
751
|
+
|
|
752
|
+
| Parameter | Required | Description | Status |
|
|
753
|
+
|-----------|----------|-------------|--------|
|
|
754
|
+
| `alias` | Yes | The alias name to create or update. | <sub></sub> |
|
|
755
|
+
| `target` or `model` | Yes | The model name, canonical ID or other alias to point at, which is not checked to exist. | <sub></sub> |
|
|
756
|
+
|
|
757
|
+
### Example
|
|
758
|
+
|
|
759
|
+
=== "PowerShell"
|
|
760
|
+
|
|
761
|
+
```powershell
|
|
762
|
+
Invoke-WebRequest `
|
|
763
|
+
-Uri "http://localhost:13305/internal/aliases" `
|
|
764
|
+
-Method POST `
|
|
765
|
+
-Headers @{ "Content-Type" = "application/json" } `
|
|
766
|
+
-Body '{"alias": "my-alias-1", "target": "user.custom-llama"}'
|
|
767
|
+
```
|
|
768
|
+
|
|
769
|
+
=== "Bash"
|
|
770
|
+
|
|
771
|
+
```bash
|
|
772
|
+
curl http://localhost:13305/internal/aliases \
|
|
773
|
+
-H "Content-Type: application/json" \
|
|
774
|
+
-d '{"alias": "my-alias-1", "target": "user.custom-llama"}'
|
|
775
|
+
```
|
|
776
|
+
|
|
777
|
+
**Response:** `200 OK`
|
|
778
|
+
|
|
779
|
+
```json
|
|
780
|
+
{"status": "ok", "alias": "my-alias-1", "target": "user.custom-llama"}
|
|
781
|
+
```
|
|
782
|
+
|
|
783
|
+
### Response format
|
|
784
|
+
|
|
785
|
+
- `status` - Always `ok`.
|
|
786
|
+
- `alias` - The alias name.
|
|
787
|
+
- `target` - Model or alias the alias now points to.
|
|
788
|
+
|
|
789
|
+
### Errors
|
|
790
|
+
|
|
791
|
+
| Status | Code | When |
|
|
792
|
+
|--------|------|------|
|
|
793
|
+
| `400` | `invalid_json` | The body is not a JSON object |
|
|
794
|
+
| `400` | `missing_argument` | A required argument is missing |
|
|
795
|
+
| `400` | `invalid_argument` | An argument or a value nested in it has the wrong JSON type, or a value its schema does not list |
|
|
796
|
+
| `400` | `invalid_request` | `alias` or `target` is blank |
|
|
797
|
+
| `400` | `invalid_alias` | `alias` equals `target` |
|
|
798
|
+
| `400` | `invalid_alias` | `alias` starts with `user.`, `extra.` or `builtin.` |
|
|
799
|
+
| `409` | `alias_conflict` | `alias` is already a registered model's name |
|
|
800
|
+
| `409` | `alias_conflict` | The alias would form a cycle or an alias chain deeper than 10 links |
|
|
801
|
+
| `500` | `internal_error` | The alias manager is not initialized, or reading or checking the alias file fails |
|
|
802
|
+
<!-- END GENERATED: internal.aliases_create -->
|
|
803
|
+
|
|
804
|
+
<!-- BEGIN GENERATED: internal.aliases_delete -->
|
|
805
|
+
## `DELETE /internal/aliases/{alias}`
|
|
806
|
+
<sub></sub>
|
|
807
|
+
|
|
808
|
+
Removes a model alias.
|
|
809
|
+
|
|
810
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
811
|
+
|
|
812
|
+
### Parameters
|
|
813
|
+
|
|
814
|
+
| Parameter | Required | Description | Status |
|
|
815
|
+
|-----------|----------|-------------|--------|
|
|
816
|
+
| `alias` (path) | Yes | The alias name to remove. | <sub></sub> |
|
|
817
|
+
|
|
818
|
+
### Example
|
|
819
|
+
|
|
820
|
+
=== "PowerShell"
|
|
821
|
+
|
|
822
|
+
```powershell
|
|
823
|
+
Invoke-WebRequest `
|
|
824
|
+
-Uri "http://localhost:13305/internal/aliases/my-alias-1" `
|
|
825
|
+
-Method DELETE
|
|
826
|
+
```
|
|
827
|
+
|
|
828
|
+
=== "Bash"
|
|
829
|
+
|
|
830
|
+
```bash
|
|
831
|
+
curl -X DELETE http://localhost:13305/internal/aliases/my-alias-1
|
|
832
|
+
```
|
|
833
|
+
|
|
834
|
+
**Response:** `200 OK`
|
|
835
|
+
|
|
836
|
+
```json
|
|
837
|
+
{"status": "deleted", "alias": "my-alias-1"}
|
|
838
|
+
```
|
|
839
|
+
|
|
840
|
+
### Response format
|
|
841
|
+
|
|
842
|
+
- `status` - Always `deleted`.
|
|
843
|
+
- `alias` - The alias that was removed.
|
|
844
|
+
|
|
845
|
+
### Errors
|
|
846
|
+
|
|
847
|
+
| Status | Code | When |
|
|
848
|
+
|--------|------|------|
|
|
849
|
+
| `404` | `alias_not_found` | `alias` names no alias |
|
|
850
|
+
| `500` | `internal_error` | The alias manager is not initialized, or reading or checking the alias file fails |
|
|
851
|
+
<!-- END GENERATED: internal.aliases_delete -->
|
|
852
|
+
|
|
853
|
+
<!-- BEGIN GENERATED: internal.routes -->
|
|
854
|
+
## `GET /internal/routes`
|
|
855
|
+
<sub></sub>
|
|
856
|
+
|
|
857
|
+
Returns the specification of every HTTP route `lemond` serves, in registration order: its URLs, arguments, the schema of each response format, and its example requests.
|
|
858
|
+
|
|
859
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
860
|
+
|
|
861
|
+
### Parameters
|
|
862
|
+
|
|
863
|
+
| Parameter | Required | Description | Status |
|
|
864
|
+
|-----------|----------|-------------|--------|
|
|
865
|
+
| `id` (query) | No | Return only the route with this id, such as `lemonade.health`. | <sub></sub> |
|
|
866
|
+
|
|
867
|
+
`docs/tools/gen_api_boilerplate.py` reads this endpoint to generate the API reference, including this page. See the [HTTP Server Spec](../dev/specs/http-server.md#route-base-classes) for each field.
|
|
868
|
+
|
|
869
|
+
### Example
|
|
870
|
+
|
|
871
|
+
=== "PowerShell"
|
|
872
|
+
|
|
873
|
+
```powershell
|
|
874
|
+
Invoke-WebRequest -Uri "http://localhost:13305/internal/routes?id=internal.telemetry_flush"
|
|
875
|
+
```
|
|
876
|
+
|
|
877
|
+
=== "Bash"
|
|
878
|
+
|
|
879
|
+
```bash
|
|
880
|
+
curl "http://localhost:13305/internal/routes?id=internal.telemetry_flush"
|
|
881
|
+
```
|
|
882
|
+
|
|
883
|
+
**Response:** `200 OK`
|
|
884
|
+
|
|
885
|
+
```json
|
|
886
|
+
[
|
|
887
|
+
{
|
|
888
|
+
"id": "internal.telemetry_flush",
|
|
889
|
+
"methods": ["POST"],
|
|
890
|
+
"paths": ["telemetry/flush"],
|
|
891
|
+
"prefixes": "Internal",
|
|
892
|
+
"summary": "Force-flush all queued telemetry trace spans",
|
|
893
|
+
"modality": "",
|
|
894
|
+
"description": "Sends every queued telemetry trace span to the configured OTLP collector, and answers once the queue is empty.",
|
|
895
|
+
"notes": [
|
|
896
|
+
"A batch the collector rejects or does not receive is dropped without further retries, and the answer is still `flushed`."
|
|
897
|
+
],
|
|
898
|
+
"experimental": false,
|
|
899
|
+
"args": [],
|
|
900
|
+
"responses": [
|
|
901
|
+
{
|
|
902
|
+
"format": "Json",
|
|
903
|
+
"schema": {
|
|
904
|
+
"type": "object",
|
|
905
|
+
"required": ["status"],
|
|
906
|
+
"properties": {"status": {"const": "flushed", "description": "Always `flushed`."}}
|
|
907
|
+
}
|
|
908
|
+
}
|
|
909
|
+
],
|
|
910
|
+
"examples": [{"name": "", "description": "", "format": "Json", "args": {}, "setup": []}],
|
|
911
|
+
"errors": [],
|
|
912
|
+
"error_format": "Plain",
|
|
913
|
+
"request_format": "Raw",
|
|
914
|
+
"quiet_log": false
|
|
915
|
+
}
|
|
916
|
+
]
|
|
917
|
+
```
|
|
918
|
+
|
|
919
|
+
### Response format
|
|
920
|
+
|
|
921
|
+
- `id` - `<page>.<name>`; names the route's generated region in `docs/api/<page>.md`.
|
|
922
|
+
- `methods` - HTTP methods the route answers, such as `POST`.
|
|
923
|
+
- `paths` - Paths without their prefix; entries after the first are aliases.
|
|
924
|
+
- `prefixes` - `Quad` serves each path under `/v1/` and older prefixes kept for existing clients; `Internal` under `/internal/`; `Root` as written.
|
|
925
|
+
- `summary` - One-line summary for the page's summary table; empty when hand-written rows replace it.
|
|
926
|
+
- `modality` - The route's input and output for the summary table's Modality column, such as `messages -> completion`; empty on pages without that column.
|
|
927
|
+
- `description` - Reference text for the route's section.
|
|
928
|
+
- `notes` - Sections the docs place after the Parameters table.
|
|
929
|
+
- `experimental` - The route may still change.
|
|
930
|
+
- `args` - Every argument the route takes.
|
|
931
|
+
- `name` - The argument's name.
|
|
932
|
+
- `in` - Where the argument goes: the JSON body, the query string, the path, or a form field.
|
|
933
|
+
- `schema` - JSON Schema of the argument's value.
|
|
934
|
+
- `required` - Whether a request must send the argument.
|
|
935
|
+
- `supported` - `available`; `partial`, honored only in part as the argument's description says; or `not_available`, accepted but ignored.
|
|
936
|
+
- `description` - What the argument controls.
|
|
937
|
+
- `aliases` - Other names a request may send the argument under.
|
|
938
|
+
- `forwarded` - Whether `lemond` passes the argument to the backend unchecked.
|
|
939
|
+
- `responses` - One entry per response format the route returns.
|
|
940
|
+
- `format` - The response format.
|
|
941
|
+
- `schema` - JSON Schema of the body, each NDJSON line, or each event's data; null for other formats.
|
|
942
|
+
- `examples` - The first is the route's main example.
|
|
943
|
+
- `name` - Subsection heading; empty when the route has one example.
|
|
944
|
+
- `description` - What the example shows, opening its subsection; empty when the route's other text covers it.
|
|
945
|
+
- `format` - The response format the reply is validated against.
|
|
946
|
+
- `args` - Argument values of the request.
|
|
947
|
+
- `setup` - Requests that run first to put `lemond` in the state this example needs.
|
|
948
|
+
- `route` - The `id` of the route the step requests.
|
|
949
|
+
- `example` - The name of the route's example to run.
|
|
950
|
+
- `args` - Set instead of example: a request sent as is and left out of the docs.
|
|
951
|
+
- `errors` - Every error status the route answers with, including those its base class adds.
|
|
952
|
+
- `status` - The HTTP status.
|
|
953
|
+
- `code` - The code the error body carries, or empty when it carries none.
|
|
954
|
+
- `when` - The condition that produces the error.
|
|
955
|
+
- `error_format` - The body shape of the errors `lemond` writes when a request is not valid JSON or an argument is missing or invalid.
|
|
956
|
+
- `request_format` - How `lemond` parses the request body: `Raw` not at all, `Json` as JSON, `OptionalJson` as JSON with an empty or `null` body read as `{}`, `Form` as a multipart form.
|
|
957
|
+
- `quiet_log` - Whether the route is left out of the access log.
|
|
958
|
+
<!-- END GENERATED: internal.routes -->
|
|
959
|
+
|
|
960
|
+
<!-- BEGIN GENERATED: internal.model_sources -->
|
|
961
|
+
## `GET /internal/model-sources`
|
|
962
|
+
<sub></sub>
|
|
963
|
+
|
|
964
|
+
Returns each remote registry's and each model catalog's compiled descriptor: its token, aliases, documentation text, credential environment variables, cache layout and endpoint defaults.
|
|
965
|
+
|
|
966
|
+
Requires `LEMONADE_ADMIN_API_KEY` when it is set, otherwise `LEMONADE_API_KEY` when that is set.
|
|
967
|
+
|
|
968
|
+
### Parameters
|
|
969
|
+
|
|
970
|
+
This endpoint does not take any parameters.
|
|
971
|
+
|
|
972
|
+
`docs/tools/gen_model_source_boilerplate.py` copies the model-source tables from this response, so every token, endpoint and environment variable a committed page names is one the server answering this request actually has.
|
|
973
|
+
|
|
974
|
+
A catalog answers `host_specific: true` when its rows depend on this machine: a configured `model_catalogs` entry, the models an installed backend reports, and a cloud provider's listing. The generator writes no such row to a committed page, so the page stays the same on every machine even though this response is not.
|
|
975
|
+
|
|
976
|
+
### Example
|
|
977
|
+
|
|
978
|
+
=== "PowerShell"
|
|
979
|
+
|
|
980
|
+
```powershell
|
|
981
|
+
Invoke-WebRequest -Uri "http://localhost:13305/internal/model-sources"
|
|
982
|
+
```
|
|
983
|
+
|
|
984
|
+
=== "Bash"
|
|
985
|
+
|
|
986
|
+
```bash
|
|
987
|
+
curl http://localhost:13305/internal/model-sources
|
|
988
|
+
```
|
|
989
|
+
|
|
990
|
+
**Response:** `200 OK`
|
|
991
|
+
|
|
992
|
+
```json
|
|
993
|
+
{
|
|
994
|
+
"registries": [
|
|
995
|
+
{
|
|
996
|
+
"token": "huggingface",
|
|
997
|
+
"aliases": ["huggingface", "hf", ""],
|
|
998
|
+
"display_name": "Hugging Face",
|
|
999
|
+
"title": "Hugging Face",
|
|
1000
|
+
"summary": "The default remote registry: GGUF repositories over HTTPS.",
|
|
1001
|
+
"notes": [
|
|
1002
|
+
"A token-gated repository needs `HF_TOKEN`.",
|
|
1003
|
+
"`HF_ENDPOINT` moves the API to a mirror or gateway; the cache layout does not change."
|
|
1004
|
+
],
|
|
1005
|
+
"example": "https://huggingface.co/{repo}",
|
|
1006
|
+
"endpoint_default": "https://huggingface.co",
|
|
1007
|
+
"scheme": "hf://",
|
|
1008
|
+
"cache_dir_prefix": "models--",
|
|
1009
|
+
"default_revision": "main",
|
|
1010
|
+
"revision_semantics": "branch, tag or commit SHA",
|
|
1011
|
+
"staleness_basis": "the commit SHA the API reports for the revision",
|
|
1012
|
+
"download_protocol": "hf-hub",
|
|
1013
|
+
"search": true,
|
|
1014
|
+
"env": [
|
|
1015
|
+
{"name": "HF_TOKEN", "purpose": "read access to gated repositories", "required": false},
|
|
1016
|
+
... 1 more
|
|
1017
|
+
]
|
|
1018
|
+
},
|
|
1019
|
+
... 1 more
|
|
1020
|
+
],
|
|
1021
|
+
"catalogs": [
|
|
1022
|
+
{
|
|
1023
|
+
"id": "server_models.json",
|
|
1024
|
+
"title": "Built-in registry",
|
|
1025
|
+
"summary": "The model list shipped with the server.",
|
|
1026
|
+
"notes": ["Nothing writes this file back: a new row ships in a release."],
|
|
1027
|
+
"example": "",
|
|
1028
|
+
"required": false,
|
|
1029
|
+
"config_key": "",
|
|
1030
|
+
"host_specific": false
|
|
1031
|
+
},
|
|
1032
|
+
... 3 more
|
|
1033
|
+
],
|
|
1034
|
+
"source_tokens": ["huggingface", "modelscope"],
|
|
1035
|
+
"local_origin_tokens": ["local_upload", "local_path", "extra_models_dir"],
|
|
1036
|
+
"pull_source_values": [
|
|
1037
|
+
"huggingface",
|
|
1038
|
+
"modelscope",
|
|
1039
|
+
"local_upload",
|
|
1040
|
+
"local_path",
|
|
1041
|
+
"extra_models_dir"
|
|
1042
|
+
]
|
|
1043
|
+
}
|
|
1044
|
+
```
|
|
1045
|
+
|
|
1046
|
+
### Response format
|
|
1047
|
+
|
|
1048
|
+
- `registries` - One entry per remote registry the server can download from.
|
|
1049
|
+
- `token` - The registry's canonical name, as `source` and a checkpoint URL spell it.
|
|
1050
|
+
- `aliases` - Every name a request may send for this registry, its canonical token among them. Hugging Face answers the empty string too, which is what a checkpoint naming no registry spells.
|
|
1051
|
+
- `display_name` - The registry's name in prose and error messages.
|
|
1052
|
+
- `title` - Heading the generated docs give this registry.
|
|
1053
|
+
- `summary` - The registry's one-line description.
|
|
1054
|
+
- `notes` - Anything the generated page says about this registry beyond the table row.
|
|
1055
|
+
- `example` - A checkpoint the docs show for this registry.
|
|
1056
|
+
- `endpoint_default` - Plain download host used when no environment variable overrides it; empty where the host serves files only through its own API, which the generated page prints instead.
|
|
1057
|
+
- `scheme` - URL scheme a checkpoint URL for this registry carries, such as `hf://`; empty when a plain URL names it.
|
|
1058
|
+
- `cache_dir_prefix` - Prefix of this registry's directory under the models cache.
|
|
1059
|
+
- `default_revision` - Revision assumed when a request names none.
|
|
1060
|
+
- `revision_semantics` - What a revision names here, such as `branch, tag or commit SHA`.
|
|
1061
|
+
- `staleness_basis` - How the server decides a download is out of date.
|
|
1062
|
+
- `download_protocol` - How a file is fetched from this registry.
|
|
1063
|
+
- `search` - Whether [`/v1/registry/search`](./lemonade.md#get-v1registrysearch) answers for this registry.
|
|
1064
|
+
- `env` - Environment variables that configure this registry.
|
|
1065
|
+
- `name` - Variable name.
|
|
1066
|
+
- `purpose` - What it sets.
|
|
1067
|
+
- `required` - Whether the registry is unusable without it.
|
|
1068
|
+
- `catalogs` - One entry per model catalog the server reads.
|
|
1069
|
+
- `id` - The catalog's id: the name a `model_catalogs` entry gives it, or a fixed one such as `server_models.json`, `user_models.json`, `extra_models_dir`, `discovered:<recipe>` or `listed:<provider>`.
|
|
1070
|
+
- `title` - Heading the generated docs give this catalog.
|
|
1071
|
+
- `summary` - The catalog's one-line description.
|
|
1072
|
+
- `notes` - Anything the generated page says about this catalog beyond the table row.
|
|
1073
|
+
- `example` - A configuration example the docs show.
|
|
1074
|
+
- `required` - Whether a model set missing this catalog invalidates the set.
|
|
1075
|
+
- `config_key` - The config.json key that configures it: `model_catalogs` for a configured catalog, the top-level `extra_models_dir` for the scanned directory; empty when nothing configures it.
|
|
1076
|
+
- `host_specific` - `true` when its rows depend on this machine, which keeps its rows out of committed pages.
|
|
1077
|
+
- `source_tokens` - Every registry token, in the order the registries are listed.
|
|
1078
|
+
- `local_origin_tokens` - Tokens naming where a model came from without a registry, such as `local_upload`.
|
|
1079
|
+
- `pull_source_values` - Every value [`/v1/pull`](./lemonade.md#post-v1pull) accepts for `source`: each registry token and each local origin.
|
|
1080
|
+
<!-- END GENERATED: internal.model_sources -->
|