metrillm 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -2
- package/dist/index.mjs +860 -289
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -114,19 +114,36 @@ By default, production builds upload shared results to the official MetriLLM lea
|
|
|
114
114
|
|
|
115
115
|
If these variables are set to placeholder values (from templates), MetriLLM falls back to official defaults.
|
|
116
116
|
|
|
117
|
+
## Windows Users
|
|
118
|
+
|
|
119
|
+
PowerShell's default execution policy blocks npm global scripts. If you see `PSSecurityException` or `UnauthorizedAccess` when running `metrillm`, run this once:
|
|
120
|
+
|
|
121
|
+
```powershell
|
|
122
|
+
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Alternatively, use `npx metrillm` which bypasses the issue entirely.
|
|
126
|
+
|
|
117
127
|
## Runtime Backends
|
|
118
128
|
|
|
119
129
|
| Backend | Flag | Default URL | Required env |
|
|
120
130
|
|---|---|---|---|
|
|
121
131
|
| Ollama | `--backend ollama` | `http://127.0.0.1:11434` | `OLLAMA_HOST` (optional) |
|
|
122
|
-
| LM Studio | `--backend lm-studio` | `http://127.0.0.1:1234` | `LM_STUDIO_BASE_URL` (optional), `LM_STUDIO_API_KEY` (optional)
|
|
132
|
+
| LM Studio | `--backend lm-studio` | `http://127.0.0.1:1234` | `LM_STUDIO_BASE_URL` (optional), `LM_STUDIO_API_KEY` (optional) |
|
|
133
|
+
|
|
134
|
+
Shared runtime env:
|
|
135
|
+
- `METRILLM_STREAM_STALL_TIMEOUT_MS` (optional): stream watchdog for all backends, default `30000`, `0` disables it
|
|
136
|
+
|
|
137
|
+
LM Studio benchmark runs now use the native REST inference endpoint (`/api/v1/chat`) for both streaming and non-streaming generation.
|
|
138
|
+
The previous OpenAI-compatible inference path (`/v1/chat/completions`) has been retired from MetriLLM so tok/s and TTFT can rely on native LM Studio stats when available.
|
|
139
|
+
If a LM Studio response omits native token stats, MetriLLM still computes a score and shows the throughput as `estimated`.
|
|
123
140
|
|
|
124
141
|
For very large models, tune timeout flags:
|
|
125
142
|
- `--perf-warmup-timeout-ms` (default `300000`)
|
|
126
143
|
- `--perf-prompt-timeout-ms` (default `120000`)
|
|
127
144
|
- `--quality-timeout-ms` (default `120000`)
|
|
128
145
|
- `--coding-timeout-ms` (default `240000`)
|
|
129
|
-
- `--
|
|
146
|
+
- `--stream-stall-timeout-ms` (default `30000`, `0` disables stall timeout for any backend)
|
|
130
147
|
|
|
131
148
|
Benchmark Profile v1 (applied to all benchmark prompts):
|
|
132
149
|
- `temperature=0`
|