metrillm-mcp 0.1.2 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -7
- package/dist/index.js +1348 -140
- package/dist/index.js.map +1 -1
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -45,8 +45,10 @@ Add to your editor's MCP configuration:
|
|
|
45
45
|
## Prerequisites
|
|
46
46
|
|
|
47
47
|
- Node.js >= 20
|
|
48
|
-
-
|
|
49
|
-
-
|
|
48
|
+
- One local runtime available:
|
|
49
|
+
- [Ollama](https://ollama.com) (`ollama serve`)
|
|
50
|
+
- [LM Studio](https://lmstudio.ai/) (Local Server enabled)
|
|
51
|
+
- At least one model available on the selected runtime
|
|
50
52
|
|
|
51
53
|
## Available Tools
|
|
52
54
|
|
|
@@ -56,7 +58,7 @@ List all locally available LLM models.
|
|
|
56
58
|
|
|
57
59
|
| Param | Type | Default | Description |
|
|
58
60
|
|---|---|---|---|
|
|
59
|
-
| `runtime` | `"ollama"` | `"ollama"` | Inference runtime |
|
|
61
|
+
| `runtime` | `"ollama" \| "lm-studio"` | `"ollama"` | Inference runtime |
|
|
60
62
|
|
|
61
63
|
**Example response:**
|
|
62
64
|
```json
|
|
@@ -75,7 +77,7 @@ Run a full benchmark (performance + quality) on a local model.
|
|
|
75
77
|
| Param | Type | Default | Description |
|
|
76
78
|
|---|---|---|---|
|
|
77
79
|
| `model` | `string` | *(required)* | Model name (e.g. `"llama3.2:3b"`) |
|
|
78
|
-
| `runtime` | `"ollama"` | `"ollama"` | Inference runtime |
|
|
80
|
+
| `runtime` | `"ollama" \| "lm-studio"` | `"ollama"` | Inference runtime |
|
|
79
81
|
| `perfOnly` | `boolean` | `false` | If `true`, measure performance only (skip quality) |
|
|
80
82
|
|
|
81
83
|
**Example response:**
|
|
@@ -102,7 +104,7 @@ Retrieve previous benchmark results stored locally.
|
|
|
102
104
|
| Param | Type | Default | Description |
|
|
103
105
|
|---|---|---|---|
|
|
104
106
|
| `model` | `string` | *(optional)* | Filter by model name (substring match) |
|
|
105
|
-
| `runtime` | `"ollama"` | `"ollama"` | Inference runtime |
|
|
107
|
+
| `runtime` | `"ollama" \| "lm-studio"` | `"ollama"` | Inference runtime |
|
|
106
108
|
|
|
107
109
|
### `share_result`
|
|
108
110
|
|
|
@@ -112,7 +114,9 @@ Upload a result to the public MetriLLM leaderboard.
|
|
|
112
114
|
|---|---|---|
|
|
113
115
|
| `resultFile` | `string` | Absolute path to the result JSON file |
|
|
114
116
|
|
|
115
|
-
|
|
117
|
+
Uses official MetriLLM upload defaults out of the box.
|
|
118
|
+
|
|
119
|
+
Optional overrides for self-hosted/staging deployments:
|
|
116
120
|
- `METRILLM_SUPABASE_URL`
|
|
117
121
|
- `METRILLM_SUPABASE_ANON_KEY`
|
|
118
122
|
- `METRILLM_PUBLIC_RESULT_BASE_URL`
|
|
@@ -136,7 +140,7 @@ No code duplication — the MCP server imports CLI modules directly.
|
|
|
136
140
|
| Runtime | Status |
|
|
137
141
|
|---|---|
|
|
138
142
|
| Ollama | Supported |
|
|
139
|
-
| LM Studio |
|
|
143
|
+
| LM Studio | Supported |
|
|
140
144
|
| MLX | Planned |
|
|
141
145
|
| llama.cpp | Planned |
|
|
142
146
|
| vLLM | Planned |
|