whichmodel 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +373 -0
- package/dist/index.d.ts +1 -0
- package/dist/index.js +5025 -0
- package/dist/index.js.map +1 -0
- package/package.json +65 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Oussama Nahiz
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,373 @@
|
|
|
1
|
+
# whichmodel
|
|
2
|
+
|
|
3
|
+
`whichmodel` helps you choose the right AI model fast.
|
|
4
|
+
|
|
5
|
+
You describe a task in plain English, and the CLI recommends:
|
|
6
|
+
|
|
7
|
+
- cheapest
|
|
8
|
+
- balanced
|
|
9
|
+
- best
|
|
10
|
+
|
|
11
|
+
It uses live provider catalogs and task-aware reasoning so you can make a decision quickly instead of manually comparing models for 30 minutes.
|
|
12
|
+
|
|
13
|
+
## What You Get
|
|
14
|
+
|
|
15
|
+
- Task-aware recommendations (not just raw price sorting)
|
|
16
|
+
- Multi-modality support: text, image, video, audio, vision, embedding, multimodal
|
|
17
|
+
- Cost-aware picks with reasoning
|
|
18
|
+
- Script-friendly JSON mode
|
|
19
|
+
- Catalog exploration commands (`list`, `stats`, `compare`, `cache`)
|
|
20
|
+
|
|
21
|
+
## Status
|
|
22
|
+
|
|
23
|
+
- Version: `1.0.0`
|
|
24
|
+
- Stability: Stable
|
|
25
|
+
- Runtime: Node.js 20+
|
|
26
|
+
|
|
27
|
+
## Install
|
|
28
|
+
|
|
29
|
+
Global install with npm:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
npm install -g whichmodel
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Run without installing:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npx whichmodel "summarize legal contracts"
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Install with Homebrew (after tap setup):
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
brew tap useit015/tap
|
|
45
|
+
brew install whichmodel
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Install from source:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
git clone https://github.com/useit015/whichmodel.git
|
|
52
|
+
cd whichmodel
|
|
53
|
+
npm install
|
|
54
|
+
npm run build
|
|
55
|
+
npm link
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## Quick Start
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
export OPENROUTER_API_KEY="sk-or-v1-..."
|
|
62
|
+
whichmodel "summarize legal contracts and flag risks"
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
JSON output:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
whichmodel "summarize legal contracts and flag risks" --json
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Verbose metadata (tokens/cost/timing):
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
whichmodel "summarize legal contracts and flag risks" --verbose
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Example Recommendation Output
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
whichmodel "summarize legal contracts and flag risks" --no-color --no-cache
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Sample output:
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
- Fetching model catalog...
|
|
87
|
+
🔍 Task Analysis
|
|
88
|
+
Modality: TEXT
|
|
89
|
+
Analyze text-based legal contracts to produce summaries and identify potential risks.
|
|
90
|
+
The task involves processing and understanding written legal language (contracts) to generate textual summaries and risk assessments. No image, audio, or video input is mentioned.
|
|
91
|
+
|
|
92
|
+
💰 Cheapest — openrouter::liquid/lfm2-8b-a1b
|
|
93
|
+
$0.01 per 1M prompt tokens, $0.02 per 1M completion tokens
|
|
94
|
+
Extremely low cost ($0.01/$0.02 per 1M tokens) makes it viable for bulk processing of simple contracts where basic summarization is acceptable. However, its small 8B parameter size and 32K context limit mean it may miss nuanced legal risks and struggle with very long documents.
|
|
95
|
+
Est. $0.15 for processing ten 50k-token contracts (500k input tokens) and generating 25k tokens of summary/risk output.
|
|
96
|
+
|
|
97
|
+
⚖️ Balanced — openrouter::deepseek/deepseek-v3.2
|
|
98
|
+
$0.26 per 1M prompt tokens, $0.38 per 1M completion tokens
|
|
99
|
+
Excellent balance of cost, strong reasoning (DeepSeek family), and a 163K context window. At $0.26/$0.38 per 1M tokens, it offers significantly better analytical capability than the cheapest options, making it suitable for reliable contract summarization and risk identification without premium pricing.
|
|
100
|
+
Est. $2.30 for processing ten 50k-token contracts and generating detailed 50k tokens of analysis.
|
|
101
|
+
|
|
102
|
+
🏆 Best — openrouter::anthropic/claude-opus-4.6
|
|
103
|
+
$5 per 1M prompt tokens, $25 per 1M completion tokens
|
|
104
|
+
Claude Opus is renowned for its exceptional reasoning, nuance, and instruction-following, making it the top choice for high-stakes legal analysis. Its 1M token context handles the longest contracts. While expensive, the quality and reliability for risk flagging justify the cost for critical legal work.
|
|
105
|
+
Est. $37.50 for processing one very long 200k-token contract and generating a comprehensive 10k token risk analysis.
|
|
106
|
+
|
|
107
|
+
💡 Tip: A vision model could be used if contracts are provided as scanned images or PDFs requiring OCR, but the core task of summarization and risk analysis remains textual. The provided catalog's vision models (e.g., Claude Opus 4.6 vision) could handle image input but at a higher cost for the same textual analysis capability.
|
|
108
|
+
|
|
109
|
+
⚡ This recommendation cost $0.0076 (deepseek/deepseek-v3.2)
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
## Command Cookbook
|
|
113
|
+
|
|
114
|
+
### 1. Recommend (default)
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
whichmodel "build a customer support chatbot" --json
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### 2. Compare two models head-to-head
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
whichmodel compare \
|
|
124
|
+
"openrouter::anthropic/claude-sonnet-4" \
|
|
125
|
+
"openrouter::openai/gpt-4o" \
|
|
126
|
+
--task "write production-ready TypeScript API code" \
|
|
127
|
+
--json --no-cache
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Sample output:
|
|
131
|
+
|
|
132
|
+
```json
|
|
133
|
+
{
|
|
134
|
+
"winner": "B",
|
|
135
|
+
"reasoning": "GPT-4o is more cost-effective for code generation with lower token pricing, and its 128K context window is sufficient for writing production-ready TypeScript API code. Claude Sonnet's longer context is unnecessary overhead for this specific task.",
|
|
136
|
+
"modelA": {
|
|
137
|
+
"id": "openrouter::anthropic/claude-sonnet-4",
|
|
138
|
+
"name": "Anthropic: Claude Sonnet 4",
|
|
139
|
+
"strengths": [
|
|
140
|
+
"Extremely large 1M token context window",
|
|
141
|
+
"Strong reasoning capabilities for complex logic"
|
|
142
|
+
],
|
|
143
|
+
"weaknesses": [
|
|
144
|
+
"Higher cost per token",
|
|
145
|
+
"Overkill context for typical API code"
|
|
146
|
+
],
|
|
147
|
+
"estimatedCost": "$18 per 1M output tokens",
|
|
148
|
+
"suitedFor": [
|
|
149
|
+
"Massive document analysis",
|
|
150
|
+
"Extremely long-form content generation",
|
|
151
|
+
"Complex multi-step reasoning tasks"
|
|
152
|
+
]
|
|
153
|
+
},
|
|
154
|
+
"modelB": {
|
|
155
|
+
"id": "openrouter::openai/gpt-4o",
|
|
156
|
+
"name": "OpenAI: GPT-4o",
|
|
157
|
+
"strengths": [
|
|
158
|
+
"Lower cost per token",
|
|
159
|
+
"Sufficient context for API development",
|
|
160
|
+
"Fast response times"
|
|
161
|
+
],
|
|
162
|
+
"weaknesses": ["Smaller context window than Claude Sonnet"],
|
|
163
|
+
"estimatedCost": "$12.5 per 1M output tokens",
|
|
164
|
+
"suitedFor": [
|
|
165
|
+
"Code generation and review",
|
|
166
|
+
"API development",
|
|
167
|
+
"General programming tasks",
|
|
168
|
+
"Cost-sensitive applications"
|
|
169
|
+
]
|
|
170
|
+
}
|
|
171
|
+
}
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
### 3. List models
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
whichmodel list --source openrouter --limit 5 --no-cache
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Sample output:
|
|
181
|
+
|
|
182
|
+
```text
|
|
183
|
+
> whichmodel@1.0.0 dev
|
|
184
|
+
> tsx src/index.ts list --source openrouter --limit 5 --no-color --no-cache
|
|
185
|
+
|
|
186
|
+
- Fetching catalog...
|
|
187
|
+
305 models (showing top 5, sorted by price)
|
|
188
|
+
|
|
189
|
+
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
|
|
190
|
+
│ ID │ Name │ Pricing │ Context │ Source │
|
|
191
|
+
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
|
|
192
|
+
│ openrouter::liquid/lfm2-8b-a1b │ LiquidAI: LFM2-8B-A1B │ $0.01 / $0.02 │ 33K │ openrouter │
|
|
193
|
+
│ openrouter::liquid/lfm-2.2-6b │ LiquidAI: LFM2-2.6B │ $0.01 / $0.02 │ 33K │ openrouter │
|
|
194
|
+
│ openrouter::ibm-granite/granite-4.0-h-micro │ IBM: Granite 4.0 Micro │ $0.02 / $0.11 │ 131K │ openrouter │
|
|
195
|
+
│ openrouter::google/gemma-3n-e4b-it │ Google: Gemma 3n 4B │ $0.02 / $0.04 │ 33K │ openrouter │
|
|
196
|
+
│ openrouter::meta-llama/llama-guard-3-8b │ Llama Guard 3 8B │ $0.02 / $0.06 │ 131K │ openrouter │
|
|
197
|
+
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
|
|
198
|
+
|
|
199
|
+
Use --limit 305 to show all, or filter with --modality, --source
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
JSON mode:
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
whichmodel list --source openrouter --limit 20 --json
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
### 4. Stats (catalog snapshot)
|
|
209
|
+
|
|
210
|
+
```bash
|
|
211
|
+
whichmodel stats --json --no-cache
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Sample output:
|
|
215
|
+
|
|
216
|
+
```json
|
|
217
|
+
{
|
|
218
|
+
"totalModels": 305,
|
|
219
|
+
"sources": ["openrouter"],
|
|
220
|
+
"byModality": {
|
|
221
|
+
"vision": { "count": 110, "priceRange": { "min": 0.04, "max": 150 } },
|
|
222
|
+
"text": { "count": 188, "priceRange": { "min": 0.01, "max": 30 } },
|
|
223
|
+
"multimodal": { "count": 7, "priceRange": { "min": 0.3, "max": 10 } }
|
|
224
|
+
},
|
|
225
|
+
"configuredSources": ["openrouter"],
|
|
226
|
+
"missingSources": [
|
|
227
|
+
{
|
|
228
|
+
"name": "fal",
|
|
229
|
+
"envVar": "FAL_API_KEY",
|
|
230
|
+
"getUrl": "https://fal.ai/dashboard/keys"
|
|
231
|
+
},
|
|
232
|
+
{
|
|
233
|
+
"name": "replicate",
|
|
234
|
+
"envVar": "REPLICATE_API_TOKEN",
|
|
235
|
+
"getUrl": "https://replicate.com/account/api-tokens"
|
|
236
|
+
},
|
|
237
|
+
{
|
|
238
|
+
"name": "elevenlabs",
|
|
239
|
+
"envVar": "ELEVENLABS_API_KEY",
|
|
240
|
+
"getUrl": "https://elevenlabs.io/app/settings/api-keys"
|
|
241
|
+
},
|
|
242
|
+
{
|
|
243
|
+
"name": "together",
|
|
244
|
+
"envVar": "TOGETHER_API_KEY",
|
|
245
|
+
"getUrl": "https://api.together.xyz/settings/api-keys"
|
|
246
|
+
}
|
|
247
|
+
]
|
|
248
|
+
}
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
### 5. Cache inspection and clear
|
|
252
|
+
|
|
253
|
+
```bash
|
|
254
|
+
whichmodel cache --stats
|
|
255
|
+
whichmodel cache --clear
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
Example output:
|
|
259
|
+
|
|
260
|
+
```text
|
|
261
|
+
Cache Statistics:
|
|
262
|
+
Location: ~/.cache/whichmodel
|
|
263
|
+
|
|
264
|
+
Source Age TTL Models
|
|
265
|
+
───────────────────────────────────────
|
|
266
|
+
fal 17m ago 3600s 2
|
|
267
|
+
openrouter 16m ago 3600s 305
|
|
268
|
+
replicate 17m ago 3600s 2
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
### 6. Update default recommender model
|
|
272
|
+
|
|
273
|
+
```bash
|
|
274
|
+
whichmodel --update-recommender
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
This analyzes current catalog candidates and updates your config only when a better default is found.
|
|
278
|
+
|
|
279
|
+
## Main Options
|
|
280
|
+
|
|
281
|
+
- `--json`: JSON output
|
|
282
|
+
- `--verbose`: add token/cost/timing metadata
|
|
283
|
+
- `--modality <type>`: force modality
|
|
284
|
+
- `--max-price <number>`: max unit price filter
|
|
285
|
+
- `--min-context <tokens>`: min context length filter
|
|
286
|
+
- `--min-resolution <WxH>`: min resolution filter for image/video
|
|
287
|
+
- `--exclude <ids>`: exclude model IDs (comma-separated, supports `*` suffix wildcard)
|
|
288
|
+
- `--sources <list>`: catalog sources (comma-separated)
|
|
289
|
+
- `--model <id>`: override recommender model
|
|
290
|
+
- `--estimate <workload>`: workload-based estimated costs
|
|
291
|
+
- `--no-color`: disable color output
|
|
292
|
+
- `--no-cache`: bypass cache and fetch fresh catalogs
|
|
293
|
+
- `--update-recommender`: update default recommender model
|
|
294
|
+
|
|
295
|
+
Global flags like `--json`, `--no-color`, and `--no-cache` apply to subcommands too.
|
|
296
|
+
|
|
297
|
+
## Sources
|
|
298
|
+
|
|
299
|
+
Currently supported:
|
|
300
|
+
|
|
301
|
+
- `openrouter`
|
|
302
|
+
- `fal`
|
|
303
|
+
- `replicate`
|
|
304
|
+
|
|
305
|
+
Recognized but not yet implemented:
|
|
306
|
+
|
|
307
|
+
- `elevenlabs`
|
|
308
|
+
- `together`
|
|
309
|
+
|
|
310
|
+
Default source is `openrouter`.
|
|
311
|
+
|
|
312
|
+
## API Keys
|
|
313
|
+
|
|
314
|
+
Required for recommendation and compare:
|
|
315
|
+
|
|
316
|
+
- `OPENROUTER_API_KEY`
|
|
317
|
+
|
|
318
|
+
Optional by source:
|
|
319
|
+
|
|
320
|
+
- `FAL_API_KEY`
|
|
321
|
+
- `REPLICATE_API_TOKEN`
|
|
322
|
+
|
|
323
|
+
Notes:
|
|
324
|
+
|
|
325
|
+
- `list` and `stats` can run without `OPENROUTER_API_KEY` using public OpenRouter catalog access.
|
|
326
|
+
- `compare` and default recommendation flow require `OPENROUTER_API_KEY`.
|
|
327
|
+
|
|
328
|
+
## Modalities
|
|
329
|
+
|
|
330
|
+
- `text`
|
|
331
|
+
- `image`
|
|
332
|
+
- `video`
|
|
333
|
+
- `audio_tts`
|
|
334
|
+
- `audio_stt`
|
|
335
|
+
- `audio_generation`
|
|
336
|
+
- `vision`
|
|
337
|
+
- `embedding`
|
|
338
|
+
- `multimodal`
|
|
339
|
+
|
|
340
|
+
## Development
|
|
341
|
+
|
|
342
|
+
Install dependencies:
|
|
343
|
+
|
|
344
|
+
```bash
|
|
345
|
+
npm install
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
Run dev mode:
|
|
349
|
+
|
|
350
|
+
```bash
|
|
351
|
+
npm run dev -- "generate product photos" --json
|
|
352
|
+
```
|
|
353
|
+
|
|
354
|
+
Catalog fetch script:
|
|
355
|
+
|
|
356
|
+
```bash
|
|
357
|
+
npm run catalog:fetch -- --sources openrouter
|
|
358
|
+
npm run catalog:fetch -- --sources openrouter,fal
|
|
359
|
+
npm run catalog:fetch -- --sources openrouter,replicate
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
Quality checks:
|
|
363
|
+
|
|
364
|
+
```bash
|
|
365
|
+
npm run lint
|
|
366
|
+
npm test
|
|
367
|
+
npm run test:coverage
|
|
368
|
+
npm run build
|
|
369
|
+
```
|
|
370
|
+
|
|
371
|
+
## Author
|
|
372
|
+
|
|
373
|
+
Oussama Nahiz
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
#!/usr/bin/env node
|