ai-speedometer 1.2.5 → 1.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -1
- package/cli.js +406 -97
- package/dist/ai-speedometer +58 -74
- package/package.json +4 -7
package/README.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
A CLI tool for benchmarking AI models across multiple providers with parallel execution and performance metrics.
|
|
4
4
|
|
|
5
|
+
|
|
6
|
+
|
|
5
7
|
## Install
|
|
6
8
|
|
|
7
9
|
```bash
|
|
@@ -15,6 +17,12 @@ npm install -g ai-speedometer
|
|
|
15
17
|
- **Tokens/Second** - Real-time throughput
|
|
16
18
|
- **Token Counts** - Input, output, and total tokens used
|
|
17
19
|
|
|
20
|
+
## New Features
|
|
21
|
+
|
|
22
|
+
- **REST API Default** - REST API benchmarking is now the default method for better compatibility
|
|
23
|
+
- **Headless Mode** - Run benchmarks without interactive CLI using command-line arguments
|
|
24
|
+
- **Streaming Support** - Full streaming support now available in REST API benchmarks
|
|
25
|
+
|
|
18
26
|
## Quick Setup
|
|
19
27
|
|
|
20
28
|
1. **Set Model**
|
|
@@ -39,7 +47,8 @@ npm install -g ai-speedometer
|
|
|
39
47
|
4. **Run Benchmark**
|
|
40
48
|
```bash
|
|
41
49
|
ai-speedometer
|
|
42
|
-
# Select "Run Benchmark (
|
|
50
|
+
# Select "Run Benchmark (REST API)" → Choose models → Press ENTER
|
|
51
|
+
# Note: REST API is now the default benchmark method
|
|
43
52
|
```
|
|
44
53
|
|
|
45
54
|
## Usage
|
|
@@ -53,6 +62,13 @@ aispeed
|
|
|
53
62
|
|
|
54
63
|
# Debug mode
|
|
55
64
|
ai-speedometer --debug
|
|
65
|
+
|
|
66
|
+
# Headless mode - run benchmark directly
|
|
67
|
+
ai-speedometer --bench openai:gpt-4
|
|
68
|
+
# With custom API key
|
|
69
|
+
ai-speedometer --bench openai:gpt-4 --api-key "sk-your-key"
|
|
70
|
+
# Use AI SDK instead of REST API
|
|
71
|
+
ai-speedometer --bench openai:gpt-4 --ai-sdk
|
|
56
72
|
```
|
|
57
73
|
|
|
58
74
|
## Configuration Files
|