token-speed-tester 1.0.0 → 1.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/LICENSE +21 -21
  2. package/README.md +215 -215
  3. package/dist/index.js +0 -0
  4. package/package.json +1 -1
package/LICENSE CHANGED
@@ -1,21 +1,21 @@
1
- MIT License
2
-
3
- Copyright (c) 2025
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining a copy
6
- of this software and associated documentation files (the "Software"), to deal
7
- in the Software without restriction, including without limitation the rights
8
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
- copies of the Software, and to permit persons to whom the Software is
10
- furnished to do so, subject to the following conditions:
11
-
12
- The above copyright notice and this permission notice shall be included in all
13
- copies or substantial portions of the Software.
14
-
15
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
- SOFTWARE.
1
+ MIT License
2
+
3
+ Copyright (c) 2025
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,215 +1,215 @@
1
- # Token Speed Tester
2
-
3
- A CLI tool to test LLM API token output speed for Anthropic and OpenAI compatible APIs.
4
-
5
- ## Features
6
-
7
- - **Dual Protocol Support**: Works with both Anthropic Messages API and OpenAI Chat Completions API
8
- - **Streaming Performance**: Measures each token's arrival time with precision
9
- - **Comprehensive Metrics**: TTFT, average speed, peak speed, TPS curves
10
- - **Statistical Analysis**: Mean, min, max, and standard deviation across multiple test runs
11
- - **ASCII Visualization**: Beautiful terminal-based charts and tables
12
- - **Custom Endpoints**: Test third-party APIs that are compatible with OpenAI/Anthropic protocols
13
-
14
- ## Installation
15
-
16
- ### Global Installation
17
-
18
- ```bash
19
- npm install -g token-speed-tester
20
- ```
21
-
22
- ### Local Installation
23
-
24
- ```bash
25
- npm install token-speed-tester
26
- ```
27
-
28
- ## Usage
29
-
30
- ### Basic Usage
31
-
32
- ```bash
33
- # Test Anthropic API (default)
34
- token-speed-test --api-key sk-ant-xxx
35
-
36
- # Test OpenAI API
37
- token-speed-test --api-key sk-xxx --provider openai
38
- ```
39
-
40
- ### Advanced Options
41
-
42
- ```bash
43
- # Custom model and multiple test runs
44
- token-speed-test \
45
- --api-key sk-ant-xxx \
46
- --provider anthropic \
47
- --model claude-3-5-sonnet-20241022 \
48
- --runs 5
49
-
50
- # Test with custom endpoint and prompt
51
- token-speed-test \
52
- --api-key sk-xxx \
53
- --provider openai \
54
- --url https://api.example.com/v1 \
55
- --model custom-model \
56
- --prompt "Explain quantum computing" \
57
- --max-tokens 2048 \
58
- --runs 10
59
- ```
60
-
61
- ### Local Development
62
-
63
- ```bash
64
- # Clone and install dependencies
65
- git clone https://github.com/Cansiny0320/token-speed-tester.git
66
- cd token-speed-tester
67
- npm install
68
-
69
- # Run directly with tsx
70
- npm run dev -- --api-key sk-ant-xxx
71
-
72
- # Or build and run
73
- npm run build
74
- node dist/index.js --api-key sk-ant-xxx
75
- ```
76
-
77
- ## Command Line Options
78
-
79
- | Option | Short | Description | Default |
80
- |--------|-------|-------------|---------|
81
- | `--api-key` | `-k` | API Key (required) | - |
82
- | `--provider` | `-p` | API type: `anthropic` or `openai` | `anthropic` |
83
- | `--model` | `-m` | Model name | Auto-selected by provider |
84
- | `--url` | `-u` | Custom API endpoint | Official endpoint |
85
- | `--runs` | `-r` | Number of test runs | `3` |
86
- | `--prompt` | | Test prompt | "写一篇关于 AI 的短文" |
87
- | `--max-tokens` | | Maximum output tokens | `1024` |
88
-
89
- ### Default Models
90
-
91
- - **Anthropic**: `claude-3-5-sonnet-20241022`
92
- - **OpenAI**: `gpt-4o-mini`
93
-
94
- ## Output Example
95
-
96
- ```
97
- 🚀 Token 速度测试工具
98
- ──────────────────────────────────────────
99
- Provider: anthropic
100
- Model: claude-3-5-sonnet-20241022
101
- Max Tokens: 1024
102
- Runs: 3
103
- Prompt: 写一篇关于 AI 的短文
104
- ──────────────────────────────────────────
105
-
106
- ⏳ 正在运行测试...
107
-
108
- [运行 1]
109
- TTFT: 523ms
110
- 总耗时: 3245ms
111
- 总 Token 数: 412
112
- 平均速度: 126.96 tokens/s
113
- 峰值速度: 156.32 tokens/s
114
-
115
- [运行 2]
116
- TTFT: 487ms
117
- 总耗时: 3189ms
118
- 总 Token 数: 398
119
- 平均速度: 124.84 tokens/s
120
- 峰值速度: 158.41 tokens/s
121
-
122
- [运行 3]
123
- TTFT: 501ms
124
- 总耗时: 3312ms
125
- 总 Token 数: 405
126
- 平均速度: 122.28 tokens/s
127
- 峰值速度: 154.23 tokens/s
128
-
129
- ======================================================================
130
- Token 速度测试报告
131
- ======================================================================
132
-
133
- 统计汇总 (N=3)
134
- ┌──────────────────────────────────────────────────────────────────────┐
135
- │ 指标 │ 均值 │ 最小值 │ 最大值 │ 标准差 │
136
- ├──────────────────────────────────────────────────────────────────────┤
137
- │ TTFT (ms) │ 503.67 │ 487.00 │ 523.00 │ 14.57 │
138
- ├──────────────────────────────────────────────────────────────────────┤
139
- │ 总耗时 (ms) │ 3248.67 │ 3189.00 │ 3312.00 │ 51.92 │
140
- ├──────────────────────────────────────────────────────────────────────┤
141
- │ 总 Token 数 │ 405.00 │ 398.00 │ 412.00 │ 5.35 │
142
- ├──────────────────────────────────────────────────────────────────────┤
143
- │ 平均速度 │ 124.69 │ 122.28 │ 126.96 │ 1.88 │
144
- ├──────────────────────────────────────────────────────────────────────┤
145
- │ 峰值速度 │ 156.32 │ 154.23 │ 158.41 │ 1.82 │
146
- └──────────────────────────────────────────────────────────────────────┘
147
-
148
- Token 速度趋势图 (TPS)
149
- ┌────────────────────────────────────────┐
150
- │ 120 ┤ █ │
151
- │ 100 ┤ █ █ █ █ │
152
- │ 80 ┤ █ █ █ █ █ █ █ │
153
- │ 60 ┤ █ █ █ █ █ █ █ █ █ █ │
154
- │ 40 ┤ █ █ █ █ █ █ █ █ █ █ █ █ │
155
- │ 20 ┤ █ █ █ █ █ █ █ █ █ █ █ █ █ █ │
156
- │ 0 └────────────────────────────────── │
157
- │ 0s 1s 2s 3s 4s 5s 6s │
158
- └────────────────────────────────────────┘
159
-
160
- TPS 分布
161
- 0.0-12.0 │██████████████████████████████████████████████████ 45
162
- 12.0-24.0 │██ 3
163
- 24.0-36.0 │ 0
164
- 36.0-48.0 │ 0
165
- 48.0-60.0 │ 0
166
- 60.0-72.0 │ 0
167
- 72.0-84.0 │ 0
168
- 84.0-96.0 │ 0
169
- 96.0-108.0 │ 0
170
- 108.0-120.0 │ 0
171
-
172
- ✅ 测试完成!
173
- ```
174
-
175
- ## Metrics Explained
176
-
177
- - **TTFT (Time to First Token)**: Time from request start to first token arrival
178
- - **Total Time**: Total duration from request to completion
179
- - **Total Tokens**: Number of output tokens received
180
- - **Average Speed**: Mean tokens per second (totalTokens / totalTime)
181
- - **Peak Speed**: Fastest speed measured over a 10-token window
182
- - **TPS Curve**: Tokens received per second throughout the stream
183
-
184
- ## Development
185
-
186
- ### Running Tests
187
-
188
- ```bash
189
- # Run tests
190
- npm test
191
-
192
- # Run tests with UI
193
- npm run test:ui
194
-
195
- # Generate coverage report
196
- npm run test:coverage
197
- ```
198
-
199
- ### Building
200
-
201
- ```bash
202
- npm run build
203
- ```
204
-
205
- ### Test Coverage
206
-
207
- This project maintains 100% code coverage across all modules.
208
-
209
- ## License
210
-
211
- MIT License - see [LICENSE](LICENSE) for details.
212
-
213
- ## Contributing
214
-
215
- Contributions are welcome! Please feel free to submit a Pull Request.
1
+ # Token Speed Tester
2
+
3
+ A CLI tool to test LLM API token output speed for Anthropic and OpenAI compatible APIs.
4
+
5
+ ## Features
6
+
7
+ - **Dual Protocol Support**: Works with both Anthropic Messages API and OpenAI Chat Completions API
8
+ - **Streaming Performance**: Measures each token's arrival time with precision
9
+ - **Comprehensive Metrics**: TTFT, average speed, peak speed, TPS curves
10
+ - **Statistical Analysis**: Mean, min, max, and standard deviation across multiple test runs
11
+ - **ASCII Visualization**: Beautiful terminal-based charts and tables
12
+ - **Custom Endpoints**: Test third-party APIs that are compatible with OpenAI/Anthropic protocols
13
+
14
+ ## Installation
15
+
16
+ ### Global Installation
17
+
18
+ ```bash
19
+ npm install -g token-speed-tester
20
+ ```
21
+
22
+ ### Local Installation
23
+
24
+ ```bash
25
+ npm install token-speed-tester
26
+ ```
27
+
28
+ ## Usage
29
+
30
+ ### Basic Usage
31
+
32
+ ```bash
33
+ # Test Anthropic API (default)
34
+ token-speed-test --api-key sk-ant-xxx
35
+
36
+ # Test OpenAI API
37
+ token-speed-test --api-key sk-xxx --provider openai
38
+ ```
39
+
40
+ ### Advanced Options
41
+
42
+ ```bash
43
+ # Custom model and multiple test runs
44
+ token-speed-test \
45
+ --api-key sk-ant-xxx \
46
+ --provider anthropic \
47
+ --model claude-3-5-sonnet-20241022 \
48
+ --runs 5
49
+
50
+ # Test with custom endpoint and prompt
51
+ token-speed-test \
52
+ --api-key sk-xxx \
53
+ --provider openai \
54
+ --url https://api.example.com/v1 \
55
+ --model custom-model \
56
+ --prompt "Explain quantum computing" \
57
+ --max-tokens 2048 \
58
+ --runs 10
59
+ ```
60
+
61
+ ### Local Development
62
+
63
+ ```bash
64
+ # Clone and install dependencies
65
+ git clone https://github.com/Cansiny0320/token-speed-tester.git
66
+ cd token-speed-tester
67
+ npm install
68
+
69
+ # Run directly with tsx
70
+ npm run dev -- --api-key sk-ant-xxx
71
+
72
+ # Or build and run
73
+ npm run build
74
+ node dist/index.js --api-key sk-ant-xxx
75
+ ```
76
+
77
+ ## Command Line Options
78
+
79
+ | Option | Short | Description | Default |
80
+ |--------|-------|-------------|---------|
81
+ | `--api-key` | `-k` | API Key (required) | - |
82
+ | `--provider` | `-p` | API type: `anthropic` or `openai` | `anthropic` |
83
+ | `--model` | `-m` | Model name | Auto-selected by provider |
84
+ | `--url` | `-u` | Custom API endpoint | Official endpoint |
85
+ | `--runs` | `-r` | Number of test runs | `3` |
86
+ | `--prompt` | | Test prompt | "写一篇关于 AI 的短文" |
87
+ | `--max-tokens` | | Maximum output tokens | `1024` |
88
+
89
+ ### Default Models
90
+
91
+ - **Anthropic**: `claude-3-5-sonnet-20241022`
92
+ - **OpenAI**: `gpt-4o-mini`
93
+
94
+ ## Output Example
95
+
96
+ ```
97
+ 🚀 Token 速度测试工具
98
+ ──────────────────────────────────────────
99
+ Provider: anthropic
100
+ Model: claude-3-5-sonnet-20241022
101
+ Max Tokens: 1024
102
+ Runs: 3
103
+ Prompt: 写一篇关于 AI 的短文
104
+ ──────────────────────────────────────────
105
+
106
+ ⏳ 正在运行测试...
107
+
108
+ [运行 1]
109
+ TTFT: 523ms
110
+ 总耗时: 3245ms
111
+ 总 Token 数: 412
112
+ 平均速度: 126.96 tokens/s
113
+ 峰值速度: 156.32 tokens/s
114
+
115
+ [运行 2]
116
+ TTFT: 487ms
117
+ 总耗时: 3189ms
118
+ 总 Token 数: 398
119
+ 平均速度: 124.84 tokens/s
120
+ 峰值速度: 158.41 tokens/s
121
+
122
+ [运行 3]
123
+ TTFT: 501ms
124
+ 总耗时: 3312ms
125
+ 总 Token 数: 405
126
+ 平均速度: 122.28 tokens/s
127
+ 峰值速度: 154.23 tokens/s
128
+
129
+ ======================================================================
130
+ Token 速度测试报告
131
+ ======================================================================
132
+
133
+ 统计汇总 (N=3)
134
+ ┌──────────────────────────────────────────────────────────────────────┐
135
+ │ 指标 │ 均值 │ 最小值 │ 最大值 │ 标准差 │
136
+ ├──────────────────────────────────────────────────────────────────────┤
137
+ │ TTFT (ms) │ 503.67 │ 487.00 │ 523.00 │ 14.57 │
138
+ ├──────────────────────────────────────────────────────────────────────┤
139
+ │ 总耗时 (ms) │ 3248.67 │ 3189.00 │ 3312.00 │ 51.92 │
140
+ ├──────────────────────────────────────────────────────────────────────┤
141
+ │ 总 Token 数 │ 405.00 │ 398.00 │ 412.00 │ 5.35 │
142
+ ├──────────────────────────────────────────────────────────────────────┤
143
+ │ 平均速度 │ 124.69 │ 122.28 │ 126.96 │ 1.88 │
144
+ ├──────────────────────────────────────────────────────────────────────┤
145
+ │ 峰值速度 │ 156.32 │ 154.23 │ 158.41 │ 1.82 │
146
+ └──────────────────────────────────────────────────────────────────────┘
147
+
148
+ Token 速度趋势图 (TPS)
149
+ ┌────────────────────────────────────────┐
150
+ │ 120 ┤ █ │
151
+ │ 100 ┤ █ █ █ █ │
152
+ │ 80 ┤ █ █ █ █ █ █ █ │
153
+ │ 60 ┤ █ █ █ █ █ █ █ █ █ █ │
154
+ │ 40 ┤ █ █ █ █ █ █ █ █ █ █ █ █ │
155
+ │ 20 ┤ █ █ █ █ █ █ █ █ █ █ █ █ █ █ │
156
+ │ 0 └────────────────────────────────── │
157
+ │ 0s 1s 2s 3s 4s 5s 6s │
158
+ └────────────────────────────────────────┘
159
+
160
+ TPS 分布
161
+ 0.0-12.0 │██████████████████████████████████████████████████ 45
162
+ 12.0-24.0 │██ 3
163
+ 24.0-36.0 │ 0
164
+ 36.0-48.0 │ 0
165
+ 48.0-60.0 │ 0
166
+ 60.0-72.0 │ 0
167
+ 72.0-84.0 │ 0
168
+ 84.0-96.0 │ 0
169
+ 96.0-108.0 │ 0
170
+ 108.0-120.0 │ 0
171
+
172
+ ✅ 测试完成!
173
+ ```
174
+
175
+ ## Metrics Explained
176
+
177
+ - **TTFT (Time to First Token)**: Time from request start to first token arrival
178
+ - **Total Time**: Total duration from request to completion
179
+ - **Total Tokens**: Number of output tokens received
180
+ - **Average Speed**: Mean tokens per second (totalTokens / totalTime)
181
+ - **Peak Speed**: Fastest speed measured over a 10-token window
182
+ - **TPS Curve**: Tokens received per second throughout the stream
183
+
184
+ ## Development
185
+
186
+ ### Running Tests
187
+
188
+ ```bash
189
+ # Run tests
190
+ npm test
191
+
192
+ # Run tests with UI
193
+ npm run test:ui
194
+
195
+ # Generate coverage report
196
+ npm run test:coverage
197
+ ```
198
+
199
+ ### Building
200
+
201
+ ```bash
202
+ npm run build
203
+ ```
204
+
205
+ ### Test Coverage
206
+
207
+ This project maintains 100% code coverage across all modules.
208
+
209
+ ## License
210
+
211
+ MIT License - see [LICENSE](LICENSE) for details.
212
+
213
+ ## Contributing
214
+
215
+ Contributions are welcome! Please feel free to submit a Pull Request.
package/dist/index.js CHANGED
File without changes
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "token-speed-tester",
3
- "version": "1.0.0",
3
+ "version": "1.0.3",
4
4
  "description": "A CLI tool to test LLM API token output speed",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",