token-speed-tester 1.0.3 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.en.md +272 -0
  2. package/README.md +112 -55
  3. package/package.json +6 -1
package/README.en.md ADDED
@@ -0,0 +1,272 @@
1
+ # Token Speed Tester
2
+
3
+ > A CLI tool to measure and analyze LLM API token streaming performance
4
+
5
+ [中文文档](README.md) |
6
+
7
+ [![npm version](https://badge.fury.io/js/token-speed-tester.svg)](https://www.npmjs.com/package/token-speed-tester)
8
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
9
+ [![Tests](https://github.com/Cansiny0320/token-speed-tester/actions/workflows/publish.yml/badge.svg)](https://github.com/Cansiny0320/token-speed-tester/actions)
10
+
11
+ A powerful command-line tool for testing token output speed of LLM APIs. Supports **Anthropic** and **OpenAI** compatible APIs, providing detailed metrics including TTFT (Time to First Token), throughput, peak speed, and statistical analysis across multiple runs.
12
+
13
+ ## Features
14
+
15
+ - **Dual Protocol Support**: Works with Anthropic Messages API and OpenAI Chat Completions API
16
+ - **Streaming Performance**: Measures each token's arrival time with millisecond precision
17
+ - **Comprehensive Metrics**:
18
+ - **TTFT** (Time to First Token): Latency before first token arrives
19
+ - **Average Speed**: Mean tokens per second
20
+ - **Peak Speed**: Fastest speed over a 10-token window
21
+ - **TPS Curve**: Tokens received per second throughout the stream
22
+ - **Statistical Analysis**: Mean, min, max, and standard deviation across multiple test runs
23
+ - **ASCII Visualization**: Beautiful terminal-based charts and tables
24
+ - **Custom Endpoints**: Test third-party APIs compatible with OpenAI/Anthropic protocols
25
+
26
+ ## Installation
27
+
28
+ ### Global Installation (Recommended)
29
+
30
+ ```bash
31
+ npm install -g token-speed-tester
32
+ ```
33
+
34
+ ### Using npx (No Installation)
35
+
36
+ ```bash
37
+ npx token-speed-tester --api-key sk-xxx
38
+ ```
39
+
40
+ ### Local Installation
41
+
42
+ ```bash
43
+ npm install token-speed-tester
44
+ ```
45
+
46
+ ## Usage
47
+
48
+ ### Basic Usage
49
+
50
+ ```bash
51
+ # Test Anthropic API (default)
52
+ token-speed-test --api-key sk-ant-xxx
53
+
54
+ # Test OpenAI API
55
+ token-speed-test --api-key sk-xxx --provider openai
56
+ ```
57
+
58
+ ### Advanced Options
59
+
60
+ ```bash
61
+ # Custom model and multiple test runs
62
+ token-speed-test \
63
+ --api-key sk-ant-xxx \
64
+ --provider anthropic \
65
+ --model claude-3-5-sonnet-20241022 \
66
+ --runs 5
67
+
68
+ # Test with custom endpoint and prompt
69
+ token-speed-test \
70
+ --api-key sk-xxx \
71
+ --provider openai \
72
+ --url https://api.example.com/v1 \
73
+ --model custom-model \
74
+ --prompt "Explain quantum computing" \
75
+ --max-tokens 2048 \
76
+ --runs 10
77
+ ```
78
+
79
+ ### Local Development
80
+
81
+ ```bash
82
+ # Clone and install dependencies
83
+ git clone https://github.com/Cansiny0320/token-speed-tester.git
84
+ cd token-speed-tester
85
+ npm install
86
+
87
+ # Run directly with tsx
88
+ npm run dev -- --api-key sk-ant-xxx
89
+
90
+ # Or build and run
91
+ npm run build
92
+ node dist/index.js --api-key sk-ant-xxx
93
+ ```
94
+
95
+ ## Command Line Options
96
+
97
+ | Option | Short | Description | Default |
98
+ |--------|-------|-------------|---------|
99
+ | `--api-key` | `-k` | API Key (required) | - |
100
+ | `--provider` | `-p` | API type: `anthropic` or `openai` | `anthropic` |
101
+ | `--model` | `-m` | Model name | Auto-selected by provider |
102
+ | `--url` | `-u` | Custom API endpoint | Official endpoint |
103
+ | `--runs` | `-r` | Number of test runs | `3` |
104
+ | `--prompt` | | Test prompt | "写一篇关于 AI 的短文" |
105
+ | `--max-tokens` | | Maximum output tokens | `1024` |
106
+
107
+ ### Default Models
108
+
109
+ - **Anthropic**: `claude-3-5-sonnet-20241022`
110
+ - **OpenAI**: `gpt-4o-mini`
111
+
112
+ ## Output Example
113
+
114
+ ```
115
+ 🚀 Token 速度测试工具
116
+ ──────────────────────────────────────────
117
+ Provider: anthropic
118
+ Model: claude-3-5-sonnet-20241022
119
+ Max Tokens: 1024
120
+ Runs: 3
121
+ Prompt: 写一篇关于 AI 的短文
122
+ ──────────────────────────────────────────
123
+
124
+ ⏳ 正在运行测试...
125
+
126
+ [运行 1]
127
+ TTFT: 523ms
128
+ 总耗时: 3245ms
129
+ 总 Token 数: 412
130
+ 平均速度: 126.96 tokens/s
131
+ 峰值速度: 156.32 tokens/s
132
+
133
+ [运行 2]
134
+ TTFT: 487ms
135
+ 总耗时: 3189ms
136
+ 总 Token 数: 398
137
+ 平均速度: 124.84 tokens/s
138
+ 峰值速度: 158.41 tokens/s
139
+
140
+ [运行 3]
141
+ TTFT: 501ms
142
+ 总耗时: 3312ms
143
+ 总 Token 数: 405
144
+ 平均速度: 122.28 tokens/s
145
+ 峰值速度: 154.23 tokens/s
146
+
147
+ ======================================================================
148
+ Token 速度测试报告
149
+ ======================================================================
150
+
151
+ 统计汇总 (N=3)
152
+ ┌──────────────────────────────────────────────────────────────────────┐
153
+ │ 指标 │ 均值 │ 最小值 │ 最大值 │ 标准差 │
154
+ ├──────────────────────────────────────────────────────────────────────┤
155
+ │ TTFT (ms) │ 503.67 │ 487.00 │ 523.00 │ 14.57 │
156
+ ├──────────────────────────────────────────────────────────────────────┤
157
+ │ 总耗时 (ms) │ 3248.67 │ 3189.00 │ 3312.00 │ 51.92 │
158
+ ├──────────────────────────────────────────────────────────────────────┤
159
+ │ 总 Token 数 │ 405.00 │ 398.00 │ 412.00 │ 5.35 │
160
+ ├──────────────────────────────────────────────────────────────────────┤
161
+ │ 平均速度 │ 124.69 │ 122.28 │ 126.96 │ 1.88 │
162
+ ├──────────────────────────────────────────────────────────────────────┤
163
+ │ 峰值速度 │ 156.32 │ 154.23 │ 158.41 │ 1.82 │
164
+ └──────────────────────────────────────────────────────────────────────┘
165
+
166
+ Token 速度趋势图 (TPS)
167
+ ┌────────────────────────────────────────┐
168
+ │ 120 ┤ █ │
169
+ │ 100 ┤ █ █ █ █ │
170
+ │ 80 ┤ █ █ █ █ █ █ █ │
171
+ │ 60 ┤ █ █ █ █ █ █ █ █ █ █ │
172
+ │ 40 ┤ █ █ █ █ █ █ █ █ █ █ █ █ │
173
+ │ 20 ┤ █ █ █ █ █ █ █ █ █ █ █ █ █ █ │
174
+ │ 0 └────────────────────────────────── │
175
+ │ 0s 1s 2s 3s 4s 5s 6s │
176
+ └────────────────────────────────────────┘
177
+
178
+ TPS 分布
179
+ 0.0-12.0 │██████████████████████████████████████████████████ 45
180
+ 12.0-24.0 │██ 3
181
+ 24.0-36.0 │ 0
182
+ 36.0-48.0 │ 0
183
+ 48.0-60.0 │ 0
184
+ 60.0-72.0 │ 0
185
+ 72.0-84.0 │ 0
186
+ 84.0-96.0 │ 0
187
+ 96.0-108.0 │ 0
188
+ 108.0-120.0 │ 0
189
+
190
+ ✅ 测试完成!
191
+ ```
192
+
193
+ ## Metrics Explained
194
+
195
+ | Metric | Description |
196
+ |--------|-------------|
197
+ | **TTFT** | Time to First Token - latency from request to first token arrival |
198
+ | **Total Time** | Complete duration from request to stream completion |
199
+ | **Total Tokens** | Number of output tokens received |
200
+ | **Average Speed** | Mean tokens per second (totalTokens / totalTime × 1000) |
201
+ | **Peak Speed** | Fastest speed measured over a sliding 10-token window |
202
+ | **TPS Curve** | Tokens received per second throughout the streaming response |
203
+
204
+ ## Development
205
+
206
+ ### Running Tests
207
+
208
+ ```bash
209
+ # Run tests
210
+ npm test
211
+
212
+ # Run tests with UI
213
+ npm run test:ui
214
+
215
+ # Generate coverage report
216
+ npm run test:coverage
217
+ ```
218
+
219
+ ### Building
220
+
221
+ ```bash
222
+ npm run build
223
+ ```
224
+
225
+ ### Release
226
+
227
+ This project uses [semantic-release](https://github.com/semantic-release/semantic-release) for automated versioning and publishing.
228
+
229
+ Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/) specification:
230
+
231
+ ```bash
232
+ # Patch release (1.0.0 -> 1.0.1)
233
+ git commit -m "fix: fix some bug"
234
+
235
+ # Minor release (1.0.0 -> 1.1.0)
236
+ git commit -m "feat: add new feature"
237
+
238
+ # Major release (1.0.0 -> 2.0.0)
239
+ git commit -m "feat: add breaking change\n\nBREAKING CHANGE: deprecate old API"
240
+ ```
241
+
242
+ After pushing to `master` branch, GitHub Actions will automatically:
243
+ - Analyze commit types to determine version
244
+ - Update CHANGELOG.md
245
+ - Create Git tag
246
+ - Publish to npm
247
+ - Create GitHub Release
248
+
249
+ ## Test Coverage
250
+
251
+ This project maintains high code coverage:
252
+
253
+ | Coverage Type | Percentage |
254
+ |---------------|------------|
255
+ | Statements | 99.19% |
256
+ | Branches | 94.73% |
257
+ | Functions | 100% |
258
+
259
+ ## License
260
+
261
+ MIT © [Cansiny0320](https://github.com/Cansiny0320)
262
+
263
+ ## Contributing
264
+
265
+ Contributions are welcome! Please feel free to submit a Pull Request.
266
+
267
+ ## Links
268
+
269
+ - [npm Package](https://www.npmjs.com/package/token-speed-tester)
270
+ - [GitHub Repository](https://github.com/Cansiny0320/token-speed-tester)
271
+ - [Issues](https://github.com/Cansiny0320/token-speed-tester/issues)
272
+ - [Changelog](https://github.com/Cansiny0320/token-speed-tester/blob/master/CHANGELOG.md)
package/README.md CHANGED
@@ -1,97 +1,115 @@
1
- # Token Speed Tester
1
+ # Token 速度测试工具
2
2
 
3
- A CLI tool to test LLM API token output speed for Anthropic and OpenAI compatible APIs.
3
+ > 用于测量和分析 LLM API Token 流式输出性能的命令行工具
4
4
 
5
- ## Features
5
+ [English](README.en.md) |
6
6
 
7
- - **Dual Protocol Support**: Works with both Anthropic Messages API and OpenAI Chat Completions API
8
- - **Streaming Performance**: Measures each token's arrival time with precision
9
- - **Comprehensive Metrics**: TTFT, average speed, peak speed, TPS curves
10
- - **Statistical Analysis**: Mean, min, max, and standard deviation across multiple test runs
11
- - **ASCII Visualization**: Beautiful terminal-based charts and tables
12
- - **Custom Endpoints**: Test third-party APIs that are compatible with OpenAI/Anthropic protocols
7
+ [![npm 版本](https://badge.fury.io/js/token-speed-tester.svg)](https://www.npmjs.com/package/token-speed-tester)
8
+ [![开源协议](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
9
+ [![测试](https://github.com/Cansiny0320/token-speed-tester/actions/workflows/publish.yml/badge.svg)](https://github.com/Cansiny0320/token-speed-tester/actions)
13
10
 
14
- ## Installation
11
+ 一个强大的命令行工具，用于测试 LLM API 的 Token 输出速度。支持 **Anthropic** 和 **OpenAI** 兼容的 API，提供详细指标包括 TTFT（首字延迟）、吞吐量、峰值速度以及多次运行的统计分析。
15
12
 
16
- ### Global Installation
13
+ ## 特性
14
+
15
+ - **双协议支持**：兼容 Anthropic Messages API 和 OpenAI Chat Completions API
16
+ - **流式性能测量**：以毫秒级精度记录每个 Token 的到达时间
17
+ - **全面指标**：
18
+ - **TTFT**（首字延迟）：首个 Token 到达前的延迟
19
+ - **平均速度**：每秒平均 Token 数
20
+ - **峰值速度**：10 个 Token 滑动窗口内的最快速度
21
+ - **TPS 曲线**：整个流式响应中每秒接收的 Token 数
22
+ - **统计分析**：多次测试运行的均值、最小值、最大值和标准差
23
+ - **ASCII 可视化**：精美的终端图表和数据表格
24
+ - **自定义端点**：测试兼容 OpenAI/Anthropic 协议的第三方 API
25
+
26
+ ## 安装
27
+
28
+ ### 全局安装（推荐）
17
29
 
18
30
  ```bash
19
31
  npm install -g token-speed-tester
20
32
  ```
21
33
 
22
- ### Local Installation
34
+ ### 使用 npx（无需安装）
35
+
36
+ ```bash
37
+ npx token-speed-tester --api-key sk-xxx
38
+ ```
39
+
40
+ ### 本地安装
23
41
 
24
42
  ```bash
25
43
  npm install token-speed-tester
26
44
  ```
27
45
 
28
- ## Usage
46
+ ## 使用方法
29
47
 
30
- ### Basic Usage
48
+ ### 基本用法
31
49
 
32
50
  ```bash
33
- # Test Anthropic API (default)
51
+ # 测试 Anthropic API（默认）
34
52
  token-speed-test --api-key sk-ant-xxx
35
53
 
36
- # Test OpenAI API
54
+ # 测试 OpenAI API
37
55
  token-speed-test --api-key sk-xxx --provider openai
38
56
  ```
39
57
 
40
- ### Advanced Options
58
+ ### 高级选项
41
59
 
42
60
  ```bash
43
- # Custom model and multiple test runs
61
+ # 自定义模型和多次测试
44
62
  token-speed-test \
45
63
  --api-key sk-ant-xxx \
46
64
  --provider anthropic \
47
65
  --model claude-3-5-sonnet-20241022 \
48
66
  --runs 5
49
67
 
50
- # Test with custom endpoint and prompt
68
+ # 测试自定义端点和提示词
51
69
  token-speed-test \
52
70
  --api-key sk-xxx \
53
71
  --provider openai \
54
72
  --url https://api.example.com/v1 \
55
73
  --model custom-model \
56
- --prompt "Explain quantum computing" \
74
+ --prompt "解释量子计算" \
57
75
  --max-tokens 2048 \
58
76
  --runs 10
59
77
  ```
60
78
 
61
- ### Local Development
79
+ ### 本地开发
62
80
 
63
81
  ```bash
64
- # Clone and install dependencies
82
+ # 克隆并安装依赖
65
83
  git clone https://github.com/Cansiny0320/token-speed-tester.git
66
84
  cd token-speed-tester
67
85
  npm install
68
86
 
69
- # Run directly with tsx
87
+ # 使用 tsx 直接运行
70
88
  npm run dev -- --api-key sk-ant-xxx
71
89
 
72
- # Or build and run
90
+ # 或构建后运行
73
91
  npm run build
74
92
  node dist/index.js --api-key sk-ant-xxx
75
93
  ```
76
94
 
77
- ## Command Line Options
95
+ ## 命令行选项
78
96
 
79
- | Option | Short | Description | Default |
80
- |--------|-------|-------------|---------|
81
- | `--api-key` | `-k` | API Key (required) | - |
82
- | `--provider` | `-p` | API type: `anthropic` or `openai` | `anthropic` |
83
- | `--model` | `-m` | Model name | Auto-selected by provider |
84
- | `--url` | `-u` | Custom API endpoint | Official endpoint |
85
- | `--runs` | `-r` | Number of test runs | `3` |
86
- | `--prompt` | | Test prompt | "写一篇关于 AI 的短文" |
87
- | `--max-tokens` | | Maximum output tokens | `1024` |
97
+ | 选项 | 简写 | 说明 | 默认值 |
98
+ |------|------|------|--------|
99
+ | `--api-key` | `-k` | API Key（必填） | - |
100
+ | `--provider` | `-p` | API 类型：`anthropic` 或 `openai` | `anthropic` |
101
+ | `--model` | `-m` | 模型名称 | 根据提供商自动选择 |
102
+ | `--url` | `-u` | 自定义 API 端点 | 官方端点 |
103
+ | `--runs` | `-r` | 测试次数 | `3` |
104
+ | `--prompt` | | 测试提示词 | "写一篇关于 AI 的短文" |
105
+ | `--max-tokens` | | 最大输出 Token 数 | `1024` |
88
106
 
89
- ### Default Models
107
+ ### 默认模型
90
108
 
91
109
  - **Anthropic**: `claude-3-5-sonnet-20241022`
92
110
  - **OpenAI**: `gpt-4o-mini`
93
111
 
94
- ## Output Example
112
+ ## 输出示例
95
113
 
96
114
  ```
97
115
  🚀 Token 速度测试工具
@@ -172,44 +190,83 @@ TPS 分布
172
190
  ✅ 测试完成!
173
191
  ```
174
192
 
175
- ## Metrics Explained
193
+ ## 指标说明
176
194
 
177
- - **TTFT (Time to First Token)**: Time from request start to first token arrival
178
- - **Total Time**: Total duration from request to completion
179
- - **Total Tokens**: Number of output tokens received
180
- - **Average Speed**: Mean tokens per second (totalTokens / totalTime)
181
- - **Peak Speed**: Fastest speed measured over a 10-token window
182
- - **TPS Curve**: Tokens received per second throughout the stream
195
+ | 指标 | 说明 |
196
+ |------|------|
197
+ | **TTFT** | 首字延迟 - 从请求到首个 Token 到达的时间 |
198
+ | **总耗时** | 从请求到流式响应完成的完整时长 |
199
+ | **总 Token 数** | 接收到的输出 Token 数量 |
200
+ | **平均速度** | 每秒平均 Token 数（totalTokens / totalTime × 1000） |
201
+ | **峰值速度** | 10 个 Token 滑动窗口内测量的最快速度 |
202
+ | **TPS 曲线** | 整个流式响应中每秒接收的 Token 数 |
183
203
 
184
- ## Development
204
+ ## 开发
185
205
 
186
- ### Running Tests
206
+ ### 运行测试
187
207
 
188
208
  ```bash
189
- # Run tests
209
+ # 运行测试
190
210
  npm test
191
211
 
192
- # Run tests with UI
212
+ # 使用 UI 运行测试
193
213
  npm run test:ui
194
214
 
195
- # Generate coverage report
215
+ # 生成覆盖率报告
196
216
  npm run test:coverage
197
217
  ```
198
218
 
199
- ### Building
219
+ ### 构建
200
220
 
201
221
  ```bash
202
222
  npm run build
203
223
  ```
204
224
 
205
- ### Test Coverage
225
+ ### 发布
226
+
227
+ 本项目使用 [semantic-release](https://github.com/semantic-release/semantic-release) 自动管理版本和发布。
228
+
229
+ 提交信息遵循 [约定式提交](https://www.conventionalcommits.org/zh-hans/) 规范：
230
+
231
+ ```bash
232
+ # 补丁版本 (1.0.0 -> 1.0.1)
233
+ git commit -m "fix: 修复某个问题"
234
+
235
+ # 次要版本 (1.0.0 -> 1.1.0)
236
+ git commit -m "feat: 添加新功能"
237
+
238
+ # 主要版本 (1.0.0 -> 2.0.0)
239
+ git commit -m "feat: 添加破坏性变更\n\nBREAKING CHANGE: 废弃旧 API"
240
+ ```
241
+
242
+ 推送到 `master` 分支后，GitHub Actions 会自动：
243
+ - 分析 commit 类型确定版本号
244
+ - 更新 CHANGELOG.md
245
+ - 创建 Git tag
246
+ - 发布到 npm
247
+ - 创建 GitHub Release
248
+
249
+ ## 测试覆盖率
250
+
251
+ 本项目保持高代码覆盖率：
252
+
253
+ | 覆盖率类型 | 百分比 |
254
+ |---------------|------------|
255
+ | 语句覆盖率 | 99.19% |
256
+ | 分支覆盖率 | 94.73% |
257
+ | 函数覆盖率 | 100% |
258
+
259
+ ## 开源协议
206
260
 
207
- This project maintains 100% code coverage across all modules.
261
+ MIT © [Cansiny0320](https://github.com/Cansiny0320)
208
262
 
209
- ## License
263
+ ## 贡献
210
264
 
211
- MIT License - see [LICENSE](LICENSE) for details.
265
+ 欢迎贡献！请随时提交 Pull Request。
212
266
 
213
- ## Contributing
267
+ ## 相关链接
214
268
 
215
- Contributions are welcome! Please feel free to submit a Pull Request.
269
+ - [npm 包](https://www.npmjs.com/package/token-speed-tester)
270
+ - [GitHub 仓库](https://github.com/Cansiny0320/token-speed-tester)
271
+ - [问题反馈](https://github.com/Cansiny0320/token-speed-tester/issues)
272
+ - [更新日志](https://github.com/Cansiny0320/token-speed-tester/blob/master/CHANGELOG.md)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "token-speed-tester",
3
- "version": "1.0.3",
3
+ "version": "1.3.0",
4
4
  "description": "A CLI tool to test LLM API token output speed",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
@@ -10,6 +10,7 @@
10
10
  "files": [
11
11
  "dist",
12
12
  "README.md",
13
+ "README.en.md",
13
14
  "LICENSE"
14
15
  ],
15
16
  "scripts": {
@@ -54,6 +55,9 @@
54
55
  },
55
56
  "devDependencies": {
56
57
  "@eslint/js": "^9.39.2",
58
+ "@semantic-release/changelog": "^6.0.3",
59
+ "@semantic-release/git": "^10.0.1",
60
+ "@semantic-release/npm": "^13.1.3",
57
61
  "@types/node": "^22.10.6",
58
62
  "@vitest/coverage-v8": "^2.1.8",
59
63
  "@vitest/ui": "^2.1.8",
@@ -62,6 +66,7 @@
62
66
  "eslint-plugin-prettier": "^5.5.4",
63
67
  "parser": "^0.1.4",
64
68
  "prettier": "^3.7.4",
69
+ "semantic-release": "^25.0.2",
65
70
  "tsup": "^8.5.1",
66
71
  "tsx": "^4.19.2",
67
72
  "typescript": "^5.7.3",