servediff 0.3.2__tar.gz → 0.3.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {servediff-0.3.2 → servediff-0.3.4}/CHANGELOG.md +17 -0
- {servediff-0.3.2 → servediff-0.3.4}/PKG-INFO +8 -6
- {servediff-0.3.2 → servediff-0.3.4}/README.md +7 -5
- {servediff-0.3.2 → servediff-0.3.4}/pyproject.toml +1 -1
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff/cli.py +14 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff.egg-info/PKG-INFO +8 -6
- {servediff-0.3.2 → servediff-0.3.4}/tests/test_cli.py +44 -0
- {servediff-0.3.2 → servediff-0.3.4}/CONTRIBUTING.md +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/LICENSE +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/MANIFEST.in +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/README.zh-CN.md +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/examples/issue-excerpts/vllm-47047-v0.23-default.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/examples/issue-excerpts/vllm-47047-v0.23-marlin.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/examples/issue-excerpts/vllm-53472-v0.23.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/examples/issue-excerpts/vllm-53472-v0.26-flag.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/examples/issue-excerpts/vllm-53472-v0.26.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/setup.cfg +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff/__init__.py +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff/__main__.py +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff/env.py +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff.egg-info/SOURCES.txt +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff.egg-info/dependency_links.txt +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff.egg-info/entry_points.txt +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff.egg-info/requires.txt +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/src/servediff.egg-info/top_level.txt +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/fixtures/README.md +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/fixtures/vllm-47047-v0.23-default.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/fixtures/vllm-47047-v0.23-marlin.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/fixtures/vllm-53472-v0.23.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/fixtures/vllm-53472-v0.26-flag.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/fixtures/vllm-53472-v0.26.log +0 -0
- {servediff-0.3.2 → servediff-0.3.4}/tests/test_env.py +0 -0
|
@@ -1,5 +1,22 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.4 — 2026-11-02
|
|
4
|
+
|
|
5
|
+
- Verified the parallel-config rules against a real TP=8 startup log (vLLM 0.23.1,
|
|
6
|
+
issue #48591): every rank prints its own `world_size=N` line and the role
|
|
7
|
+
assignment has no ETP field. `parallel.world_size`, `parallel.roles`,
|
|
8
|
+
`compile.fusions` and `model_runner` all parse correctly; added a regression test
|
|
9
|
+
distilled from that log.
|
|
10
|
+
|
|
11
|
+
## 0.3.3 — 2026-11-02
|
|
12
|
+
|
|
13
|
+
- Extract `model_loading.gib` / `model_loading.seconds` and the torch.compile
|
|
14
|
+
outcome (`compile.artifacts`, `compile.total_s`); detect the V1 runner via the
|
|
15
|
+
`gpu_model_runner.py` module when the "Using VN Model Runner" line is absent
|
|
16
|
+
(it is V2-only). All found by continuing to dogfood the full 0.28-V1 / 0.30-V2
|
|
17
|
+
startup logs: a V1-vs-V2 diff now shows the runner change, model-loading time,
|
|
18
|
+
compile artifacts and CUDA-graph capture differences.
|
|
19
|
+
|
|
3
20
|
## 0.3.2 — 2026-11-02
|
|
4
21
|
|
|
5
22
|
- Extract the actual parallel configuration (`parallel.world_size` and the
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: servediff
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.4
|
|
4
4
|
Summary: Diff what your vLLM / SGLang server actually ran with: auto-selected backends, CUDA graphs, startup failures, dependency versions.
|
|
5
5
|
License-Expression: Apache-2.0
|
|
6
6
|
Project-URL: Homepage, https://github.com/apex-mochen/servediff
|
|
@@ -148,9 +148,11 @@ contains `TOKEN`, `KEY`, `SECRET`, `PASSWORD`, `CREDENTIAL` or `AUTH` are replac
|
|
|
148
148
|
| attention backend | `Using X attention backend` | `attention_backend` arg |
|
|
149
149
|
| MoE backend | three log formats seen in 0.23–0.30 | `moe_runner_backend` arg |
|
|
150
150
|
| CUDA graph | eager mode, `cudagraph_mode`, capture sets, pool size | prefill CUDA graph disabled |
|
|
151
|
-
| model runner, UVA fallback | `Using V2 Model Runner
|
|
151
|
+
| model runner, UVA fallback | `Using V2 Model Runner`; V1 runner detected via `gpu_model_runner.py` module; pinned-memory fallback | — |
|
|
152
152
|
| parallel config | `world_size`, DP/PP/PCP/TP/ETP/EP/EPLB role assignment | — |
|
|
153
153
|
| KV cache layout | `Using LBNHC KV cache layout` (0.30+) | — |
|
|
154
|
+
| model loading | `Model loading took … GiB memory and … seconds` | — |
|
|
155
|
+
| torch.compile | `collected artifacts: …`, `torch.compile took … s in total` | — |
|
|
154
156
|
| startup outcome | ready / failed + root-cause error | ready |
|
|
155
157
|
|
|
156
158
|
Every fact keeps the log line it came from (`servediff show <log>` or `--json`).
|
|
@@ -187,10 +189,10 @@ Exit codes: `0` no differences, `1` differences, `2` nothing recognised.
|
|
|
187
189
|
## Limits
|
|
188
190
|
|
|
189
191
|
- Tested on: log excerpts posted in vllm#47047 (v0.23) and vllm#53472 (v0.23, v0.26);
|
|
190
|
-
full startup logs of vLLM 0.28.0 and 0.30.0 and one SGLang log
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
192
|
+
full startup logs of vLLM 0.28.0 and 0.30.0 and one SGLang log (single-GPU dense);
|
|
193
|
+
a real TP=8 startup log from vllm#48591 (v0.23.1, one line per worker + role
|
|
194
|
+
assignment). The parallel-config rules cover TP>1 real logs; MoE models in full
|
|
195
|
+
logs and other versions are not yet verified.
|
|
194
196
|
- Parsing is regex-based on log text, which changes between versions. A missed line
|
|
195
197
|
shows up as `<absent>`, never as a wrong value — please report it.
|
|
196
198
|
|
|
@@ -127,9 +127,11 @@ contains `TOKEN`, `KEY`, `SECRET`, `PASSWORD`, `CREDENTIAL` or `AUTH` are replac
|
|
|
127
127
|
| attention backend | `Using X attention backend` | `attention_backend` arg |
|
|
128
128
|
| MoE backend | three log formats seen in 0.23–0.30 | `moe_runner_backend` arg |
|
|
129
129
|
| CUDA graph | eager mode, `cudagraph_mode`, capture sets, pool size | prefill CUDA graph disabled |
|
|
130
|
-
| model runner, UVA fallback | `Using V2 Model Runner
|
|
130
|
+
| model runner, UVA fallback | `Using V2 Model Runner`; V1 runner detected via `gpu_model_runner.py` module; pinned-memory fallback | — |
|
|
131
131
|
| parallel config | `world_size`, DP/PP/PCP/TP/ETP/EP/EPLB role assignment | — |
|
|
132
132
|
| KV cache layout | `Using LBNHC KV cache layout` (0.30+) | — |
|
|
133
|
+
| model loading | `Model loading took … GiB memory and … seconds` | — |
|
|
134
|
+
| torch.compile | `collected artifacts: …`, `torch.compile took … s in total` | — |
|
|
133
135
|
| startup outcome | ready / failed + root-cause error | ready |
|
|
134
136
|
|
|
135
137
|
Every fact keeps the log line it came from (`servediff show <log>` or `--json`).
|
|
@@ -166,10 +168,10 @@ Exit codes: `0` no differences, `1` differences, `2` nothing recognised.
|
|
|
166
168
|
## Limits
|
|
167
169
|
|
|
168
170
|
- Tested on: log excerpts posted in vllm#47047 (v0.23) and vllm#53472 (v0.23, v0.26);
|
|
169
|
-
full startup logs of vLLM 0.28.0 and 0.30.0 and one SGLang log
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
171
|
+
full startup logs of vLLM 0.28.0 and 0.30.0 and one SGLang log (single-GPU dense);
|
|
172
|
+
a real TP=8 startup log from vllm#48591 (v0.23.1, one line per worker + role
|
|
173
|
+
assignment). The parallel-config rules cover TP>1 real logs; MoE models in full
|
|
174
|
+
logs and other versions are not yet verified.
|
|
173
175
|
- Parsing is regex-based on log text, which changes between versions. A missed line
|
|
174
176
|
shows up as `<absent>`, never as a wrong value — please report it.
|
|
175
177
|
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "servediff"
|
|
7
|
-
version = "0.3.
|
|
7
|
+
version = "0.3.4"
|
|
8
8
|
description = "Diff what your vLLM / SGLang server actually ran with: auto-selected backends, CUDA graphs, startup failures, dependency versions."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.10"
|
|
@@ -76,6 +76,10 @@ LINE_RULES: list[tuple[re.Pattern, callable]] = [
|
|
|
76
76
|
(re.compile(r"FlashInfer top-p/top-k sampling (disabled|enabled)"),
|
|
77
77
|
lambda m: [("backend.sampling_flashinfer", m[1])]),
|
|
78
78
|
(re.compile(r"Using (V\d) Model Runner"), lambda m: [("model_runner", m[1])]),
|
|
79
|
+
# V1 runner predates the "Using VN Model Runner" line; its identity is the
|
|
80
|
+
# gpu_model_runner.py module (V2 uses model_runner.py).
|
|
81
|
+
(re.compile(r"\[gpu_model_runner\.py:\d+\]"),
|
|
82
|
+
lambda m: [("model_runner", "V1")]),
|
|
79
83
|
# Parallel configuration: what TP/EP/PP the process actually runs with. This is
|
|
80
84
|
# the only place TP>1 (and expert parallel) shows up in the startup log.
|
|
81
85
|
(re.compile(r"world_size=(\d+) rank=(\d+) local_rank=(\d+)"),
|
|
@@ -83,6 +87,16 @@ LINE_RULES: list[tuple[re.Pattern, callable]] = [
|
|
|
83
87
|
(re.compile(r"rank \d+ in world size \d+ is assigned as (DP rank \d+, PP rank \d+, PCP rank \d+, TP rank \d+(?:, ETP rank \S+)?, EP rank \S+, EPLB rank \S+)"),
|
|
84
88
|
lambda m: [("parallel.roles", m[1])]),
|
|
85
89
|
(re.compile(r"Using (\S+) KV cache layout"), lambda m: [("kv_cache.layout", m[1])]),
|
|
90
|
+
# Model loading cost: memory footprint and wall time. Both V1 (gpu_model_runner)
|
|
91
|
+
# and V2 (model_runner) print the same message.
|
|
92
|
+
(re.compile(r"Model loading took ([\d.]+) GiB memory and ([\d.]+) seconds"),
|
|
93
|
+
lambda m: [("model_loading.gib", float(m[1])), ("model_loading.seconds", float(m[2]))]),
|
|
94
|
+
# torch.compile outcome: artifact count/size reflects compile behaviour; total
|
|
95
|
+
# wall time is comparable between versions on the same machine.
|
|
96
|
+
(re.compile(r"collected artifacts: (.+?) total"),
|
|
97
|
+
lambda m: [("compile.artifacts", m[1].strip())]),
|
|
98
|
+
(re.compile(r"torch.compile took ([\d.]+) s in total"),
|
|
99
|
+
lambda m: [("compile.total_s", float(m[1]))]),
|
|
86
100
|
(re.compile(r"Pinned memory is not available.*falling back to (.+?) for UVA buffers"),
|
|
87
101
|
lambda m: [("uva_buffers", f"fallback: {m[1]}")]),
|
|
88
102
|
# Startup outcome. Without it, facts missing after a crash look like "not printed".
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: servediff
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.4
|
|
4
4
|
Summary: Diff what your vLLM / SGLang server actually ran with: auto-selected backends, CUDA graphs, startup failures, dependency versions.
|
|
5
5
|
License-Expression: Apache-2.0
|
|
6
6
|
Project-URL: Homepage, https://github.com/apex-mochen/servediff
|
|
@@ -148,9 +148,11 @@ contains `TOKEN`, `KEY`, `SECRET`, `PASSWORD`, `CREDENTIAL` or `AUTH` are replac
|
|
|
148
148
|
| attention backend | `Using X attention backend` | `attention_backend` arg |
|
|
149
149
|
| MoE backend | three log formats seen in 0.23–0.30 | `moe_runner_backend` arg |
|
|
150
150
|
| CUDA graph | eager mode, `cudagraph_mode`, capture sets, pool size | prefill CUDA graph disabled |
|
|
151
|
-
| model runner, UVA fallback | `Using V2 Model Runner
|
|
151
|
+
| model runner, UVA fallback | `Using V2 Model Runner`; V1 runner detected via `gpu_model_runner.py` module; pinned-memory fallback | — |
|
|
152
152
|
| parallel config | `world_size`, DP/PP/PCP/TP/ETP/EP/EPLB role assignment | — |
|
|
153
153
|
| KV cache layout | `Using LBNHC KV cache layout` (0.30+) | — |
|
|
154
|
+
| model loading | `Model loading took … GiB memory and … seconds` | — |
|
|
155
|
+
| torch.compile | `collected artifacts: …`, `torch.compile took … s in total` | — |
|
|
154
156
|
| startup outcome | ready / failed + root-cause error | ready |
|
|
155
157
|
|
|
156
158
|
Every fact keeps the log line it came from (`servediff show <log>` or `--json`).
|
|
@@ -187,10 +189,10 @@ Exit codes: `0` no differences, `1` differences, `2` nothing recognised.
|
|
|
187
189
|
## Limits
|
|
188
190
|
|
|
189
191
|
- Tested on: log excerpts posted in vllm#47047 (v0.23) and vllm#53472 (v0.23, v0.26);
|
|
190
|
-
full startup logs of vLLM 0.28.0 and 0.30.0 and one SGLang log
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
192
|
+
full startup logs of vLLM 0.28.0 and 0.30.0 and one SGLang log (single-GPU dense);
|
|
193
|
+
a real TP=8 startup log from vllm#48591 (v0.23.1, one line per worker + role
|
|
194
|
+
assignment). The parallel-config rules cover TP>1 real logs; MoE models in full
|
|
195
|
+
logs and other versions are not yet verified.
|
|
194
196
|
- Parsing is regex-based on log text, which changes between versions. A missed line
|
|
195
197
|
shows up as `<absent>`, never as a wrong value — please report it.
|
|
196
198
|
|
|
@@ -214,6 +214,50 @@ def test_tp4_ep_parallel_roles_are_parsed():
|
|
|
214
214
|
assert f["kv_cache.layout"].value == "LBNHC"
|
|
215
215
|
|
|
216
216
|
|
|
217
|
+
def test_model_loading_and_compile_facts():
|
|
218
|
+
text = (
|
|
219
|
+
"(EngineCore pid=100) INFO 10-28 11:28:34 [model_runner.py:428] Model loading took 0.93 GiB memory and 19.122619 seconds\n"
|
|
220
|
+
"(EngineCore pid=100) INFO 10-28 11:29:11 [backends.py:920] collected artifacts: 25 entries, 3 artifacts, 4996920 bytes total\n"
|
|
221
|
+
"(EngineCore pid=100) INFO 10-28 11:29:11 [monitor.py:53] torch.compile took 31.08 s in total\n")
|
|
222
|
+
f = servediff.parse(text)["facts"]
|
|
223
|
+
assert f["model_loading.gib"].value == 0.93
|
|
224
|
+
assert abs(f["model_loading.seconds"].value - 19.122619) < 1e-6
|
|
225
|
+
assert f["compile.artifacts"].value == "25 entries, 3 artifacts, 4996920 bytes"
|
|
226
|
+
assert abs(f["compile.total_s"].value - 31.08) < 1e-9
|
|
227
|
+
|
|
228
|
+
|
|
229
|
+
def test_v1_runner_detected_from_gpu_model_runner_module():
|
|
230
|
+
text = ("(EngineCore pid=100) INFO 10-28 11:35:38 "
|
|
231
|
+
"[gpu_model_runner.py:5419] Starting to load model /models/Qwen\n")
|
|
232
|
+
f = servediff.parse(text)["facts"]
|
|
233
|
+
assert f["model_runner"].value == "V1"
|
|
234
|
+
|
|
235
|
+
|
|
236
|
+
# Real TP>1 log excerpt, distilled from vLLM issue #48591 (0.23.1, TP=8,
|
|
237
|
+
# dspark spec decode crash): every rank prints its own world_size line, the
|
|
238
|
+
# assigned-as line has no ETP field, and the run fails at CUDA graph time.
|
|
239
|
+
TP8 = """\
|
|
240
|
+
(Worker pid=100) INFO 07-14 16:40:12 [parallel_state.py:1607] world_size=8 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:53729 backend=nccl
|
|
241
|
+
(Worker pid=101) INFO 07-14 16:40:13 [parallel_state.py:1607] world_size=8 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:53729 backend=nccl
|
|
242
|
+
(Worker pid=102) INFO 07-14 16:40:13 [parallel_state.py:1607] world_size=8 rank=7 local_rank=7 distributed_init_method=tcp://127.0.0.1:53729 backend=nccl
|
|
243
|
+
(Worker pid=103) INFO 07-14 16:40:23 [parallel_state.py:1942] rank 0 in world size 8 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
|
|
244
|
+
(Worker pid=104) INFO 07-14 16:40:23 [gpu_worker.py:378] Using V2 Model Runner
|
|
245
|
+
INFO 07-14 16:38:11 [compilation.py:312] Enabled custom fusions: allreduce_rms
|
|
246
|
+
(Worker_TP7 pid=105) INFO 07-14 16:40:24 [model_runner.py:281] Loading model from scratch...
|
|
247
|
+
(Worker_TP7 pid=106) ERROR 07-14 16:40:44 [model_runner.py:999] RuntimeError: Engine core initialization failed. See root cause above.
|
|
248
|
+
"""
|
|
249
|
+
|
|
250
|
+
|
|
251
|
+
def test_tp8_multiple_worker_lines_parse_parallel_config():
|
|
252
|
+
f = servediff.parse(TP8)["facts"]
|
|
253
|
+
assert f["parallel.world_size"].value == 8
|
|
254
|
+
assert f["parallel.roles"].value == ("DP rank 0, PP rank 0, PCP rank 0, TP rank 0, "
|
|
255
|
+
"EP rank N/A, EPLB rank N/A")
|
|
256
|
+
assert f["compile.fusions"].value == "allreduce_rms"
|
|
257
|
+
assert f["model_runner"].value == "V2"
|
|
258
|
+
assert f["startup"].value == "failed"
|
|
259
|
+
|
|
260
|
+
|
|
217
261
|
def test_roles_without_etp_field_and_kv_layout_absent():
|
|
218
262
|
text = ("(EngineCore pid=3811) INFO 10-28 11:26:32 [parallel_state.py:1638] "
|
|
219
263
|
"world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_x backend=nccl\n"
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|