xinfer-ai 0.14.6 → 0.14.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +6 -6
  2. package/package.json +2 -2
package/README.md CHANGED
@@ -114,17 +114,17 @@ Add `--kvcache-dtype` to compress KV cache and extend context length:
114
114
  | Ministral-3-3B (**Multimodal**) | ISQ (BF16→Q4K) | 3B | **193.67** tokens/s |
115
115
  | Qwen3-VL-8B-Instruct (**Multimodal**) | Q8_0 | 8B | **112.51** tokens/s |
116
116
  | Llama-3.1-8B | ISQ (BF16→Q4K) | 8B | **133.10** tokens/s |
117
- | DeepSeek-R1-0528-Qwen3-8B | Q4_K_M | 8B | **139.25** tokens/s |
118
- | GLM-4-9B-0414 | Q4_K_M | 9B | **77.48** tokens/s |
119
- | QwQ-32B | Q4_K_M | 32B | **46.02** tokens/s |
117
+ | DeepSeek-R1-0528-Qwen3-8B | Q4_K_M | 8B | **161.60** tokens/s |
118
+ | GLM-4-9B-0414 | Q4_K_M | 9B | **92.10** tokens/s |
119
+ | QwQ-32B | Q4_K_M | 32B | **55.65** tokens/s |
120
120
  | **Qwen3-30B-A3B** | NVFP4 | **30B (MoE)** | **197.29** tokens/s (**RTX 5090**) |
121
121
  | **Qwen3-30B-A3B** | NVFP4 | **30B (MoE)** | **72.86** tokens/s (**V100, Software FP4**) |
122
122
  | **Qwen3-Next-80B** | AWQ | **80B (MoE)** | **90** tokens/s (**Hopper**) |
123
- | **Qwen3.5/3.6/3.8 27B** (**Multimodal**) | Q4_K_M/IQ4 | **27B (Dense)** | **58** tokens/s |
124
- | **Qwen3.5/3.6/3.8 27B** | FP8 | **27B (Dense)** | **45** tokens/s (**Hopper**) |
123
+ | **Qwen3.5/3.6/3.8 27B** (**Multimodal**) | Q4_K_M/IQ4 | **27B (Dense)** | **61.9** tokens/s |
124
+ | **Qwen3.5/3.6/3.8 27B** | FP8 | **27B (Dense)** | **52.2** tokens/s (**Hopper**) |
125
125
  | **Qwen3.6-35B-A3B** (**Multimodal**) | FP8 | **35B (MoE)** | **120** tokens/s (**Hopper**) |
126
126
  | **GLM4.7 Flash** | NVFP4 | **30B (MoE)** | **79** tokens/s (**Hopper, Software FP4**) |
127
- | **Gemma4-31B** | ISQ (BF16→Q4K) | **31B (Dense)** | **47** tokens/s (**Hopper**) |
127
+ | **Gemma4-31B** | ISQ (BF16→Q4K) | **31B (Dense)** | **55** tokens/s (**Hopper**) |
128
128
  | **Gemma4-26B-A4B** | NVFP4 | **26B (MoE)** | **137.23** tokens/s (**RTX 5090**) |
129
129
  | **MiniMax-M2.5** | NVFP4 | **229B (MoE)** | **64.50** tokens/s (**Hopper, Software FP4, TP=2**) |
130
130
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "xinfer-ai",
3
- "version": "0.14.6",
4
- "assetVersion": "0.14.6",
3
+ "version": "0.14.7",
4
+ "assetVersion": "0.14.7",
5
5
  "description": "xInfer — a high-performance LLM inference engine in Rust with CUDA/Metal acceleration",
6
6
  "license": "MIT",
7
7
  "homepage": "https://github.com/guoqingbao/xinfer#readme",