xinfer-ai 0.14.3 → 0.14.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +3 -3
  2. package/package.json +2 -2
package/README.md CHANGED
@@ -99,8 +99,8 @@ Add `--kvcache-dtype` to compress KV cache and extend context length:
99
99
  | **Qwen3-30B-A3B** | NVFP4 | **30B (MoE)** | **197.29** tokens/s (**RTX 5090**) |
100
100
  | **Qwen3-30B-A3B** | NVFP4 | **30B (MoE)** | **72.86** tokens/s (**V100, Software FP4**) |
101
101
  | **Qwen3-Next-80B** | AWQ | **80B (MoE)** | **90** tokens/s (**Hopper**) |
102
- | **Qwen3.5-27B** (**Multimodal**) | Q4_K_M | **27B (Dense)** | **49.33** tokens/s |
103
- | **Qwen3.5-27B/Qwen3.6-27B** | FP8 | **27B (Dense)** | **45** tokens/s (**Hopper**) |
102
+ | **Qwen3.5/3.6/3.8 27B** (**Multimodal**) | Q4_K_M/IQ4 | **27B (Dense)** | **58** tokens/s |
103
+ | **Qwen3.5/3.6/3.8 27B** | FP8 | **27B (Dense)** | **45** tokens/s (**Hopper**) |
104
104
  | **Qwen3.6-35B-A3B** (**Multimodal**) | FP8 | **35B (MoE)** | **120** tokens/s (**Hopper**) |
105
105
  | **GLM4.7 Flash** | NVFP4 | **30B (MoE)** | **79** tokens/s (**Hopper, Software FP4**) |
106
106
  | **Gemma4-31B** | ISQ (BF16→Q4K) | **31B (Dense)** | **47** tokens/s (**Hopper**) |
@@ -133,7 +133,7 @@ Add `--kvcache-dtype` to compress KV cache and extend context length:
133
133
  * ✅ Qwen (Qwen2, Qwen3)
134
134
  * ✅ Qwen2/Qwen3 MoE
135
135
  * ✅ Qwen3 Next
136
- * ✅ Qwen3.5/3.6 Dense/MoE (27B, 35B, 122B, 397B, Multimodal model)
136
+ * ✅ Qwen3.5/3.6/3.8 Dense/MoE (27B, 35B, 122B, 397B, Multimodal model)
137
137
  * ✅ Mistral v1, v2
138
138
  * ✅ Mistral-3-VL Reasoning (3B, 8B, 14B, Multimodal model)
139
139
  * ✅ GLM4 (0414)
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "xinfer-ai",
3
- "version": "0.14.3",
4
- "assetVersion": "0.14.3",
3
+ "version": "0.14.4",
4
+ "assetVersion": "0.14.4",
5
5
  "description": "xInfer — a high-performance LLM inference engine in Rust with CUDA/Metal acceleration",
6
6
  "license": "MIT",
7
7
  "homepage": "https://github.com/guoqingbao/xinfer#readme",