xinfer-ai 0.13.1 → 0.13.2

package/README.md

@@ -46,7 +46,7 @@ xinfer --m Qwen/Qwen3.6-27B-FP8 --kvcache-dtype turbo4 --ui-server
 
 **Using local model path:**
 ```bash
-xinfer --w /home/Qwen3.6-35B-A3B --d 0,1 --ui-server
+xinfer --m /home/Qwen3.6-35B-A3B --d 0,1 --ui-server
 ```
 
 **Python usage:**
@@ -226,7 +226,7 @@ xinfer --m Qwen/Qwen3.6-35B-A3B --isq q4k --kvcache-dtype fp8
 xinfer --m Qwen/Qwen3.6-35B-A3B --isq q4k --kvcache-dtype turbo4
 
 # Metal ISQ
-xinfer --w /path/Qwen3-4B --isq q6k
+xinfer --m /path/Qwen3-4B --isq q6k
 ```
 
 </details>
@@ -239,7 +239,7 @@ xinfer --w /path/Qwen3-4B --isq q6k
 xinfer --m unsloth/Qwen3.5-27B-GGUF --f Qwen3.5-27B-Q4_K_M.gguf
 
 # Multi-GPU — GGUF
-xinfer --d 0,1 --f /path/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
+xinfer --d 0,1 --m /path/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
 ```
 
 </details>
@@ -354,7 +354,7 @@ Split prefill (prompt processing) and decode (token generation) across GPUs or m
 xinfer --d 0,1 --m Qwen/Qwen3-30B-A3B-Instruct-2507 --pd-server
 
 # PD Client (decode GPU + API)
-xinfer --d 2,3 --w /path/Qwen3-30B-A3B-Instruct-2507 --isq q4k --ui-server --port 8000 --pd-client
+xinfer --d 2,3 --m /path/Qwen3-30B-A3B-Instruct-2507 --isq q4k --ui-server --port 8000 --pd-client
 ```
 
 **Multinode** (tcp mode)
@@ -363,7 +363,7 @@ xinfer --d 2,3 --w /path/Qwen3-30B-A3B-Instruct-2507 --isq q4k --ui-server --por
 target/release/xinfer --d 0,1 --m Qwen/... --pd-server --pd-url tcp://0.0.0.0:8100
 
 # Client machine
-target/release/xinfer --d 0,1 --w /path/... --pd-client --pd-url tcp://192.168.1.100:8100 --ui-server --port 8000
+target/release/xinfer --d 0,1 --m /path/... --pd-client --pd-url tcp://192.168.1.100:8100 --ui-server --port 8000
 ```
 
 > Metal/macOS requires `--pd-url` (no LocalIPC support).
@@ -380,7 +380,7 @@ target/release/xinfer --d 0,1 --m Qwen/... --pd-server --pd-url file:///sockets
 
 # Client container
 docker run --gpus '"device=2,3"' -v /tmp/pd-sockets:/sockets ...
-target/release/xinfer --d 0,1 --w /path/... --pd-client --pd-url file:///sockets --ui-server --port 8000
+target/release/xinfer --d 0,1 --m /path/... --pd-client --pd-url file:///sockets --ui-server --port 8000
 ```
 
 </details>
@@ -432,9 +432,9 @@ Constraint-based generation via llguidance — Lark grammars, regex, JSON Schema
 
 | Flag | Description |
 |---|---|
-| `--m` | HuggingFace model ID (auto-download) |
-| `--w` | Local Safetensors model path |
-| `--f` | GGUF file path (or filename when `--m` is given) |
+| `--m` | Model source: HuggingFace model ID, local Safetensors directory, or local GGUF file |
+| `--w` | Legacy alias for local Safetensors directory; prefer `--m <local_dir>` |
+| `--f` | Local GGUF file when used alone; remote GGUF filename when paired with `--m <model_id>` |
 | `--d` | Device IDs (e.g. `--d 0,1`) |
 | `--ui-server` | API server + built-in ChatGPT-style web UI |
 | `--server` | API server only (no web UI) |

package/docs/docker.md

@@ -67,7 +67,7 @@ docker run --rm -it --gpus all --network host -v /home:/home -v /data:/data xinf
 ```
 Start the UI + API server
 ```bash
-xinfer --w /home/path/Qwen3-Coder-30B-A3B-Instruct-FP8 --ui-server
+xinfer --m /home/path/Qwen3-Coder-30B-A3B-Instruct-FP8 --ui-server
 ```
 
-**Note:** if `Ctrl+C` not working in docker, you need `Ctrl+P` then `Ctrl+Q` to stop the server.
+**Note:** if `Ctrl+C` not working in docker, you need `Ctrl+P` then `Ctrl+Q` to stop the server.

package/docs/get_started.md

@@ -20,9 +20,9 @@ cargo build --release --features metal
 ```
 
 ## Model formats
-- **Safetensors (HF layout)**: `--m <hf_id>` for cached download, or `--w <local_dir>` for offline weights + configs.
+- **Safetensors (HF layout)**: `--m <hf_id>` for cached download, or `--m <local_dir>` for offline weights + configs. `--w <local_dir>` still works as a legacy alias.
 - **Safetensors (HF layout) + ISQ**: in-situ quantize into GGUF with args `--isq <q4k|q2k|q6k|...>`.
-- **GGUF**: `--f <gguf_file>`; no configs needed.
+- **GGUF**: `--m <local.gguf>` or `--f <local.gguf>` for a local file. Use `--m <hf_id> --f <remote_file.gguf>` for a GGUF file hosted in a Hugging Face repo.
 - **Vision-Language** (Qwen3-VL, Gemma3, Mistral3-VL): require image tokens; use `--ui-server` for uploads or send image_url/base64 in the request.
 
 
@@ -38,7 +38,12 @@ cargo build --release --features metal
   ```
 - **GGUF quantized**  
   ```bash
-  target/release/xinfer --f /path/model-Q4_K_M.gguf --max-model-len 65536  ```
+  target/release/xinfer --m /path/model-Q4_K_M.gguf --max-model-len 65536
+  ```
+- **Remote GGUF quantized**  
+  ```bash
+  target/release/xinfer --m unsloth/Qwen3-0.6B-GGUF --f Qwen3-0.6B-Q4_K_M.gguf --ui-server
+  ```
 - **Embeddings** (same server; OpenAI `/v1/embeddings`)  
   ```bash
   target/release/xinfer --m Qwen/Qwen2.5-7B-Instruct  # curl -d '{"input":"hello","embedding_type":"mean"}' http://localhost:8000/v1/embeddings
@@ -46,7 +51,8 @@ cargo build --release --features metal
 - **Multimodal**  
   ```bash
   # Update image in the Chat UI
-  target/release/xinfer --m Qwen/Qwen3-VL-8B-Instruct --ui-server  ```
+  target/release/xinfer --m Qwen/Qwen3-VL-8B-Instruct --ui-server
+  ```
 
 Common runtime knobs: `--max-model-len`, `--max-num-seqs`, `--kv-fraction` (CUDA KV share), `--cpu-mem-fold` (CPU swap ratio), `--port`, `--kvcache-dtype` (fp8/turbo8/turbo4/turbo3), `--prefix-cache`, `--prefix-cache-max-tokens`, `--ui-server`, `--batch` (perf test).
 
@@ -63,11 +69,13 @@ Reasoning defaults to enabled when a request omits `thinking` / `enable_thinking
 - **PD server (prefill host, usually memory-rich)**  
   ```bash
   target/release/xinfer --pd-server --port 8000 \
-    --m Qwen/Qwen3-30B-A3B-Instruct-2507  ```
+    --m Qwen/Qwen3-30B-A3B-Instruct-2507
+  ```
 - **PD client (decode host)**  
   ```bash
   target/release/xinfer --server --pd-client --pd-url 0.0.0.0:8000 \
-    --m Qwen/Qwen3-30B-A3B-Instruct-2507  ```
+    --m Qwen/Qwen3-30B-A3B-Instruct-2507
+  ```
 - Same weights/config on both ends; Local IPC used automatically on same node CUDA, TCP when `--pd-url` is set. Monitor logs for transfer and swap events.
 
 ## Prefix cache

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "xinfer-ai",
-  "version": "0.13.1",
-  "assetVersion": "0.13.1",
+  "version": "0.13.2",
+  "assetVersion": "0.13.2",
   "description": "xInfer — a high-performance LLM inference engine in Rust with CUDA/Metal acceleration",
   "license": "MIT",
   "homepage": "https://github.com/guoqingbao/xinfer#readme",