intel-npu-server 0.3.0__tar.gz

intel_npu_server-0.3.0/PKG-INFO

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: intel-npu-server
+Version: 0.3.0
+Summary: Local OpenAI-compatible server for OpenVINO GenAI models (primarily on Intel NPU)
+Author: Alex Spirin
+License-Expression: MIT
+Requires-Dist: fastapi>=0.115,<1
+Requires-Dist: huggingface-hub>=0.34,<2
+Requires-Dist: openvino-genai==2026.4.1.0
+Requires-Dist: pydantic>=2.10,<3
+Requires-Dist: transformers==5.20.0
+Requires-Dist: uvicorn[standard]>=0.34,<1
+Requires-Dist: optimum-intel>=2.0.0
+Requires-Python: >=3.12, <3.13
+Description-Content-Type: text/markdown
+
+# LLM on Intel NPU / OpenVINO GenAI / OpenAI-compatible API
+
+Modular local inference server for OpenCode. The project uses Python 3.12 and `uv`; no PowerShell or batch scripts are required. LFM3 is the default checkpoint, but any exportable OpenVINO model works (see section 4.1).
+
+## Project layout
+
+```text
+.
+|-- pyproject.toml
+|-- uv.lock                    # generated by uv lock / uv sync
+|-- model_configuration.json   # generated automatically
+|-- opencode.jsonc
+`-- npu_server/
+    |-- __main__.py            # thin entry point
+    |-- tool_call_parsers.py
+    |-- cli/
+    |   |-- arguments_cli.py     # ModelArgumentsParser
+    |   |-- model_config_cli.py  # ModelConfig
+    |   `-- server_cli.py        # CommandLineInterface
+    |-- generation/
+    |   |-- events.py            # StreamEvent, GenerationRequest, GenerationOutcome
+    |   |-- streaming.py         # ToolSafeStreamBuffer, TokenStreamCallback
+    |   `-- engine.py            # GenerationEngine
+    |-- load_model/
+    |   |-- runtime.py           # GenerationError, ModelRuntime, RuntimeLengths
+    |   |-- preparer.py          # OpenvinoModelPreparer (download + IR export)
+    |   `-- loader.py            # OpenvinoModelLoader (pipeline compilation)
+    `-- server/
+        |-- openai_schemas.py    # CompletionResult, ChatCompletionChunkBuilder
+        |-- message_normalizer.py# ChatMessageNormalizer
+        |-- http_app.py          # NpuServer (routes + uvicorn)
+```
+
+`model_dir` defaults to `Path.cwd().parent / "model"`. The resolved path is written into `model_configuration.json` even when `--model-dir` is omitted. This ensures that a later `serve` command opens the same exported model directory.
+
+Each model lives in its own subfolder named after `model_name`, so several models can share one `model_dir`:
+
+```
+model/
+`-- lfm3-uncensored/
+    |-- checkpoint/        # source Hugging Face checkpoint (downloaded once)
+    `-- exported/          # OpenVINO IR, tokenizer files, model_manifest.json
+```
+
+The global Hugging Face cache is never used. A legacy flat layout (`model_dir\<files>` + `model_dir\checkpoint`) is migrated automatically on the first start of the new version.
+
+## 1. Install and synchronize
+
+Install `uv` once using the official installer for your operating system. Then, from the project root:
+
+```powershell
+uv python install 3.12
+uv sync
+```
+
+The `optimum-intel` dependency is installed from the author's `lfm3` development branch, and `transformers==5.20.0` is used as required by the LFM3 support PR. The package metadata on that branch may still declare an older upper bound, so `tool.uv.override-dependencies` explicitly overrides it. This resolves the dependency solver conflict; it does not itself guarantee runtime compatibility.
+
+## 2. Run the server
+
+First launch:
+
+```powershell
+uv run python -m npu_server run
+```
+
+The `run` command:
+
+1. Reads the configuration file and applies command-line overrides.
+2. Migrates a legacy flat model layout if one is found.
+3. Downloads the Hugging Face checkpoint into `model_dir\<model_name>\checkpoint` (unless `--model-id` points to a local directory).
+4. Loads the tokenizer from that checkpoint.
+5. Invokes the Optimum Intel Python CLI entry point in-process to export OpenVINO IR using INT4 symmetric weight compression and group size 128.
+6. Saves the tokenizer into `model_dir\<model_name>\exported` next to the IR files.
+7. Compiles/loads `openvino_genai.LLMPipeline` on the selected device. With the default `auto` lengths on NPU it tries `MAX_PROMPT_LEN=8192, MIN_RESPONSE_LEN=1024` first and falls back down the ladder `4096/1024 -> 2048/512 -> 1024/256` when the device refuses to compile; explicit `--max-prompt-len`/`--min-response-len` values disable the ladder.
+8. Starts the HTTP server and keeps the model in memory until the process is stopped with Ctrl+C. On shutdown (Ctrl+C or uvicorn graceful exit) the pipeline reference is dropped and garbage-collected, releasing the model and its memory from the device before the process exits.
+
+Subsequent `run` calls reuse an existing `.xml` + `.bin` IR export and the saved tokenizer.
+
+## 3. Individual commands
+
+Export only:
+
+```powershell
+uv run python -m npu_server prepare
+```
+
+Run the HTTP server using an existing IR export:
+
+```powershell
+uv run python -m npu_server serve
+```
+
+Check the local OpenVINO installation and device visibility:
+
+```powershell
+uv run python -m npu_server doctor
+```
+
+Force a fresh export:
+
+```powershell
+uv run python -m npu_server run --force-export
+```
+
+## 4. Configuration overrides
+
+All values can be stored in `model_configuration.json` or passed as command-line arguments. Command-line arguments take precedence over the JSON file. If a value is omitted, the value from JSON is used; otherwise the `ModelConfig` default is used.
+
+`model_id` and `device` are required: they must be present in `model_configuration.json` or passed as command-line arguments, otherwise startup fails with an error. `model_name` defaults to `model_id` when omitted.
+
+```powershell
+uv run python -m npu_server run `
+  --model-id OpenIntelligenceNet/LFM3-1.7B-Uncensored `
+  --model-dir ..\model `
+  --device NPU `
+  --host 127.0.0.1 `
+  --port 8000 `
+  --max-prompt-len 1024 `
+  --min-response-len 128 `
+  --max-output-tokens 128
+```
+
+`--model-dir` is optional. If omitted, it defaults to the resolved path `Path.cwd().parent / "model"`, and that value is persisted in `model_configuration.json`.
+
+Length parameters accept either a positive integer or `auto`. On NPU, `auto` resolves to safe defaults (`MAX_PROMPT_LEN=8192`, `MIN_RESPONSE_LEN=1024`) with an automatic fallback ladder when the device refuses to compile. On CPU/GPU, `auto` omits the corresponding OpenVINO property so shapes stay dynamic; `max_output_tokens=auto` leaves the generation-length limit to the library default. For tight NPU memory budgets set the lengths explicitly.
+
+## 4.1. Other models
+
+The server works with any Hugging Face checkpoint that Optimum Intel can export to OpenVINO IR; LFM3 is only the default. Pass `--model-id <repo>`, set `--model-name`, and match the tool-call format if the model supports tools:
+
+`--tool-parser` selects the converter for the model's tool-call markup:
+
+| value | markup |
+| --- | --- |
+| `auto` | guess from the model id/name: `lfm*` -> lfm3, `qwen`/`hermes`/`glm` -> hermes, otherwise none |
+| `lfm3` | LFM2/LFM3 XML-ish `<function=...><parameter=...>` blocks |
+| `hermes` | JSON payloads `{"name": ..., "arguments": ...}` (Qwen/Hermes style) |
+| `none` | no tool-call parsing (text-only) |
+
+`--enable-thinking` / `--no-enable-thinking` controls the `enable_thinking` chat-template flag for thinking-capable models.
+
+`--full-attention-only` converts `sliding_attention` layers to `full_attention` in the downloaded checkpoint before export (removes `sliding_window`, keeps the `full_attention` RoPE group). Intel NPU (NPUW) fails to compile hybrid sliding/full attention models with `NPUW: conflicting attention mask detection for the same SDPA node`; this flag unifies the attention pattern. It changes the attention semantics of the sliding layers (RoPE theta reverts to the `full_attention` group), so use it only when the NPU compile fails without it. The flag is persisted in `model_configuration.json`, recorded in `model_manifest.json`, and requires `--force-export` to re-export an existing IR.
+
+## 5. HTTP API
+
+The server binds to `127.0.0.1:8000` by default and exposes:
+
+- `GET /health`
+- `GET /v1/models`
+- `POST /v1/chat/completions`
+
+Example health request:
+
+```powershell
+curl.exe http://127.0.0.1:8000/health
+```
+
+The API applies the Hugging Face tokenizer's chat template and converts the model's tool-call markup into OpenAI-compatible `tool_calls`. Streaming responses are emitted token-by-token through an OpenVINO GenAI streamer; fragments that might belong to a tool-call block are withheld by a holdback buffer and flushed after generation, so tool calls parse reliably.
+
+## 6. OpenCode
+
+Merge the contents of `opencode.jsonc` into your OpenCode configuration. The configured model is `local/lfm3-npu` and the base URL is `http://127.0.0.1:8000/v1`.
+
+## Technical notes
+
+- OpenVINO's current NPU guidance recommends 4-bit symmetric compression and group-wise quantization (group size 128 for smaller models). The exporter uses these settings.
+- NPU pipeline configuration uses `CACHE_DIR` always; `MAX_PROMPT_LEN`/`MIN_RESPONSE_LEN` default to 8192/1024 on NPU (`auto`) and are compiled with a fallback ladder; explicit values disable the ladder.
+- Works with any checkpoint Optimum Intel can export; match `--tool-parser` to the model's tool-call markup. Validate export, tokenizer behavior, tool calls, and long-context behavior on the target device.
+- `trust_remote_code=True` permits repository-provided Python code to run locally. Use it only with a repository whose code you trust.
+- `uv.lock` is machine-generated; create it after reviewing the project dependencies with `uv lock`.

intel_npu_server-0.3.0/README.md

@@ -0,0 +1,170 @@
+# LLM on Intel NPU / OpenVINO GenAI / OpenAI-compatible API
+
+Modular local inference server for OpenCode. The project uses Python 3.12 and `uv`; no PowerShell or batch scripts are required. LFM3 is the default checkpoint, but any exportable OpenVINO model works (see section 4.1).
+
+## Project layout
+
+```text
+.
+|-- pyproject.toml
+|-- uv.lock                    # generated by uv lock / uv sync
+|-- model_configuration.json   # generated automatically
+|-- opencode.jsonc
+`-- npu_server/
+    |-- __main__.py            # thin entry point
+    |-- tool_call_parsers.py
+    |-- cli/
+    |   |-- arguments_cli.py     # ModelArgumentsParser
+    |   |-- model_config_cli.py  # ModelConfig
+    |   `-- server_cli.py        # CommandLineInterface
+    |-- generation/
+    |   |-- events.py            # StreamEvent, GenerationRequest, GenerationOutcome
+    |   |-- streaming.py         # ToolSafeStreamBuffer, TokenStreamCallback
+    |   `-- engine.py            # GenerationEngine
+    |-- load_model/
+    |   |-- runtime.py           # GenerationError, ModelRuntime, RuntimeLengths
+    |   |-- preparer.py          # OpenvinoModelPreparer (download + IR export)
+    |   `-- loader.py            # OpenvinoModelLoader (pipeline compilation)
+    `-- server/
+        |-- openai_schemas.py    # CompletionResult, ChatCompletionChunkBuilder
+        |-- message_normalizer.py# ChatMessageNormalizer
+        |-- http_app.py          # NpuServer (routes + uvicorn)
+```
+
+`model_dir` defaults to `Path.cwd().parent / "model"`. The resolved path is written into `model_configuration.json` even when `--model-dir` is omitted. This ensures that a later `serve` command opens the same exported model directory.
+
+Each model lives in its own subfolder named after `model_name`, so several models can share one `model_dir`:
+
+```
+model/
+`-- lfm3-uncensored/
+    |-- checkpoint/        # source Hugging Face checkpoint (downloaded once)
+    `-- exported/          # OpenVINO IR, tokenizer files, model_manifest.json
+```
+
+The global Hugging Face cache is never used. A legacy flat layout (`model_dir\<files>` + `model_dir\checkpoint`) is migrated automatically on the first start of the new version.
+
+## 1. Install and synchronize
+
+Install `uv` once using the official installer for your operating system. Then, from the project root:
+
+```powershell
+uv python install 3.12
+uv sync
+```
+
+The `optimum-intel` dependency is installed from the author's `lfm3` development branch, and `transformers==5.20.0` is used as required by the LFM3 support PR. The package metadata on that branch may still declare an older upper bound, so `tool.uv.override-dependencies` explicitly overrides it. This resolves the dependency solver conflict; it does not itself guarantee runtime compatibility.
+
+## 2. Run the server
+
+First launch:
+
+```powershell
+uv run python -m npu_server run
+```
+
+The `run` command:
+
+1. Reads the configuration file and applies command-line overrides.
+2. Migrates a legacy flat model layout if one is found.
+3. Downloads the Hugging Face checkpoint into `model_dir\<model_name>\checkpoint` (unless `--model-id` points to a local directory).
+4. Loads the tokenizer from that checkpoint.
+5. Invokes the Optimum Intel Python CLI entry point in-process to export OpenVINO IR using INT4 symmetric weight compression and group size 128.
+6. Saves the tokenizer into `model_dir\<model_name>\exported` next to the IR files.
+7. Compiles/loads `openvino_genai.LLMPipeline` on the selected device. With the default `auto` lengths on NPU it tries `MAX_PROMPT_LEN=8192, MIN_RESPONSE_LEN=1024` first and falls back down the ladder `4096/1024 -> 2048/512 -> 1024/256` when the device refuses to compile; explicit `--max-prompt-len`/`--min-response-len` values disable the ladder.
+8. Starts the HTTP server and keeps the model in memory until the process is stopped with Ctrl+C. On shutdown (Ctrl+C or uvicorn graceful exit) the pipeline reference is dropped and garbage-collected, releasing the model and its memory from the device before the process exits.
+
+Subsequent `run` calls reuse an existing `.xml` + `.bin` IR export and the saved tokenizer.
+
+## 3. Individual commands
+
+Export only:
+
+```powershell
+uv run python -m npu_server prepare
+```
+
+Run the HTTP server using an existing IR export:
+
+```powershell
+uv run python -m npu_server serve
+```
+
+Check the local OpenVINO installation and device visibility:
+
+```powershell
+uv run python -m npu_server doctor
+```
+
+Force a fresh export:
+
+```powershell
+uv run python -m npu_server run --force-export
+```
+
+## 4. Configuration overrides
+
+All values can be stored in `model_configuration.json` or passed as command-line arguments. Command-line arguments take precedence over the JSON file. If a value is omitted, the value from JSON is used; otherwise the `ModelConfig` default is used.
+
+`model_id` and `device` are required: they must be present in `model_configuration.json` or passed as command-line arguments, otherwise startup fails with an error. `model_name` defaults to `model_id` when omitted.
+
+```powershell
+uv run python -m npu_server run `
+  --model-id OpenIntelligenceNet/LFM3-1.7B-Uncensored `
+  --model-dir ..\model `
+  --device NPU `
+  --host 127.0.0.1 `
+  --port 8000 `
+  --max-prompt-len 1024 `
+  --min-response-len 128 `
+  --max-output-tokens 128
+```
+
+`--model-dir` is optional. If omitted, it defaults to the resolved path `Path.cwd().parent / "model"`, and that value is persisted in `model_configuration.json`.
+
+Length parameters accept either a positive integer or `auto`. On NPU, `auto` resolves to safe defaults (`MAX_PROMPT_LEN=8192`, `MIN_RESPONSE_LEN=1024`) with an automatic fallback ladder when the device refuses to compile. On CPU/GPU, `auto` omits the corresponding OpenVINO property so shapes stay dynamic; `max_output_tokens=auto` leaves the generation-length limit to the library default. For tight NPU memory budgets set the lengths explicitly.
+
+## 4.1. Other models
+
+The server works with any Hugging Face checkpoint that Optimum Intel can export to OpenVINO IR; LFM3 is only the default. Pass `--model-id <repo>`, set `--model-name`, and match the tool-call format if the model supports tools:
+
+`--tool-parser` selects the converter for the model's tool-call markup:
+
+| value | markup |
+| --- | --- |
+| `auto` | guess from the model id/name: `lfm*` -> lfm3, `qwen`/`hermes`/`glm` -> hermes, otherwise none |
+| `lfm3` | LFM2/LFM3 XML-ish `<function=...><parameter=...>` blocks |
+| `hermes` | JSON payloads `{"name": ..., "arguments": ...}` (Qwen/Hermes style) |
+| `none` | no tool-call parsing (text-only) |
+
+`--enable-thinking` / `--no-enable-thinking` controls the `enable_thinking` chat-template flag for thinking-capable models.
+
+`--full-attention-only` converts `sliding_attention` layers to `full_attention` in the downloaded checkpoint before export (removes `sliding_window`, keeps the `full_attention` RoPE group). Intel NPU (NPUW) fails to compile hybrid sliding/full attention models with `NPUW: conflicting attention mask detection for the same SDPA node`; this flag unifies the attention pattern. It changes the attention semantics of the sliding layers (RoPE theta reverts to the `full_attention` group), so use it only when the NPU compile fails without it. The flag is persisted in `model_configuration.json`, recorded in `model_manifest.json`, and requires `--force-export` to re-export an existing IR.
+
+## 5. HTTP API
+
+The server binds to `127.0.0.1:8000` by default and exposes:
+
+- `GET /health`
+- `GET /v1/models`
+- `POST /v1/chat/completions`
+
+Example health request:
+
+```powershell
+curl.exe http://127.0.0.1:8000/health
+```
+
+The API applies the Hugging Face tokenizer's chat template and converts the model's tool-call markup into OpenAI-compatible `tool_calls`. Streaming responses are emitted token-by-token through an OpenVINO GenAI streamer; fragments that might belong to a tool-call block are withheld by a holdback buffer and flushed after generation, so tool calls parse reliably.
+
+## 6. OpenCode
+
+Merge the contents of `opencode.jsonc` into your OpenCode configuration. The configured model is `local/lfm3-npu` and the base URL is `http://127.0.0.1:8000/v1`.
+
+## Technical notes
+
+- OpenVINO's current NPU guidance recommends 4-bit symmetric compression and group-wise quantization (group size 128 for smaller models). The exporter uses these settings.
+- NPU pipeline configuration uses `CACHE_DIR` always; `MAX_PROMPT_LEN`/`MIN_RESPONSE_LEN` default to 8192/1024 on NPU (`auto`) and are compiled with a fallback ladder; explicit values disable the ladder.
+- Works with any checkpoint Optimum Intel can export; match `--tool-parser` to the model's tool-call markup. Validate export, tokenizer behavior, tool calls, and long-context behavior on the target device.
+- `trust_remote_code=True` permits repository-provided Python code to run locally. Use it only with a repository whose code you trust.
+- `uv.lock` is machine-generated; create it after reviewing the project dependencies with `uv lock`.

intel_npu_server-0.3.0/npu_server/__main__.py

@@ -0,0 +1,13 @@
+from __future__ import annotations
+
+import sys
+
+from npu_server.cli.server_cli import CommandLineInterface
+
+
+def main() -> int:
+    return CommandLineInterface().startup(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

intel_npu_server-0.3.0/npu_server/cli/__init__.py

@@ -0,0 +1,2 @@
+from npu_server.cli.arguments_cli import ModelArgumentsParser
+from npu_server.cli.model_config_cli import CONFIG_FILENAME, MANIFEST_FILENAME, ModelConfig

intel_npu_server-0.3.0/npu_server/cli/arguments_cli.py

@@ -0,0 +1,110 @@
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+from typing import Any, Literal, Sequence, cast
+
+from npu_server.cli.model_config_cli import CONFIG_FILENAME, ModelConfig
+
+
+class ModelArgumentsParser:
+    """Read model_configuration.json and apply command-line overrides onto ModelConfig."""
+
+    @staticmethod
+    def _load_config() -> tuple[dict[str, Any] | None, Path]:
+        """Load config from cwd, then its parent; otherwise return the cwd target path."""
+        cwd = Path.cwd().resolve()
+        candidates = (cwd / CONFIG_FILENAME, cwd.parent / CONFIG_FILENAME)
+
+        for config_path in candidates:
+            if config_path.is_file():
+                try:
+                    raw_config: Any = json.loads(config_path.read_text(encoding="utf-8-sig"))
+                except json.JSONDecodeError as exc:
+                    raise ValueError(f"Invalid JSON in configuration file {config_path}: {exc}") from exc
+
+                if not isinstance(raw_config, dict):
+                    raise ValueError(f"Configuration root must be a JSON object: {config_path}")
+                return cast(dict[str, Any], raw_config), config_path
+
+        return None, candidates[0]
+
+    @staticmethod
+    def _parse_int_or_auto(value: str) -> int | Literal["auto"]:
+        if value.strip().lower() == "auto":
+            return "auto"
+        try:
+            return int(value)
+        except ValueError as exc:
+            raise argparse.ArgumentTypeError("Expected a positive integer or 'auto'") from exc
+
+    @staticmethod
+    def get_model_params(argv: Sequence[str] | None = None) -> ModelConfig:
+        config_data, config_path = ModelArgumentsParser._load_config()
+        usable_config = {key: value for key, value in (config_data or {}).items() if value is not None}
+        defaults = ModelConfig.model_validate(usable_config)
+
+        parser = argparse.ArgumentParser(description="Configuration options for the local OpenVINO model server")
+
+        parser.add_argument(
+            "--model-id",
+            "--model",
+            "-m",
+            dest="model_id",
+            type=str,
+            default=defaults.model_id,
+            help="Hugging Face repository ID or local model directory",
+        )
+        parser.add_argument("--model-dir", type=Path, default=defaults.model_dir, help="Path to save/load model")
+        parser.add_argument(
+            "--model-name",
+            dest="model_name",
+            type=str,
+            default=defaults.model_name,
+            help="Model name exposed by the /v1/models endpoint",
+        )
+        parser.add_argument("--device", default=defaults.device)
+        parser.add_argument(
+            "--tool-parser",
+            dest="tool_parser",
+            default=defaults.tool_parser,
+            help="Tool call markup converter: auto, lfm3, hermes, none",
+        )
+        parser.add_argument(
+            "--enable-thinking",
+            dest="enable_thinking",
+            action=argparse.BooleanOptionalAction,
+            default=defaults.enable_thinking,
+            help="Pass enable_thinking to the chat template",
+        )
+        parser.add_argument(
+            "--full-attention-only",
+            dest="full_attention_only",
+            action=argparse.BooleanOptionalAction,
+            default=defaults.full_attention_only,
+            help="Unify all attention layers to full_attention before export (NPU/NPUW fix)",
+        )
+        parser.add_argument("--host", default=defaults.host)
+        parser.add_argument("--port", type=int, default=defaults.port)
+        parser.add_argument(
+            "--max-prompt-len",
+            type=ModelArgumentsParser._parse_int_or_auto,
+            default=defaults.max_prompt_len,
+        )
+        parser.add_argument(
+            "--min-response-len",
+            type=ModelArgumentsParser._parse_int_or_auto,
+            default=defaults.min_response_len,
+        )
+        parser.add_argument(
+            "--max-output-tokens",
+            type=ModelArgumentsParser._parse_int_or_auto,
+            default=defaults.max_output_tokens,
+        )
+
+        args = parser.parse_args(argv)
+        validated_config = ModelConfig.model_validate(vars(args))
+        validated_config.save_config(config_path)
+
+        return validated_config

intel_npu_server-0.3.0/npu_server/cli/model_config_cli.py

@@ -0,0 +1,111 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Literal
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+from npu_server.tool_call_parsers import TOOL_PARSER_NAMES
+
+CONFIG_FILENAME = "model_configuration.json"
+MANIFEST_FILENAME = "model_manifest.json"
+
+
+class ModelConfig(BaseModel):
+    """Runtime configuration persisted in model_configuration.json."""
+
+    model_config = ConfigDict(extra="ignore")
+
+    model_id: str = Field(description="Hugging Face repository ID or local model directory (required)")
+    model_dir: Path = Field(
+        default=(Path.cwd().parent / "model").resolve(),
+        description="Directory containing the exported OpenVINO IR and tokenizer",
+    )
+    model_name: str | None = Field(
+        default=None,
+        description="Model ID exposed by the /v1/models endpoint; defaults to model_id",
+    )
+    device: str = Field(
+        description="OpenVINO device (NPU, CPU, AUTO, ...) (required)",
+    )
+    host: str = Field(default="127.0.0.1", description="HTTP server bind address")
+    port: int = Field(default=8000, ge=1, le=65535)
+
+    tool_parser: str = Field(
+        default="auto",
+        description="Tool call markup format: auto, lfm3, hermes, none",
+    )
+    enable_thinking: bool = Field(
+        default=False,
+        description="Pass enable_thinking to the chat template (thinking-capable models)",
+    )
+    full_attention_only: bool = Field(
+        default=False,
+        description=(
+            "Convert sliding_attention layers to full_attention before export; "
+            "required for hybrid sliding/full LFM checkpoints on Intel NPU (NPUW "
+            "conflicting attention mask assertion)"
+        ),
+    )
+
+    max_prompt_len: int | Literal["auto"] = Field(
+        default="auto",
+        description="Maximum prompt length for the NPU pipeline, or 'auto'",
+    )
+    min_response_len: int | Literal["auto"] = Field(
+        default="auto",
+        description="Minimum response length reserved by the NPU pipeline, or 'auto'",
+    )
+    max_output_tokens: int | Literal["auto"] = Field(
+        default="auto",
+        description="Maximum generated tokens per request, or 'auto'",
+    )
+
+    @field_validator("model_id", "device", "host")
+    @classmethod
+    def validate_non_empty_strings(cls, value: str) -> str:
+        value = value.strip()
+        if not value:
+            raise ValueError("Value must not be empty")
+        return value
+
+    @field_validator("model_dir")
+    @classmethod
+    def normalize_model_dir(cls, value: Path) -> Path:
+        value = Path(value).expanduser()
+        value.mkdir(parents=True, exist_ok=True)
+        return value.resolve()
+
+    @field_validator("tool_parser")
+    @classmethod
+    def validate_tool_parser(cls, value: str) -> str:
+        value = value.strip().lower()
+        if value not in TOOL_PARSER_NAMES:
+            raise ValueError(f"tool_parser must be one of: {sorted(TOOL_PARSER_NAMES)}")
+        return value
+
+    @field_validator("max_prompt_len", "min_response_len", "max_output_tokens")
+    @classmethod
+    def validate_positive_lengths(cls, value: int | Literal["auto"]) -> int | Literal["auto"]:
+        if (isinstance(value, int) and value <= 0) or (isinstance(value, str) and value != "auto"):
+            raise ValueError("Length values must be positive integers or 'auto'")
+        return value
+
+    def save_config(self, path: str | Path) -> None:
+        config_path = Path(path).expanduser().resolve()
+        if config_path.suffix == ".json" and config_path.name == CONFIG_FILENAME:
+            config_file = config_path
+        else:
+            config_file = config_path / CONFIG_FILENAME
+        config_file.parent.mkdir(parents=True, exist_ok=True)
+        config_file.write_text(
+            json.dumps(self.model_dump(mode="json"), ensure_ascii=False, indent=1) + "\n",
+            encoding="utf-8",
+        )
+
+    @model_validator(mode="after")
+    def derive_model_name(self) -> ModelConfig:
+        if self.model_name is None:
+            self.model_name = self.model_id
+        return self