inferkit 0.1.0__py3-none-any.whl

inferkit-0.1.0.dist-info/METADATA

@@ -0,0 +1,191 @@
+Metadata-Version: 2.4
+Name: inferkit
+Version: 0.1.0
+Summary: Embeddable inference engine for small language models — the SQLite for AI
+Author-email: Milad Olad <milad.olad@gmail.com>
+License-Expression: MIT
+Keywords: ai,edge,embedding,inference,llm,local,offline,slm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Requires-Dist: huggingface-hub>=0.25
+Requires-Dist: llama-cpp-python>=0.3
+Requires-Dist: tokenizers>=0.21
+Requires-Dist: typer>=0.15
+Provides-Extra: mlx
+Requires-Dist: mlx-lm>=0.31; extra == 'mlx'
+Description-Content-Type: text/markdown
+
+# slmkit
+
+Package small language models into self-contained, embeddable `.slm` artifacts.
+
+**SQLite for AI** — one file, zero config, zero internet, 5ms inference.
+
+```python
+import slmkit
+
+classifier = slmkit.load("sentiment.slm")
+result = classifier("The aurora was breathtaking")
+# → {"label": "positive", "confidence": 1.0}
+```
+
+## Install
+
+```bash
+pip install slmkit
+```
+
+Requires Python 3.11+. The `llama-cpp-python` dependency compiles C++ on first install, which may take a few minutes.
+
+## Quickstart
+
+### 1. Compile a model
+
+Package a pre-quantized GGUF model from HuggingFace into a `.slm` artifact:
+
+```bash
+slm compile \
+  --source Qwen/Qwen2.5-0.5B-Instruct-GGUF \
+  --task classification \
+  --labels "positive,negative,neutral" \
+  --quantize q4_k_m \
+  --output sentiment.slm
+```
+
+This downloads the model (~490 MB), bakes in a task-specific prompt, and packages everything into a single file.
+
+You can also pass a local `.gguf` file as `--source` instead of a HuggingFace repo.
+
+### 2. Load and run
+
+```python
+import slmkit
+
+with slmkit.load("sentiment.slm") as model:
+    result = model("The midnight sun over Tromsø was unforgettable")
+    print(result)
+    # {"label": "positive", "confidence": 1.0}
+
+    results = model.batch([
+        "The recipe was a disaster",
+        "I finally fixed the timing belt myself",
+    ])
+    # [{"label": "negative", ...}, {"label": "positive", ...}]
+```
+
+### 3. Test and inspect
+
+```bash
+# Run inference from the CLI
+slm test sentiment.slm --input "The coral reef was thriving"
+
+# View artifact metadata
+slm info sentiment.slm
+```
+
+## Validation
+
+Bake test cases into the artifact at compile time:
+
+```bash
+slm compile \
+  --source Qwen/Qwen2.5-0.5B-Instruct-GGUF \
+  --task classification \
+  --labels "positive,negative,neutral" \
+  --quantize q4_k_m \
+  --validation tests.json \
+  --output sentiment.slm
+```
+
+Where `tests.json` contains:
+
+```json
+[
+  {"input": "The view from the summit was breathtaking", "expected_label": "positive"},
+  {"input": "The bridge collapsed during rush hour", "expected_label": "negative"}
+]
+```
+
+Then run the validation suite:
+
+```bash
+slm test sentiment.slm
+```
+
+## The `.slm` format
+
+A `.slm` file is a ZIP archive with a fixed structure:
+
+```
+sentiment.slm (ZIP)
+├── model.gguf            # Quantized model weights
+├── manifest.json         # Task type, backend, labels, source info
+├── prompt_template.txt   # Baked-in system prompt
+├── tokenizer.json        # Tokenizer config (optional)
+└── validation.json       # Test cases (optional)
+```
+
+One file. Everything included. Ship it, load it, call it.
+
+## Current scope
+
+This is an early release focused on proving the core workflow:
+
+- **Backends**: llama-cpp-python (default), MLX (Apple Silicon, `pip install slmkit[mlx]`)
+- **Task**: classification
+- **Models**: pre-quantized GGUF or MLX-format models from HuggingFace
+
+### MLX on Apple Silicon
+
+For Metal-accelerated inference on M-series Macs:
+
+```bash
+pip install slmkit[mlx]
+
+slm compile \
+  --source mlx-community/Qwen2.5-0.5B-Instruct-4bit \
+  --task classification \
+  --labels "positive,negative,neutral" \
+  --backend mlx \
+  --output sentiment-mlx.slm
+```
+
+The backend is auto-detected from the model source, or set explicitly with `--backend`.
+
+## Examples
+
+Runnable examples in [`examples/`](examples/):
+
+### Classification
+
+| Example | Model | Size | Description |
+|---------|-------|------|-------------|
+| [sentiment.py](examples/classification/sentiment.py) | Qwen2.5-0.5B | ~490 MB | Restaurant review sentiment |
+| [moderation.py](examples/classification/moderation.py) | Qwen2.5-0.5B | ~490 MB | Content moderation with validation |
+| [spam_detection.py](examples/classification/spam_detection.py) | SmolLM2-360M | ~300 MB | Binary spam vs ham |
+| [urgency_triage.py](examples/classification/urgency_triage.py) | Llama-3.2-1B | ~700 MB | Priority routing |
+| [language_detection.py](examples/classification/language_detection.py) | Qwen2.5-1.5B | ~1 GB | Six-language detection |
+| [topic_categorization.py](examples/classification/topic_categorization.py) | Gemma-2-2B | ~1.5 GB | Article topic sorting |
+| [tone_detection.py](examples/classification/tone_detection.py) | Phi-3.5-Mini | ~2.2 GB | Tone/intent classification |
+
+### MLX (Apple Silicon)
+
+| Example | Model | Size | Description |
+|---------|-------|------|-------------|
+| [sentiment.py](examples/mlx/sentiment.py) | Qwen2.5-0.5B-4bit | ~243 MB | Metal-accelerated sentiment |
+
+```bash
+uv run python examples/classification/sentiment.py
+```
+
+## License
+
+MIT

inferkit-0.1.0.dist-info/RECORD

@@ -0,0 +1,21 @@
+slmkit/__init__.py,sha256=hPIeouWwN_JjFKvXuvyeA1KosmWJ5vk-A3xJYouZAU0,293
+slmkit/cli.py,sha256=DktdvPeFmIk5BSYYyRo6kufaN_nDPbbV9W-s8a2jC-k,3547
+slmkit/compiler.py,sha256=tlvywoNOa5iFucq2wAArkWhNDAyifpPnXC_Tz03Rk5s,5379
+slmkit/model.py,sha256=OkqwWwnuXLbOHC8SVVN3DbEJFEPtzgA-jBssStA0zXA,2882
+slmkit/backends/__init__.py,sha256=JuHaE88uKrWlDYR8pba5FAQ_PnwxwBlx-aQYmIOVyww,797
+slmkit/backends/llamacpp.py,sha256=s5iJxSm3exCRlKMK2AN7beLY6FndP4yQ6fIPibp6Rk4,1619
+slmkit/backends/mlx.py,sha256=04M2zOthQTkD4vNlcC6zf0cCcc-NqK0RGQ2HR1-J4ok,1813
+slmkit/backends/onnx.py,sha256=A6G6gZkoxozvGMI619mlgHD0puyan66ITCu9CgdV3IA,47
+slmkit/formats/__init__.py,sha256=G-z8Pyey-ooaRD5dt0Z_hgjraijLDHxUIWlMnXN8-XA,47
+slmkit/formats/artifact.py,sha256=H8hhruMTAXAl2aoqhZRZF-j73HsuJiandn3RTEI0jqQ,4745
+slmkit/formats/manifest.py,sha256=yYNO3URS0DCgN4PBEOGRI_pP1YGNjZdklxFGpYvJzEs,2272
+slmkit/formats/schema.py,sha256=pGLK3-9zxsTF4ajr4FX_NJ7mvnfgaJ-jGEpv5zKj-UQ,1067
+slmkit/tasks/__init__.py,sha256=58vTSICyGHa7cZ380CZQI7nx8cDlHv05egnmBg2OSdA,981
+slmkit/tasks/classification.py,sha256=W7mFYt7E8YQMWgZEjkbE_dM-KmFgS67wt8OkIgvy5As,1330
+slmkit/tasks/embedding.py,sha256=5mFqr_bTTCMFSPRByUR64XXN_Vg4WHVUpeqyursjRbI,38
+slmkit/tasks/extraction.py,sha256=EmraQZEPE6NXp8gy3wxz1z-J7gCN1mYgWKEzEd7CWR0,43
+slmkit/tasks/generation.py,sha256=1WzRitNl_be-86PErP6AMeUBd3swZkOD0XlK8LWLr0c,47
+inferkit-0.1.0.dist-info/METADATA,sha256=Dc0VadH38f3Clh889ceGyDMtxKYPzQ2ZCOGwmoNtb78,5566
+inferkit-0.1.0.dist-info/WHEEL,sha256=QccIxa26bgl1E6uMy58deGWi-0aeIkkangHcxk2kWfw,87
+inferkit-0.1.0.dist-info/entry_points.txt,sha256=oE0uOL2wSFKert5p7dq3UuhSVTrUdQDzwwh9FzDu73k,39
+inferkit-0.1.0.dist-info/RECORD,,

inferkit-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.29.0
+Root-Is-Purelib: true
+Tag: py3-none-any

inferkit-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+slm = slmkit.cli:app

slmkit/__init__.py

@@ -0,0 +1,11 @@
+"""slmkit — Package small language models into self-contained, embeddable artifacts."""
+
+from slmkit.model import SlmModel
+
+
+def load(path: str) -> SlmModel:
+    """Load a .slm artifact and return a callable model."""
+    return SlmModel.from_artifact(path)
+
+
+__all__ = ["SlmModel", "load"]

slmkit/backends/__init__.py

@@ -0,0 +1,26 @@
+"""Backend auto-detection and dispatch."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from slmkit.formats.manifest import BackendType
+
+if TYPE_CHECKING:
+    from slmkit.backends.llamacpp import LlamaCppBackend
+    from slmkit.backends.mlx import MlxBackend
+
+
+def load_backend(backend: BackendType, model_path: Path) -> LlamaCppBackend | MlxBackend:
+    """Instantiate the requested inference backend."""
+    if backend == BackendType.LLAMACPP:
+        from slmkit.backends.llamacpp import LlamaCppBackend
+
+        return LlamaCppBackend(model_path)
+    if backend == BackendType.MLX:
+        from slmkit.backends.mlx import MlxBackend
+
+        return MlxBackend(model_path)
+    msg = f"Unsupported backend: {backend}"
+    raise ValueError(msg)

slmkit/backends/llamacpp.py

@@ -0,0 +1,59 @@
+"""llama-cpp-python inference backend."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any
+
+
+class LlamaCppBackend:
+    """Inference backend using llama-cpp-python."""
+
+    def __init__(
+        self,
+        model_path: Path,
+        *,
+        n_ctx: int = 512,
+        n_gpu_layers: int = -1,
+    ) -> None:
+        try:
+            from llama_cpp import Llama
+        except ImportError:
+            msg = (
+                "llama-cpp-python is not installed or failed to load. "
+                "Install it with: pip install llama-cpp-python"
+            )
+            raise ImportError(msg) from None
+
+        if not model_path.exists():
+            msg = f"Model file not found: {model_path}"
+            raise FileNotFoundError(msg)
+
+        self._model = Llama(
+            model_path=str(model_path),
+            n_ctx=n_ctx,
+            n_gpu_layers=n_gpu_layers,
+            verbose=False,
+        )
+
+    def chat(
+        self,
+        system_prompt: str,
+        user_message: str,
+        *,
+        max_tokens: int = 32,
+    ) -> str:
+        """Run chat completion and return the assistant response text."""
+        response: dict[str, Any] = self._model.create_chat_completion(
+            messages=[
+                {"role": "system", "content": system_prompt},
+                {"role": "user", "content": user_message},
+            ],
+            max_tokens=max_tokens,
+            temperature=0.0,
+        )
+        return response["choices"][0]["message"]["content"]
+
+    def close(self) -> None:
+        if hasattr(self, "_model"):
+            del self._model

slmkit/backends/mlx.py

@@ -0,0 +1,63 @@
+"""MLX inference backend for Apple Silicon."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+
+class MlxBackend:
+    """Inference backend using mlx-lm on Apple Silicon."""
+
+    def __init__(self, model_path: Path) -> None:
+        try:
+            from mlx_lm import load
+        except ImportError:
+            msg = (
+                "mlx-lm is not installed. "
+                "Install it with: pip install mlx-lm  (requires Apple Silicon Mac)"
+            )
+            raise ImportError(msg) from None
+
+        if not model_path.exists():
+            msg = f"Model path not found: {model_path}"
+            raise FileNotFoundError(msg)
+
+        self._model, self._tokenizer = load(str(model_path))
+
+    def chat(
+        self,
+        system_prompt: str,
+        user_message: str,
+        *,
+        max_tokens: int = 32,
+    ) -> str:
+        """Run chat completion and return the assistant response text."""
+        import mlx.core as mx
+        from mlx_lm import generate
+
+        messages = [
+            {"role": "system", "content": system_prompt},
+            {"role": "user", "content": user_message},
+        ]
+        prompt = self._tokenizer.apply_chat_template(
+            messages, add_generation_prompt=True, tokenize=False
+        )
+
+        # Greedy (temperature=0) sampling via argmax
+        def _greedy_sampler(logits: mx.array) -> mx.array:
+            return mx.argmax(logits, axis=-1)
+
+        return generate(
+            self._model,
+            self._tokenizer,
+            prompt=prompt,
+            max_tokens=max_tokens,
+            sampler=_greedy_sampler,
+            verbose=False,
+        )
+
+    def close(self) -> None:
+        if hasattr(self, "_model"):
+            del self._model
+        if hasattr(self, "_tokenizer"):
+            del self._tokenizer

slmkit/backends/onnx.py

@@ -0,0 +1 @@
+"""ONNX Runtime inference backend (future)."""

slmkit/cli.py

@@ -0,0 +1,104 @@
+"""CLI entrypoint for the `slm` command."""
+
+from __future__ import annotations
+
+import json
+from typing import Annotated
+
+import typer
+
+app = typer.Typer(help="slmkit — package and run small language models")
+
+
+@app.command()
+def compile(
+    source: Annotated[str, typer.Option(help="HuggingFace repo ID or local path")],
+    task: Annotated[str, typer.Option(help="Task type (e.g., classification)")],
+    output: Annotated[str, typer.Option(help="Output .slm file path")],
+    labels: Annotated[
+        str | None, typer.Option(help="Comma-separated labels for classification")
+    ] = None,
+    quantize: Annotated[str, typer.Option(help="Quantization level (e.g., q4_k_m)")] = "q4_k_m",
+    validation: Annotated[
+        str | None,
+        typer.Option(help="Path to validation JSON file (list of {input, expected_label})"),
+    ] = None,
+    backend: Annotated[
+        str | None,
+        typer.Option(help="Backend: llamacpp or mlx (auto-detected if omitted)"),
+    ] = None,
+) -> None:
+    """Compile a HuggingFace model into a .slm artifact."""
+    from pathlib import Path
+
+    from slmkit.compiler import compile_model
+
+    label_list = [label.strip() for label in labels.split(",")] if labels else None
+
+    validation_data = None
+    if validation:
+        validation_data = json.loads(Path(validation).read_text())
+
+    result = compile_model(
+        source,
+        task=task,
+        output=output,
+        labels=label_list,
+        quantization=quantize,
+        validation=validation_data,
+        backend=backend,
+    )
+    size_mb = result.stat().st_size / 1024 / 1024
+    typer.echo(f"Compiled: {result} ({size_mb:.1f} MB)")
+
+
+@app.command()
+def test(
+    artifact: Annotated[str, typer.Argument(help="Path to .slm artifact")],
+    input_text: Annotated[str | None, typer.Option("--input", help="Test input text")] = None,
+) -> None:
+    """Run a quick sanity check on a .slm artifact."""
+    import slmkit
+
+    with slmkit.load(artifact) as model:
+        model_info = model.info()
+        typer.echo(f"Task: {model_info['task']}")
+        typer.echo(f"Backend: {model_info['backend']}")
+        typer.echo(f"Source: {model_info['source']}")
+
+        if input_text:
+            result = model(input_text)
+            typer.echo(f"Result: {json.dumps(result)}")
+
+        # Run baked-in validation cases
+        validation = model.validation_cases()
+        if validation:
+            typer.echo(f"\nRunning {len(validation)} validation case(s)...")
+            passed = 0
+            for i, case in enumerate(validation, 1):
+                result = model(case["input"])
+                expected = case.get("expected_label")
+                actual = result["label"]
+                ok = actual == expected if expected else True
+                status = "PASS" if ok else "FAIL"
+                typer.echo(f'  [{status}] {i}. "{case["input"][:50]}" → {actual}')
+                if ok:
+                    passed += 1
+            typer.echo(f"\n{passed}/{len(validation)} passed")
+            if passed < len(validation):
+                raise typer.Exit(1)
+        elif not input_text:
+            typer.echo("Model loaded successfully. Use --input to run inference.")
+
+
+@app.command()
+def info(
+    artifact: Annotated[str, typer.Argument(help="Path to .slm artifact")],
+) -> None:
+    """Show metadata from a .slm artifact."""
+    from pathlib import Path
+
+    from slmkit.formats.artifact import read_manifest
+
+    manifest = read_manifest(Path(artifact))
+    typer.echo(json.dumps(manifest.to_dict(), indent=2))

slmkit/compiler.py

@@ -0,0 +1,172 @@
+"""Compile pipeline: HuggingFace model → .slm artifact."""
+
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+from huggingface_hub import hf_hub_download, list_repo_files, snapshot_download
+
+from slmkit.formats.artifact import write_artifact
+from slmkit.formats.manifest import BackendType, Manifest, ModelFormat, TaskType
+from slmkit.formats.schema import validate_manifest
+from slmkit.tasks import build_system_prompt
+
+logger = logging.getLogger(__name__)
+
+
+def _find_gguf_file(repo_id: str, quantization: str) -> str:
+    """Find a GGUF file in a HuggingFace repo matching the requested quantization."""
+    files = list(list_repo_files(repo_id))
+    gguf_files = [f for f in files if f.endswith(".gguf")]
+
+    if not gguf_files:
+        msg = (
+            f"No GGUF files found in '{repo_id}'. "
+            f"Try a repo with pre-quantized GGUF files (e.g., '{repo_id}-GGUF')."
+        )
+        raise FileNotFoundError(msg)
+
+    q_lower = quantization.lower().replace("-", "_")
+    matching = [f for f in gguf_files if q_lower in f.lower()]
+
+    if not matching:
+        available = ", ".join(gguf_files)
+        msg = (
+            f"No GGUF file matching quantization '{quantization}' in '{repo_id}'. "
+            f"Available: {available}"
+        )
+        raise FileNotFoundError(msg)
+
+    return matching[0]
+
+
+def _resolve_gguf_model(source: str, quantization: str) -> Path:
+    """Resolve the model source to a local GGUF file path."""
+    source_path = Path(source)
+    if source_path.is_file() and source_path.suffix == ".gguf":
+        return source_path
+
+    # Treat as HuggingFace repo ID
+    gguf_filename = _find_gguf_file(source, quantization)
+    logger.info("Downloading %s from %s", gguf_filename, source)
+    local_path = hf_hub_download(repo_id=source, filename=gguf_filename)
+    return Path(local_path)
+
+
+def _resolve_mlx_model(source: str) -> Path:
+    """Resolve the model source to a local MLX model directory."""
+    source_path = Path(source)
+    if source_path.is_dir():
+        return source_path
+
+    # Download entire MLX repo snapshot
+    logger.info("Downloading MLX model snapshot from %s", source)
+    local_dir = snapshot_download(repo_id=source)
+    return Path(local_dir)
+
+
+def _resolve_tokenizer(source: str) -> Path | None:
+    """Try to download tokenizer.json from the source repo."""
+    source_path = Path(source)
+    if source_path.is_file():
+        tokenizer = source_path.parent / "tokenizer.json"
+        return tokenizer if tokenizer.exists() else None
+
+    try:
+        local_path = hf_hub_download(repo_id=source, filename="tokenizer.json")
+        return Path(local_path)
+    except Exception:
+        logger.debug("No tokenizer.json found in %s", source)
+        return None
+
+
+def _detect_backend(source: str, backend: str | None) -> BackendType:
+    """Detect or validate the backend to use."""
+    if backend:
+        return BackendType(backend)
+
+    # Auto-detect from source
+    source_path = Path(source)
+    if source_path.is_file() and source_path.suffix == ".gguf":
+        return BackendType.LLAMACPP
+    if source_path.is_dir():
+        # Check for safetensors files → MLX
+        if any(source_path.glob("*.safetensors")):
+            return BackendType.MLX
+        return BackendType.LLAMACPP
+
+    # HuggingFace repo — check for MLX markers
+    try:
+        files = list(list_repo_files(source))
+        if any(f.endswith(".safetensors") for f in files) and not any(
+            f.endswith(".gguf") for f in files
+        ):
+            return BackendType.MLX
+    except Exception:
+        pass
+
+    return BackendType.LLAMACPP
+
+
+def compile_model(
+    source: str,
+    *,
+    task: str,
+    output: str,
+    labels: list[str] | None = None,
+    quantization: str = "q4_k_m",
+    validation: list[dict[str, object]] | None = None,
+    backend: str | None = None,
+) -> Path:
+    """Compile a model source into a .slm artifact."""
+    task_type = TaskType(task)
+    output_path = Path(output)
+
+    # Build task config
+    task_config: dict[str, object] = {}
+    if task_type == TaskType.CLASSIFICATION:
+        if not labels:
+            msg = "Classification task requires --labels"
+            raise ValueError(msg)
+        task_config["labels"] = labels
+
+    # Detect backend
+    backend_type = _detect_backend(source, backend)
+
+    # Resolve model
+    if backend_type == BackendType.MLX:
+        model_path = _resolve_mlx_model(source)
+        model_format = ModelFormat.SAFETENSORS
+        # MLX repos include their own tokenizer; skip separate fetch
+        tokenizer_path = None
+    else:
+        model_path = _resolve_gguf_model(source, quantization)
+        model_format = ModelFormat.GGUF
+        tokenizer_path = _resolve_tokenizer(source)
+
+    # Build prompt template
+    prompt_template = build_system_prompt(task_type, task_config)
+
+    # Create manifest
+    manifest = Manifest(
+        task=task_type,
+        backend=backend_type,
+        source=source,
+        quantization=quantization,
+        model_format=model_format,
+        task_config=task_config,
+    )
+    validate_manifest(manifest)
+
+    # Package artifact
+    write_artifact(
+        output_path,
+        model_path=model_path,
+        manifest=manifest,
+        prompt_template=prompt_template,
+        tokenizer_path=tokenizer_path,
+        validation=validation,
+    )
+
+    return output_path

slmkit/formats/__init__.py

@@ -0,0 +1 @@
+""".slm artifact format — ZIP read/write."""

slmkit/formats/artifact.py

@@ -0,0 +1,136 @@
+""".slm artifact file I/O."""
+
+from __future__ import annotations
+
+import json
+import tempfile
+import zipfile
+from dataclasses import dataclass
+from pathlib import Path
+
+from slmkit.formats.manifest import Manifest
+
+MANIFEST_NAME = "manifest.json"
+MODEL_NAME = "model.gguf"
+MODEL_DIR = "model/"
+PROMPT_TEMPLATE_NAME = "prompt_template.txt"
+TOKENIZER_NAME = "tokenizer.json"
+VALIDATION_NAME = "validation.json"
+
+
+@dataclass(slots=True)
+class ArtifactContent:
+    """Extracted contents of a .slm artifact."""
+
+    root: Path
+    manifest: Manifest
+    model_path: Path
+    prompt_template: str
+    tokenizer_path: Path | None
+    validation: list[dict[str, object]] | None
+
+
+def _validate_zip_entries(zf: zipfile.ZipFile) -> None:
+    """Guard against zip-slip path traversal."""
+    for info in zf.infolist():
+        if info.filename.startswith("/") or ".." in info.filename:
+            msg = f"Unsafe zip entry: {info.filename}"
+            raise ValueError(msg)
+
+
+def write_artifact(
+    output: Path,
+    *,
+    model_path: Path,
+    manifest: Manifest,
+    prompt_template: str,
+    tokenizer_path: Path | None = None,
+    validation: list[dict[str, object]] | None = None,
+) -> Path:
+    """Package model files into a .slm artifact."""
+    with zipfile.ZipFile(output, "w", zipfile.ZIP_DEFLATED) as zf:
+        zf.writestr(MANIFEST_NAME, manifest.to_json())
+        if model_path.is_dir():
+            # MLX-style model directory: bundle all files under model/
+            for file in sorted(model_path.rglob("*")):
+                if file.is_file():
+                    arcname = MODEL_DIR + str(file.relative_to(model_path))
+                    zf.write(file, arcname)
+        else:
+            zf.write(model_path, MODEL_NAME)
+        zf.writestr(PROMPT_TEMPLATE_NAME, prompt_template)
+        if tokenizer_path and tokenizer_path.exists():
+            zf.write(tokenizer_path, TOKENIZER_NAME)
+        if validation:
+            zf.writestr(VALIDATION_NAME, json.dumps(validation, indent=2))
+    return output
+
+
+def read_manifest(path: Path) -> Manifest:
+    """Read only the manifest from a .slm artifact without extracting the model."""
+    if not path.exists():
+        msg = f"Artifact not found: {path}"
+        raise FileNotFoundError(msg)
+    if not zipfile.is_zipfile(path):
+        msg = f"Not a valid .slm artifact (corrupt or wrong file): {path}"
+        raise ValueError(msg)
+    with zipfile.ZipFile(path, "r") as zf:
+        _validate_zip_entries(zf)
+        if MANIFEST_NAME not in zf.namelist():
+            msg = f"Artifact missing {MANIFEST_NAME}: {path}"
+            raise ValueError(msg)
+        manifest_data = zf.read(MANIFEST_NAME).decode()
+        return Manifest.from_json(manifest_data)
+
+
+def read_artifact(path: Path, extract_dir: Path | None = None) -> ArtifactContent:
+    """Extract a .slm artifact and return its contents."""
+    if not path.exists():
+        msg = f"Artifact not found: {path}"
+        raise FileNotFoundError(msg)
+    if not zipfile.is_zipfile(path):
+        msg = f"Not a valid .slm artifact (corrupt or wrong file): {path}"
+        raise ValueError(msg)
+
+    if extract_dir is None:
+        extract_dir = Path(tempfile.mkdtemp(prefix="slmkit_"))
+
+    with zipfile.ZipFile(path, "r") as zf:
+        _validate_zip_entries(zf)
+        names = zf.namelist()
+        has_gguf = MODEL_NAME in names
+        has_model_dir = any(n.startswith(MODEL_DIR) for n in names)
+        if not has_gguf and not has_model_dir:
+            msg = f"Artifact missing model data (expected '{MODEL_NAME}' or '{MODEL_DIR}'): {path}"
+            raise ValueError(msg)
+        for required in (MANIFEST_NAME, PROMPT_TEMPLATE_NAME):
+            if required not in names:
+                msg = f"Artifact missing required file '{required}': {path}"
+                raise ValueError(msg)
+        zf.extractall(extract_dir)
+
+    manifest = Manifest.from_path(extract_dir / MANIFEST_NAME)
+    prompt_template = (extract_dir / PROMPT_TEMPLATE_NAME).read_text()
+
+    # Determine model path: single GGUF file or model directory
+    model_dir = extract_dir / MODEL_DIR.rstrip("/")
+    gguf_path = extract_dir / MODEL_NAME
+    model_path_resolved = model_dir if model_dir.is_dir() else gguf_path
+
+    tokenizer_path = extract_dir / TOKENIZER_NAME
+    if not tokenizer_path.exists():
+        tokenizer_path = None
+
+    validation = None
+    validation_path = extract_dir / VALIDATION_NAME
+    if validation_path.exists():
+        validation = json.loads(validation_path.read_text())
+
+    return ArtifactContent(
+        root=extract_dir,
+        manifest=manifest,
+        model_path=model_path_resolved,
+        prompt_template=prompt_template,
+        tokenizer_path=tokenizer_path,
+        validation=validation,
+    )

slmkit/formats/manifest.py

@@ -0,0 +1,74 @@
+"""manifest.json parsing and creation."""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from enum import StrEnum, auto
+from pathlib import Path
+
+
+class TaskType(StrEnum):
+    CLASSIFICATION = auto()
+
+
+class BackendType(StrEnum):
+    LLAMACPP = auto()
+    MLX = auto()
+
+
+class ModelFormat(StrEnum):
+    GGUF = auto()
+    SAFETENSORS = auto()
+
+
+@dataclass(slots=True)
+class Manifest:
+    task: TaskType
+    backend: BackendType
+    source: str
+    quantization: str
+    model_format: ModelFormat = ModelFormat.GGUF
+    task_config: dict[str, object] = field(default_factory=dict)
+    version: str = "1"
+    created_at: str = field(default_factory=lambda: datetime.now(UTC).isoformat())
+
+    def to_dict(self) -> dict[str, object]:
+        return {
+            "version": self.version,
+            "task": self.task.value,
+            "backend": self.backend.value,
+            "model": {
+                "source": self.source,
+                "quantization": self.quantization,
+                "format": self.model_format.value,
+            },
+            "task_config": self.task_config,
+            "created_at": self.created_at,
+        }
+
+    def to_json(self) -> str:
+        return json.dumps(self.to_dict(), indent=2)
+
+    @classmethod
+    def from_dict(cls, data: dict[str, object]) -> Manifest:
+        model_data: dict[str, object] = data.get("model", {})  # type: ignore[assignment]
+        return cls(
+            version=str(data.get("version", "1")),
+            task=TaskType(data["task"]),  # type: ignore[arg-type]
+            backend=BackendType(data["backend"]),  # type: ignore[arg-type]
+            source=str(model_data["source"]),
+            quantization=str(model_data.get("quantization", "unknown")),
+            model_format=ModelFormat(model_data.get("format", "gguf")),  # type: ignore[arg-type]
+            task_config=data.get("task_config", {}),  # type: ignore[arg-type]
+            created_at=str(data.get("created_at", "")),
+        )
+
+    @classmethod
+    def from_json(cls, text: str) -> Manifest:
+        return cls.from_dict(json.loads(text))
+
+    @classmethod
+    def from_path(cls, path: Path) -> Manifest:
+        return cls.from_json(path.read_text())

slmkit/formats/schema.py

@@ -0,0 +1,31 @@
+"""Input/output schema validation."""
+
+from __future__ import annotations
+
+from slmkit.formats.manifest import Manifest, TaskType
+
+
+class SchemaError(ValueError):
+    """Raised when a manifest or task config fails validation."""
+
+
+def validate_manifest(manifest: Manifest) -> None:
+    """Validate that a manifest has all required fields for its task type."""
+    if manifest.task == TaskType.CLASSIFICATION:
+        _validate_classification(manifest)
+
+
+def _validate_classification(manifest: Manifest) -> None:
+    labels = manifest.task_config.get("labels")
+    if not labels:
+        raise SchemaError("Classification task requires 'labels' in task_config")
+    if not isinstance(labels, list) or len(labels) < 2:
+        raise SchemaError("Classification task requires at least 2 labels")
+    if len(labels) != len(set(labels)):
+        raise SchemaError("Classification labels must be unique")
+
+
+def validate_input(text: str) -> None:
+    """Validate model input."""
+    if not text or not text.strip():
+        raise SchemaError("Input text must not be empty")

slmkit/model.py

@@ -0,0 +1,88 @@
+"""Model loading and inference wrapper."""
+
+from __future__ import annotations
+
+import shutil
+import tempfile
+from dataclasses import asdict
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+from slmkit.backends import load_backend
+from slmkit.formats.artifact import ArtifactContent, read_artifact
+from slmkit.formats.schema import validate_input, validate_manifest
+from slmkit.tasks import parse_output
+
+if TYPE_CHECKING:
+    from slmkit.backends.llamacpp import LlamaCppBackend
+    from slmkit.backends.mlx import MlxBackend
+
+
+class SlmModel:
+    """A loaded .slm model — callable for single-shot inference."""
+
+    def __init__(
+        self,
+        artifact: ArtifactContent,
+        *,
+        _backend: LlamaCppBackend | MlxBackend,
+        _temp_dir: Path | None = None,
+    ) -> None:
+        self._artifact = artifact
+        self._backend = _backend
+        self._temp_dir = _temp_dir
+        self._system_prompt = artifact.prompt_template
+
+    @classmethod
+    def from_artifact(cls, path: str | Path) -> SlmModel:
+        """Load a .slm artifact and return a ready-to-use model."""
+        temp_dir = Path(tempfile.mkdtemp(prefix="slmkit_"))
+        artifact = read_artifact(Path(path), extract_dir=temp_dir)
+        validate_manifest(artifact.manifest)
+        backend = load_backend(artifact.manifest.backend, artifact.model_path)
+        return cls(artifact, _backend=backend, _temp_dir=temp_dir)
+
+    def __call__(self, text: str) -> dict[str, Any]:
+        """Classify a single input text."""
+        validate_input(text)
+        raw = self._backend.chat(self._system_prompt, text)
+        result = parse_output(
+            self._artifact.manifest.task,
+            raw,
+            self._artifact.manifest.task_config,
+        )
+        return asdict(result)
+
+    def batch(self, texts: list[str]) -> list[dict[str, Any]]:
+        """Run inference on a list of inputs."""
+        return [self(text) for text in texts]
+
+    def info(self) -> dict[str, Any]:
+        """Return artifact metadata."""
+        m = self._artifact.manifest
+        return {
+            "task": m.task.value,
+            "backend": m.backend.value,
+            "source": m.source,
+            "quantization": m.quantization,
+            "task_config": m.task_config,
+        }
+
+    def validation_cases(self) -> list[dict[str, object]] | None:
+        """Return baked-in validation test cases, if any."""
+        return self._artifact.validation
+
+    def close(self) -> None:
+        """Release backend resources and clean up extracted files."""
+        self._backend.close()
+        if self._temp_dir and self._temp_dir.exists():
+            shutil.rmtree(self._temp_dir, ignore_errors=True)
+
+    def __enter__(self) -> SlmModel:
+        return self
+
+    def __exit__(self, *_: object) -> None:
+        self.close()
+
+    def __del__(self) -> None:
+        self.close()

slmkit/tasks/__init__.py

@@ -0,0 +1,26 @@
+"""Task type registry."""
+
+from __future__ import annotations
+
+from slmkit.formats.manifest import TaskType
+from slmkit.tasks import classification
+
+
+def build_system_prompt(task: TaskType, task_config: dict[str, object]) -> str:
+    """Build the system prompt for a given task type and config."""
+    if task == TaskType.CLASSIFICATION:
+        labels: list[str] = task_config.get("labels", [])  # type: ignore[assignment]
+        return classification.build_system_prompt(labels)
+    msg = f"Unsupported task type: {task}"
+    raise ValueError(msg)
+
+
+def parse_output(
+    task: TaskType, raw: str, task_config: dict[str, object]
+) -> classification.ClassificationResult:
+    """Parse raw model output for a given task type."""
+    if task == TaskType.CLASSIFICATION:
+        labels: list[str] = task_config.get("labels", [])  # type: ignore[assignment]
+        return classification.parse_output(raw, labels)
+    msg = f"Unsupported task type: {task}"
+    raise ValueError(msg)

slmkit/tasks/classification.py

@@ -0,0 +1,41 @@
+"""Classification task — prompt template + output parser."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+_SYSTEM_PROMPT_TEMPLATE = (
+    "You are a text classifier. Classify the input text into exactly one of "
+    "these categories: {labels}.\n\n"
+    "Respond with ONLY the category label, nothing else."
+)
+
+
+@dataclass(slots=True, frozen=True)
+class ClassificationResult:
+    label: str
+    confidence: float
+
+
+def build_system_prompt(labels: list[str]) -> str:
+    """Build the system prompt for classification with the given labels."""
+    return _SYSTEM_PROMPT_TEMPLATE.format(labels=", ".join(labels))
+
+
+def parse_output(raw: str, labels: list[str]) -> ClassificationResult:
+    """Parse raw model output into a classification result."""
+    cleaned = raw.strip()
+    cleaned_lower = cleaned.lower()
+
+    # Exact match (case-insensitive)
+    for label in labels:
+        if cleaned_lower == label.lower():
+            return ClassificationResult(label=label, confidence=1.0)
+
+    # Substring match — model may have added extra text
+    for label in labels:
+        if label.lower() in cleaned_lower:
+            return ClassificationResult(label=label, confidence=0.8)
+
+    # No match — return raw output with zero confidence
+    return ClassificationResult(label=cleaned, confidence=0.0)

slmkit/tasks/embedding.py

@@ -0,0 +1 @@
+"""Vector embedding task (future)."""

slmkit/tasks/extraction.py

@@ -0,0 +1 @@
+"""Structured extraction task (future)."""

slmkit/tasks/generation.py

@@ -0,0 +1 @@
+"""Free-form text generation task (future)."""