troy-cli 0.1.2__tar.gz

troy_cli-0.1.2/.gitignore

@@ -0,0 +1,6 @@
+.venv/
+__pycache__/
+*.egg-info/
+dist/
+output/
+.DS_Store

troy_cli-0.1.2/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.5
+Name: troy-cli
+Version: 0.1.2
+Summary: Fine-tune LLMs on your MacBook with one YAML file. Built for Apple Silicon.
+Author: Troy
+License: Apache-2.0
+Keywords: apple-silicon,dpo,fine-tuning,llm,lora,mlx
+Requires-Python: >=3.10
+Requires-Dist: mlx-lm>=0.30
+Requires-Dist: pydantic>=2.5
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.0
+Requires-Dist: typer>=0.12
+Description-Content-Type: text/markdown
+
+# Troy
+
+**Fine-tune LLMs on your MacBook with one YAML file.**
+
+Troy is a command-line tool for fine-tuning and preference-tuning language
+models locally on Apple Silicon. No CUDA, no cloud, no training pipeline —
+write a config, run one command, and train on the machine you already own.
+
+Built on [MLX](https://github.com/ml-explore/mlx) and
+[mlx-lm](https://github.com/ml-explore/mlx-lm), Apple's ML framework for
+Apple Silicon. Unified memory means a 36 GB MacBook fine-tunes models that
+need a workstation GPU anywhere else.
+
+## Requirements
+
+- Apple Silicon Mac (M1 or later)
+- macOS 14+
+- Python 3.10–3.12
+
+## Install
+
+```bash
+brew install avirajkhare00/troy/troy
+# or from source: pip install ./cli
+```
+
+## Quickstart
+
+```bash
+troy doctor            # check your Mac: chip, memory, MLX, what you can train
+troy init              # create troy.yaml + sample data
+troy train             # fine-tune (LoRA/QLoRA via MLX)
+troy chat              # talk to the result
+troy serve             # OpenAI-compatible API at localhost:8080/v1
+troy export -f gguf    # ship it to llama.cpp / Ollama / LM Studio
+```
+
+## The config is the interface
+
+```yaml
+base: mlx-community/Qwen3-0.6B-4bit
+task: sft            # or: dpo
+
+data:
+  train: ./data/train.jsonl   # alpaca, sharegpt, chat, completions, text — auto-detected
+  val_split: 0.1
+
+training:
+  epochs: 3
+  lr: 1e-5
+  batch_size: auto   # sized from your Mac's unified memory
+  lora:
+    r: 8
+    alpha: 16
+
+output: ./output
+```
+
+## Commands
+
+| Command | Purpose |
+|---|---|
+| `troy init` | Create a config from a template (`chat`, `dpo`) |
+| `troy doctor` | Hardware + dependency check, with model-size guidance |
+| `troy train` | LoRA fine-tuning: SFT or DPO |
+| `troy chat` | Interactive REPL (or `-p` for one-shot) with your adapter |
+| `troy serve` | OpenAI-compatible API server for your model |
+| `troy export` | Fuse the adapter; export MLX or GGUF |
+| `troy data inspect` | Dataset stats and format detection |
+
+## What Troy can train on your Mac
+
+| Unified memory | Max model (4-bit QLoRA) |
+|---|---|
+| 8 GB | ~1.5B |
+| 16 GB | ~4B |
+| 24 GB | ~8B |
+| 36 GB | ~14B |
+| 64 GB | ~32B |
+| 128 GB | ~70B |
+
+## DPO without a second model
+
+DPO normally keeps a frozen reference copy of the model in memory. Troy
+zeroes the LoRA scales to recover the reference model from the policy model
+itself — no second copy, which matters on unified memory.
+
+## Data formats
+
+Auto-detected from the first record: Alpaca (`instruction`/`output`),
+ShareGPT (`conversations`), chat (`messages`), `prompt`/`completion`,
+plain `text`, and preference pairs (`prompt`/`chosen`/`rejected`) for DPO.
+Files: `.jsonl`, `.json`, `.csv`.
+
+## License
+
+Apache-2.0

troy_cli-0.1.2/README.md

@@ -0,0 +1,97 @@
+# Troy
+
+**Fine-tune LLMs on your MacBook with one YAML file.**
+
+Troy is a command-line tool for fine-tuning and preference-tuning language
+models locally on Apple Silicon. No CUDA, no cloud, no training pipeline —
+write a config, run one command, and train on the machine you already own.
+
+Built on [MLX](https://github.com/ml-explore/mlx) and
+[mlx-lm](https://github.com/ml-explore/mlx-lm), Apple's ML framework for
+Apple Silicon. Unified memory means a 36 GB MacBook fine-tunes models that
+need a workstation GPU anywhere else.
+
+## Requirements
+
+- Apple Silicon Mac (M1 or later)
+- macOS 14+
+- Python 3.10–3.12
+
+## Install
+
+```bash
+brew install avirajkhare00/troy/troy
+# or from source: pip install ./cli
+```
+
+## Quickstart
+
+```bash
+troy doctor            # check your Mac: chip, memory, MLX, what you can train
+troy init              # create troy.yaml + sample data
+troy train             # fine-tune (LoRA/QLoRA via MLX)
+troy chat              # talk to the result
+troy serve             # OpenAI-compatible API at localhost:8080/v1
+troy export -f gguf    # ship it to llama.cpp / Ollama / LM Studio
+```
+
+## The config is the interface
+
+```yaml
+base: mlx-community/Qwen3-0.6B-4bit
+task: sft            # or: dpo
+
+data:
+  train: ./data/train.jsonl   # alpaca, sharegpt, chat, completions, text — auto-detected
+  val_split: 0.1
+
+training:
+  epochs: 3
+  lr: 1e-5
+  batch_size: auto   # sized from your Mac's unified memory
+  lora:
+    r: 8
+    alpha: 16
+
+output: ./output
+```
+
+## Commands
+
+| Command | Purpose |
+|---|---|
+| `troy init` | Create a config from a template (`chat`, `dpo`) |
+| `troy doctor` | Hardware + dependency check, with model-size guidance |
+| `troy train` | LoRA fine-tuning: SFT or DPO |
+| `troy chat` | Interactive REPL (or `-p` for one-shot) with your adapter |
+| `troy serve` | OpenAI-compatible API server for your model |
+| `troy export` | Fuse the adapter; export MLX or GGUF |
+| `troy data inspect` | Dataset stats and format detection |
+
+## What Troy can train on your Mac
+
+| Unified memory | Max model (4-bit QLoRA) |
+|---|---|
+| 8 GB | ~1.5B |
+| 16 GB | ~4B |
+| 24 GB | ~8B |
+| 36 GB | ~14B |
+| 64 GB | ~32B |
+| 128 GB | ~70B |
+
+## DPO without a second model
+
+DPO normally keeps a frozen reference copy of the model in memory. Troy
+zeroes the LoRA scales to recover the reference model from the policy model
+itself — no second copy, which matters on unified memory.
+
+## Data formats
+
+Auto-detected from the first record: Alpaca (`instruction`/`output`),
+ShareGPT (`conversations`), chat (`messages`), `prompt`/`completion`,
+plain `text`, and preference pairs (`prompt`/`chosen`/`rejected`) for DPO.
+Files: `.jsonl`, `.json`, `.csv`.
+
+## License
+
+Apache-2.0

troy_cli-0.1.2/pyproject.toml

@@ -0,0 +1,26 @@
+[project]
+name = "troy-cli"
+version = "0.1.2"
+description = "Fine-tune LLMs on your MacBook with one YAML file. Built for Apple Silicon."
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "Apache-2.0" }
+authors = [{ name = "Troy" }]
+keywords = ["mlx", "apple-silicon", "fine-tuning", "llm", "lora", "dpo"]
+dependencies = [
+    "mlx-lm>=0.30",
+    "pydantic>=2.5",
+    "pyyaml>=6.0",
+    "typer>=0.12",
+    "rich>=13.0",
+]
+
+[project.scripts]
+troy = "troy.cli:app"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/troy"]

troy_cli-0.1.2/src/troy/__init__.py

@@ -0,0 +1,3 @@
+"""Troy: fine-tune LLMs on your MacBook with one YAML file."""
+
+__version__ = "0.1.2"

troy_cli-0.1.2/src/troy/chat.py

@@ -0,0 +1,57 @@
+"""Interactive chat with a trained model."""
+
+from __future__ import annotations
+
+from typing import Optional
+
+from mlx_lm.generate import stream_generate
+from mlx_lm.sample_utils import make_sampler
+from mlx_lm.utils import load
+
+
+def run_chat(
+    model_path: str,
+    adapter_path: Optional[str] = None,
+    max_tokens: int = 512,
+    temperature: float = 0.7,
+    prompt: Optional[str] = None,
+) -> None:
+    print(f"Loading {model_path} ...")
+    model, tokenizer = load(model_path, adapter_path=adapter_path)
+    sampler = make_sampler(temp=temperature)
+    history = []
+
+    def respond(user_text: str) -> None:
+        history.append({"role": "user", "content": user_text})
+        templated = tokenizer.apply_chat_template(
+            history, add_generation_prompt=True, return_dict=False
+        )
+        reply = ""
+        for response in stream_generate(
+            model, tokenizer, templated, max_tokens=max_tokens, sampler=sampler
+        ):
+            print(response.text, end="", flush=True)
+            reply += response.text
+        print()
+        history.append({"role": "assistant", "content": reply})
+
+    if prompt is not None:  # one-shot mode
+        respond(prompt)
+        return
+
+    print("Chat started. Type /exit to quit, /clear to reset history.\n")
+    while True:
+        try:
+            user_text = input(">> ").strip()
+        except (EOFError, KeyboardInterrupt):
+            print()
+            break
+        if not user_text:
+            continue
+        if user_text == "/exit":
+            break
+        if user_text == "/clear":
+            history.clear()
+            print("(history cleared)")
+            continue
+        respond(user_text)

troy_cli-0.1.2/src/troy/cli.py

@@ -0,0 +1,274 @@
+"""Troy CLI: fine-tune LLMs on your MacBook with one YAML file."""
+
+from __future__ import annotations
+
+import json
+import platform
+import sys
+from pathlib import Path
+from typing import Optional
+
+import typer
+from rich.console import Console
+from rich.table import Table
+
+from . import __version__
+
+app = typer.Typer(
+    name="troy",
+    help="Fine-tune LLMs on your MacBook with one YAML file. Built for Apple Silicon.",
+    no_args_is_help=True,
+    add_completion=False,
+)
+console = Console()
+
+
+def _require_apple_silicon() -> None:
+    if platform.system() != "Darwin" or platform.machine() != "arm64":
+        console.print(
+            "[red]Troy runs on Apple Silicon Macs only (M1 or later).[/red]\n"
+            f"Detected: {platform.system()} / {platform.machine()}"
+        )
+        raise typer.Exit(1)
+
+
+def _version_callback(value: bool) -> None:
+    if value:
+        console.print(f"troy {__version__}")
+        raise typer.Exit()
+
+
+@app.callback()
+def _main(
+    version: bool = typer.Option(
+        False, "--version", "-V", help="Show version.",
+        callback=_version_callback, is_eager=True,
+    ),
+) -> None:
+    pass
+
+
+@app.command()
+def init(
+    template: str = typer.Option(
+        "chat", help="Template: chat (SFT) or dpo (preference tuning)."
+    ),
+    path: Path = typer.Option(Path("troy.yaml"), help="Where to write the config."),
+    force: bool = typer.Option(False, "--force", help="Overwrite an existing config."),
+) -> None:
+    """Create a troy.yaml config (plus sample data) to start from."""
+    from .templates import TEMPLATES
+
+    if template not in TEMPLATES:
+        console.print(f"[red]Unknown template `{template}`.[/red] Options: {', '.join(TEMPLATES)}")
+        raise typer.Exit(1)
+    if path.exists() and not force:
+        console.print(f"[red]{path} already exists.[/red] Use --force to overwrite.")
+        raise typer.Exit(1)
+
+    config_text, data_name, sample = TEMPLATES[template]
+    path.write_text(config_text)
+
+    data_dir = path.parent / "data"
+    data_file = data_dir / data_name
+    if not data_file.exists():
+        data_dir.mkdir(parents=True, exist_ok=True)
+        with open(data_file, "w") as f:
+            for record in sample:
+                f.write(json.dumps(record) + "\n")
+        console.print(f"Wrote sample data to [bold]{data_file}[/bold] — replace it with yours.")
+
+    console.print(f"Created [bold]{path}[/bold] ({template} template).")
+    console.print("Next: edit the config, then run [bold]troy train[/bold].")
+
+
+@app.command()
+def doctor() -> None:
+    """Check this Mac's readiness for local fine-tuning."""
+    from .hardware import detect, model_guidance
+
+    hw = detect()
+    table = Table(title="troy doctor", show_header=False)
+    table.add_column(style="bold")
+    table.add_column()
+
+    ok = "[green]OK[/green]"
+    fail = "[red]FAIL[/red]"
+
+    table.add_row("Chip", hw.chip)
+    table.add_row(
+        "Apple Silicon", ok if hw.is_apple_silicon else f"{fail} ({hw.arch})"
+    )
+    table.add_row("Unified memory", f"{hw.memory_gb:.0f} GB")
+    table.add_row("macOS", hw.macos)
+    table.add_row("Free disk", f"{hw.free_disk_gb:.0f} GB")
+    table.add_row("Python", platform.python_version())
+
+    try:
+        import mlx.core as mx
+
+        table.add_row("MLX", f"{ok} (v{mx.__version__})")
+        gpu_ok = mx.default_device().type == mx.DeviceType.gpu
+        table.add_row("Metal GPU", ok if gpu_ok else f"{fail} (default device is CPU)")
+    except ImportError:
+        table.add_row("MLX", f"{fail} (not installed — `pip install mlx-lm`)")
+
+    try:
+        import mlx_lm
+
+        table.add_row("mlx-lm", f"{ok} (v{mlx_lm.__version__})")
+    except ImportError:
+        table.add_row("mlx-lm", f"{fail} (not installed)")
+
+    table.add_row("Fine-tunable models", model_guidance(hw.memory_gb))
+    console.print(table)
+
+    if not hw.is_apple_silicon:
+        raise typer.Exit(1)
+
+
+@app.command()
+def train(
+    config: Path = typer.Option(Path("troy.yaml"), "--config", "-c", help="Config file."),
+) -> None:
+    """Fine-tune a model from a troy.yaml config."""
+    _require_apple_silicon()
+    from .config import load_config
+    from .data import load_and_prepare
+
+    cfg = load_config(config)
+    train_records, valid_records, fmt = load_and_prepare(
+        cfg.data, cfg.task, cfg.training.seed
+    )
+    console.print(
+        f"Data: {len(train_records)} train / {len(valid_records)} valid "
+        f"(format: {fmt})"
+    )
+
+    if cfg.task == "sft":
+        from .train_sft import run_sft
+
+        run_sft(cfg, train_records, valid_records)
+    else:
+        from .train_dpo import run_dpo
+
+        run_dpo(cfg, train_records, valid_records)
+
+    console.print(
+        f"\nTry it: [bold]troy chat[/bold]   |   "
+        f"Export it: [bold]troy export[/bold]"
+    )
+
+
+@app.command()
+def chat(
+    config: Path = typer.Option(Path("troy.yaml"), "--config", "-c", help="Config file."),
+    model: Optional[str] = typer.Option(
+        None, help="Model path or HF repo (defaults to the config's base + trained adapter)."
+    ),
+    base_only: bool = typer.Option(False, help="Chat with the base model, no adapter."),
+    max_tokens: int = typer.Option(512),
+    temperature: float = typer.Option(0.7),
+    prompt: Optional[str] = typer.Option(None, "--prompt", "-p", help="One-shot prompt (no REPL)."),
+) -> None:
+    """Chat with your fine-tuned model."""
+    _require_apple_silicon()
+    from .chat import run_chat
+
+    adapter: Optional[str] = None
+    if model is None:
+        from .config import load_config
+
+        cfg = load_config(config)
+        model = cfg.base
+        if not base_only:
+            adapter_file = cfg.adapter_path / "adapters.safetensors"
+            if adapter_file.exists():
+                adapter = str(cfg.adapter_path)
+            else:
+                console.print(
+                    "[yellow]No trained adapter found — chatting with the base model.[/yellow]"
+                )
+    run_chat(model, adapter, max_tokens, temperature, prompt)
+
+
+@app.command()
+def serve(
+    config: Path = typer.Option(Path("troy.yaml"), "--config", "-c", help="Config file."),
+    model: Optional[str] = typer.Option(
+        None, help="Model path or HF repo (defaults to the config's base + trained adapter)."
+    ),
+    base_only: bool = typer.Option(False, help="Serve the base model, no adapter."),
+    host: str = typer.Option("127.0.0.1"),
+    port: int = typer.Option(8080),
+    max_tokens: int = typer.Option(512, help="Default max tokens per response."),
+) -> None:
+    """Serve your model over an OpenAI-compatible API."""
+    _require_apple_silicon()
+    from .serve import run_serve
+
+    adapter: Optional[str] = None
+    if model is None:
+        from .config import load_config
+
+        cfg = load_config(config)
+        model = cfg.base
+        if not base_only:
+            adapter_file = cfg.adapter_path / "adapters.safetensors"
+            if adapter_file.exists():
+                adapter = str(cfg.adapter_path)
+            else:
+                console.print(
+                    "[yellow]No trained adapter found — serving the base model.[/yellow]"
+                )
+    run_serve(model, adapter, host, port, max_tokens)
+
+
+@app.command()
+def export(
+    config: Path = typer.Option(Path("troy.yaml"), "--config", "-c", help="Config file."),
+    fmt: str = typer.Option("mlx", "--format", "-f", help="Export format: mlx or gguf."),
+    save_path: Optional[Path] = typer.Option(None, help="Output directory (default: <output>/fused)."),
+    dequantize: bool = typer.Option(False, help="Dequantize when fusing a quantized base."),
+) -> None:
+    """Merge the trained adapter into the base model and export it."""
+    _require_apple_silicon()
+    if fmt not in ("mlx", "gguf"):
+        console.print("[red]--format must be `mlx` or `gguf`.[/red]")
+        raise typer.Exit(1)
+    from .config import load_config
+    from .export import run_export
+
+    cfg = load_config(config)
+    adapter_file = cfg.adapter_path / "adapters.safetensors"
+    if not adapter_file.exists():
+        console.print(f"[red]No adapter at {adapter_file}. Run `troy train` first.[/red]")
+        raise typer.Exit(1)
+    out = save_path or (cfg.output_path / "fused")
+    run_export(cfg.base, str(cfg.adapter_path), str(out), fmt, dequantize)
+
+
+@app.command()
+def data(
+    action: str = typer.Argument(help="Action: inspect"),
+    path: Path = typer.Argument(help="Dataset file (.jsonl, .json, .csv)."),
+) -> None:
+    """Inspect a dataset: record count, detected format, sizes."""
+    if action != "inspect":
+        console.print("[red]Only `troy data inspect <path>` is supported.[/red]")
+        raise typer.Exit(1)
+    from .data import inspect_stats
+
+    stats = inspect_stats(str(path))
+    table = Table(title=str(path), show_header=False)
+    table.add_column(style="bold")
+    table.add_column()
+    table.add_row("Records", str(stats["records"]))
+    table.add_row("Detected format", stats["format"])
+    table.add_row("Avg record size", f"{stats['avg_chars']:.0f} chars")
+    table.add_row("Max record size", f"{stats['max_chars']} chars")
+    console.print(table)
+
+
+if __name__ == "__main__":
+    app()

troy_cli-0.1.2/src/troy/config.py

@@ -0,0 +1,82 @@
+"""Troy YAML config schema and loader."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Literal, Optional, Union
+
+import yaml
+from pydantic import BaseModel, Field, field_validator
+
+
+class LoraConfig(BaseModel):
+    r: int = 8
+    alpha: float = 16.0
+    dropout: float = 0.0
+    layers: Union[int, Literal["all"]] = 16  # decoder layers to adapt
+
+    @property
+    def scale(self) -> float:
+        return self.alpha / self.r
+
+
+class DpoConfig(BaseModel):
+    beta: float = 0.1
+
+
+class DataConfig(BaseModel):
+    train: str
+    valid: Optional[str] = None
+    format: Literal[
+        "auto", "alpaca", "sharegpt", "chat", "completions", "text", "preference"
+    ] = "auto"
+    val_split: float = Field(0.1, ge=0.0, lt=1.0)
+    mask_prompt: bool = False
+
+
+class TrainingConfig(BaseModel):
+    epochs: Optional[float] = None  # translated to iters from dataset size
+    iters: Optional[int] = None  # takes precedence over epochs
+    lr: float = 1e-5
+    batch_size: Union[int, Literal["auto"]] = "auto"
+    seq_len: int = 2048
+    lora: LoraConfig = LoraConfig()
+    dpo: DpoConfig = DpoConfig()
+    grad_checkpoint: bool = False
+    grad_accumulation_steps: int = 1
+    save_every: int = 100
+    seed: int = 0
+
+
+class TroyConfig(BaseModel):
+    base: str
+    task: Literal["sft", "dpo"] = "sft"
+    data: DataConfig
+    training: TrainingConfig = TrainingConfig()
+    output: str = "./output"
+
+    @field_validator("base")
+    @classmethod
+    def _non_empty(cls, v: str) -> str:
+        if not v.strip():
+            raise ValueError("`base` must be a HuggingFace repo id or local path")
+        return v
+
+    @property
+    def output_path(self) -> Path:
+        return Path(self.output).expanduser()
+
+    @property
+    def adapter_path(self) -> Path:
+        return self.output_path / "adapter"
+
+
+def load_config(path: Union[str, Path]) -> TroyConfig:
+    path = Path(path)
+    if not path.exists():
+        raise FileNotFoundError(
+            f"Config not found: {path}. Run `troy init` to create one."
+        )
+    with open(path) as f:
+        raw = yaml.safe_load(f) or {}
+    return TroyConfig.model_validate(raw)