local-llm-ctl 0.1.0__tar.gz

local_llm_ctl-0.1.0/MANIFEST.in

@@ -0,0 +1,8 @@
+include README.md
+include pyproject.toml
+include LICENSE
+recursive-include src/llm_ctl *.py
+recursive-include tests *.py
+prune src/llm_ctl.egg-info
+prune build
+prune dist

local_llm_ctl-0.1.0/PKG-INFO

@@ -0,0 +1,9 @@
+Metadata-Version: 2.4
+Name: local-llm-ctl
+Version: 0.1.0
+Summary: CLI tool to manage local LLM models and role-based server lifecycle
+Author: Renato Rozas
+License-Expression: MIT
+Requires-Python: >=3.9
+Requires-Dist: click>=8.0
+Requires-Dist: huggingface-hub>=0.20

local_llm_ctl-0.1.0/README.md

@@ -0,0 +1,221 @@
+# llm-ctl
+
+A CLI tool to manage local LLM models and role-based server lifecycle.
+
+Register models to roles (planner, reviewer, engineer, assistant), start/stop inference servers, and monitor their status — all from the terminal.
+
+## Features
+
+- **Model discovery** — Automatically scans HuggingFace cache and local GGUF files
+- **Role-based management** — Assign models to planner, reviewer, engineer, or assistant roles
+- **Server lifecycle** — Start, stop, and monitor inference servers with PID tracking
+- **Auto port assignment** — Automatically finds available ports for each role
+- **MLX & GGUF support** — Works with both Apple MLX and llama.cpp servers
+- **State persistence** — Survives restarts via JSON state file
+
+## Dependencies
+
+### Required
+
+- **Python 3.9+**
+- **click** — CLI framework
+- **huggingface-hub** — HuggingFace model management
+
+### Server backend (one required)
+
+You need at least one inference backend to serve models:
+
+- **mlx-lm** — For serving MLX models on Apple Silicon (`mlx_lm serve`)
+- **llama.cpp** — For serving GGUF models on any platform (`llama-server`)
+
+Install one or both depending on your needs:
+
+```bash
+# MLX models (Apple Silicon Macs)
+pip install mlx-lm
+
+# GGUF models (any platform)
+brew install llama.cpp
+```
+
+## Installation
+
+### Quick install (recommended)
+
+```bash
+pip install llm-ctl
+```
+
+### From source
+
+```bash
+git clone https://github.com/renatorozas/llm-ctl.git
+cd llm-ctl
+python -m venv .venv
+source .venv/bin/activate
+pip install -e .
+```
+
+### With uv
+
+```bash
+uv pip install llm-ctl
+```
+
+### With pipx (system-wide, isolated)
+
+```bash
+pipx install llm-ctl
+```
+
+## Setup
+
+### 1. Install a server backend
+
+For **MLX models** (Apple Silicon Macs):
+
+```bash
+pip install mlx-lm
+```
+
+For **GGUF models** (any platform):
+
+```bash
+# Install llama.cpp (follow instructions at https://github.com/ggerganov/llama.cpp)
+# On macOS with Homebrew:
+brew install llama.cpp
+```
+
+### 2. Download models
+
+```bash
+# Download from HuggingFace
+llm-ctl download mlx-community/Qwen3.6-27B-4bit
+
+# Or use models already in your HuggingFace cache
+llm-ctl list
+```
+
+### 3. Register a role
+
+```bash
+# Register a model to a role (auto-detects type and assigns a port)
+llm-ctl register planner mlx-community/Qwen3.6-27B-4bit
+
+# Override model type or port
+llm-ctl register reviewer /path/to/model.gguf --type gguf --port 8081
+```
+
+### 4. Start servers
+
+```bash
+# Start a single role
+llm-ctl start planner
+
+# Start all registered roles
+llm-ctl start --all
+
+# With custom timeout
+llm-ctl start planner --timeout 120
+```
+
+### 5. Check status
+
+```bash
+llm-ctl status
+```
+
+### 6. Stop servers
+
+```bash
+# Stop a single role
+llm-ctl stop planner
+
+# Stop all roles
+llm-ctl stop --all
+```
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `llm-ctl list` | List all available local models |
+| `llm-ctl register <role> <model_id>` | Register a model to a role |
+| `llm-ctl unregister <role>` | Unregister a model from a role |
+| `llm-ctl start [role]` | Start server for a role (or `--all`) |
+| `llm-ctl stop [role]` | Stop server for a role (or `--all`) |
+| `llm-ctl status` | Show status of all registered roles |
+| `llm-ctl config [show\|edit]` | View or edit the configuration file |
+| `llm-ctl download <model_id>` | Download a model from HuggingFace |
+
+## Configuration
+
+Configuration is stored in `~/.config/llm-ctl/config.json`:
+
+```json
+{
+  "roles": {
+    "planner": {
+      "model_id": "mlx-community/Qwen3.6-27B-4bit",
+      "path": "/Users/renato/models/huggingface/hub/models--mlx-community--Qwen3.6-27B-4bit/snapshots/abc123",
+      "type": "mlx",
+      "port": 8080
+    }
+  },
+  "defaults": {
+    "base_port": 8080
+  }
+}
+```
+
+View or edit the config:
+
+```bash
+llm-ctl config show
+llm-ctl config edit
+```
+
+## State
+
+Runtime state (PIDs, ports, start times) is stored in `~/.config/llm-ctl/state.json`. Stale entries are automatically cleaned up on status checks.
+
+Server logs are written to `~/.config/llm-ctl/logs/<role>.log`.
+
+## Roles
+
+| Role | Description |
+|------|-------------|
+| `planner` | Planning and task decomposition |
+| `reviewer` | Code review and quality checks |
+| `engineer` | Implementation and coding |
+| `assistant` | General assistance |
+
+## Testing
+
+```bash
+python -m pytest tests/ -v
+```
+
+## Project Structure
+
+```
+llm-ctl/
+├── src/llm_ctl/
+│   ├── __init__.py
+│   ├── __main__.py
+│   ├── cli.py          # Click CLI commands
+│   ├── config.py       # Configuration management
+│   ├── models.py       # Model discovery and management
+│   ├── server.py       # Server lifecycle management
+│   └── utils.py        # Utility functions
+├── tests/
+│   ├── test_config.py
+│   ├── test_models.py
+│   ├── test_server.py
+│   └── test_utils.py
+└── pyproject.toml
+```
+
+## License
+
+MIT

local_llm_ctl-0.1.0/pyproject.toml

@@ -0,0 +1,27 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "local-llm-ctl"
+version = "0.1.0"
+description = "CLI tool to manage local LLM models and role-based server lifecycle"
+requires-python = ">=3.9"
+license = "MIT"
+authors = [
+    {name = "Renato Rozas"}
+]
+dependencies = [
+    "click>=8.0",
+    "huggingface-hub>=0.20",
+]
+
+[project.scripts]
+llm-ctl = "llm_ctl.cli:cli"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+exclude = ["*.egg-info"]
+
+[tool.setuptools.package-data]
+llm_ctl = ["*.py"]

local_llm_ctl-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

local_llm_ctl-0.1.0/src/llm_ctl/__init__.py

@@ -0,0 +1,3 @@
+"""llm-ctl: CLI tool to manage local LLM models."""
+
+__version__ = "0.1.0"

local_llm_ctl-0.1.0/src/llm_ctl/__main__.py

@@ -0,0 +1,4 @@
+from llm_ctl.cli import cli
+
+if __name__ == "__main__":
+    cli()

local_llm_ctl-0.1.0/src/llm_ctl/cli.py

@@ -0,0 +1,294 @@
+"""CLI entry point for llm-ctl."""
+
+import json
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+import click
+
+from .config import ConfigManager, VALID_ROLES, CONFIG_PATH
+from .models import discover_models, find_model_by_id, register_model, unregister_model
+from .server import ServerManager, StateFile
+from .utils import is_port_available, find_next_available_port
+
+
+@click.group()
+@click.version_option(version="0.1.0", prog_name="llm-ctl")
+def cli():
+    """Manage local LLM models and role-based server lifecycle."""
+    pass
+
+
+@cli.command("list")
+def list_models():
+    """List all available local models."""
+    models = discover_models()
+    if not models:
+        click.echo("No models found.")
+        return
+
+    click.echo(f"\n{'Model ID':<55} {'Type':<6} {'Path'}")
+    click.echo("-" * 100)
+    for model in sorted(models, key=lambda m: m["model_id"]):
+        path_display = model["path"]
+        if len(path_display) > 40:
+            path_display = "..." + path_display[-37:]
+        click.echo(f"{model['model_id']:<55} {model['type']:<6} {path_display}")
+
+    click.echo(f"\nTotal: {len(models)} model(s)")
+
+
+@cli.command()
+@click.argument("role")
+@click.argument("model_id")
+@click.option("--port", "-p", type=int, default=None, help="Override auto-assigned port")
+@click.option("--type", "-t", "model_type", type=click.Choice(["mlx", "gguf"]), default=None,
+              help="Override auto-detected model type")
+def register(role, model_id, port, model_type):
+    """Register a model to a role.
+
+    ROLE: planner, reviewer, engineer, or assistant
+
+    MODEL_ID: HuggingFace model ID (e.g., lmstudio-community/Qwen3.6-27B-MLX-6bit)
+    """
+    if role not in VALID_ROLES:
+        click.echo(f"Error: Invalid role '{role}'. Must be one of: {', '.join(sorted(VALID_ROLES))}")
+        sys.exit(1)
+
+    try:
+        config = ConfigManager()
+        discovered = discover_models()
+        model = find_model_by_id(model_id, discovered)
+        if not model:
+            click.echo(f"Error: Model not found: {model_id}")
+            click.echo("Run 'llm-ctl list' to see available models.")
+            sys.exit(1)
+
+        # Check if role is already registered
+        existing = config.get_role(role)
+        if existing:
+            click.echo(f"Role '{role}' is already registered to: {existing['model_id']}")
+            if not click.confirm("Unregister and re-register?"):
+                return
+
+        # Check for shared-model warning
+        all_roles = config.get_all_roles()
+        for other_role, other_config in all_roles.items():
+            if other_role == role:
+                continue
+            if other_config.get("model_id") == model_id:
+                click.echo(
+                    f"Warning: Role '{other_role}' is already using '{model_id}'. "
+                    f"Each role runs a separate server process, which means duplicate "
+                    f"GPU memory usage. Consider using --all with a single role or "
+                    f"assigning different models."
+                )
+                break
+
+        # Use user-specified type or auto-detected type
+        effective_type = model_type if model_type else model["type"]
+
+        # Auto-assign or use provided port
+        used_ports = {r.get("port") for r in all_roles.values() if r.get("port") and r.get("model_id") != model_id}
+        if port:
+            if not is_port_available(port):
+                click.echo(f"Error: Port {port} is not available.")
+                sys.exit(1)
+        else:
+            base_port = config.get_base_port()
+            port = find_next_available_port(start=base_port, exclude=used_ports)
+
+        role_config = {
+            "model_id": model["model_id"],
+            "path": model["path"],
+            "type": effective_type,
+            "port": port,
+        }
+        config.register_role(role, role_config)
+        click.echo(f"Registered '{model_id}' to role '{role}' on port {port} (type: {effective_type})")
+
+    except ValueError as e:
+        click.echo(f"Error: {e}")
+        sys.exit(1)
+
+
+@cli.command()
+@click.argument("role")
+def unregister(role):
+    """Unregister a model from a role."""
+    if role not in VALID_ROLES:
+        click.echo(f"Error: Invalid role '{role}'. Must be one of: {', '.join(sorted(VALID_ROLES))}")
+        sys.exit(1)
+
+    config = ConfigManager()
+    if not config.get_role(role):
+        click.echo(f"Role '{role}' is not registered.")
+        return
+
+    # Stop server if running
+    server = ServerManager()
+    server.stop(role)
+
+    config.unregister_role(role)
+    click.echo(f"Unregistered role '{role}'")
+
+
+@cli.command()
+@click.argument("role", required=False)
+@click.option("--all", "-a", "all_roles", is_flag=True, help="Start all registered role servers")
+@click.option("--timeout", "-t", type=int, default=None, help="Override startup timeout in seconds")
+def start(role, all_roles, timeout):
+    """Start the server for a role, or all roles with --all."""
+    # Guard against --all + positional role conflict
+    if all_roles and role:
+        click.echo("Error: Cannot specify both --all and a role.")
+        sys.exit(1)
+
+    server = ServerManager()
+    if timeout:
+        server.STARTUP_TIMEOUT = timeout
+    config = ConfigManager()
+
+    if all_roles:
+        results = server.start_all()
+        for r in results:
+            status_icon = "\u2713" if r["status"] == "started" else "\u2717"
+            detail = f"port {r.get('info', {}).get('port', '?')}" if r["status"] == "started" else r.get("error", "")
+            click.echo(f"  {status_icon} {r['role']}: {r['status']} {detail}")
+        return
+
+    if not role:
+        click.echo("Error: Specify a role or use --all.")
+        click.echo(f"Available roles: {', '.join(sorted(VALID_ROLES))}")
+        sys.exit(1)
+
+    if role not in VALID_ROLES:
+        click.echo(f"Error: Invalid role '{role}'. Must be one of: {', '.join(sorted(VALID_ROLES))}")
+        sys.exit(1)
+
+    role_config = config.get_role(role)
+    if not role_config:
+        click.echo(f"Error: Role '{role}' is not registered. Run 'llm-ctl register {role} <model_id>'")
+        sys.exit(1)
+
+    try:
+        state = server.start(role, role_config)
+        click.echo(f"Started {role} server (PID {state['pid']}, port {state['port']})")
+        click.echo(f"Log: {state.get('log_file', 'N/A')}")
+    except RuntimeError as e:
+        click.echo(f"Error: {e}")
+        sys.exit(1)
+
+
+@cli.command()
+@click.argument("role", required=False)
+@click.option("--all", "-a", "all_roles", is_flag=True, help="Stop all running servers")
+def stop(role, all_roles):
+    """Stop the server for a role, or all roles with --all."""
+    # Guard against --all + positional role conflict
+    if all_roles and role:
+        click.echo("Error: Cannot specify both --all and a role.")
+        sys.exit(1)
+
+    server = ServerManager()
+
+    if all_roles:
+        results = server.stop_all()
+        for r in results:
+            click.echo(f"  {r['role']}: {r['status']}")
+        return
+
+    if not role:
+        click.echo("Error: Specify a role or use --all.")
+        sys.exit(1)
+
+    stopped = server.stop(role)
+    if stopped:
+        click.echo(f"Stopped {role} server")
+    else:
+        click.echo(f"Role '{role}' server is not running")
+
+
+@cli.command()
+def status():
+    """Show status of all registered roles and their servers."""
+    config = ConfigManager()
+    server = ServerManager()
+
+    roles = config.get_all_roles()
+    if not roles:
+        click.echo("No roles registered. Run 'llm-ctl register <role> <model_id>'")
+        return
+
+    statuses = server.get_all_status()
+
+    click.echo(f"\n{'Role':<12} {'Model':<50} {'Type':<6} {'Port':<6} {'Status':<10} {'PID'}")
+    click.echo("-" * 100)
+    for s in statuses:
+        running = "running" if s["running"] else "stopped"
+        pid = str(s["pid"]) if s["pid"] else "-"
+        model_id = s.get("model_id", "-")
+        if len(model_id) > 48:
+            model_id = "..." + model_id[-45:]
+        click.echo(f"{s['role']:<12} {model_id:<50} {s.get('type', '-'):6} {str(s.get('port', '-')):<6} {running:<10} {pid}")
+
+
+@cli.command()
+@click.argument("model_id")
+@click.option("--cache-dir", default=None, help="Custom HuggingFace cache directory")
+def download(model_id, cache_dir):
+    """Download a model from HuggingFace.
+
+    MODEL_ID: HuggingFace model ID (e.g., mlx-community/Qwen3.6-27B-4bit)
+    """
+    try:
+        from huggingface_hub import snapshot_download
+    except ImportError:
+        click.echo("Error: huggingface-hub is not installed. Run: pip install huggingface-hub")
+        sys.exit(1)
+
+    click.echo(f"Downloading {model_id}...")
+    try:
+        if cache_dir:
+            cachedir = cache_dir
+        else:
+            cachedir = str(Path.home() / "models" / "huggingface" / "hub")
+
+        path = snapshot_download(
+            repo_id=model_id,
+            cache_dir=cachedir,
+        )
+        click.echo(f"Downloaded to: {path}")
+    except Exception as e:
+        click.echo(f"Error downloading model: {e}")
+        sys.exit(1)
+
+
+@cli.command()
+@click.argument("action", required=False, default="show")
+def config(action):
+    """View or edit the configuration file.
+
+    ACTION: show (default) or edit (opens in $EDITOR)
+    """
+    cfg = ConfigManager()
+
+    if action == "show":
+        data = cfg.load()
+        click.echo(json.dumps(data, indent=2))
+    elif action == "edit":
+        editor = os.environ.get("EDITOR", "vi")
+        # Ensure file exists
+        if not cfg.config_path.exists():
+            cfg.save(cfg.load())
+        subprocess.call([editor, str(cfg.config_path)])
+        click.echo(f"Config file: {cfg.config_path}")
+    else:
+        click.echo(f"Unknown action: {action}. Use 'show' or 'edit'.")
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    cli()

local_llm_ctl-0.1.0/src/llm_ctl/config.py

@@ -0,0 +1,82 @@
+"""Configuration management for llm-ctl."""
+
+import json
+import os
+from pathlib import Path
+from typing import Any, Dict, Optional
+
+DEFAULT_CONFIG: Dict[str, Any] = {
+    "roles": {},
+    "defaults": {
+        "base_port": 8080,
+        "models_dirs": [
+            os.path.expanduser("~/models/huggingface/hub"),
+            os.path.expanduser("~/models"),
+        ]
+    }
+}
+
+VALID_ROLES = {"planner", "reviewer", "engineer", "assistant"}
+
+CONFIG_DIR = Path.home() / ".config" / "llm-ctl"
+CONFIG_PATH = CONFIG_DIR / "config.json"
+
+
+class ConfigManager:
+    def __init__(self, config_path: Optional[str] = None):
+        self.config_path = Path(config_path) if config_path else CONFIG_PATH
+
+    def load(self) -> Dict[str, Any]:
+        if not self.config_path.exists():
+            return self._default_config()
+        with open(self.config_path, "r") as f:
+            data = json.load(f)
+        # Merge with defaults to ensure new keys appear
+        merged = json.loads(json.dumps(DEFAULT_CONFIG))
+        merged["roles"].update(data.get("roles", {}))
+        if "defaults" in data:
+            merged["defaults"].update(data["defaults"])
+        return merged
+
+    def save(self, config: Dict[str, Any]) -> None:
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        with open(self.config_path, "w") as f:
+            json.dump(config, f, indent=2)
+            f.write("\n")
+
+    def register_role(self, role: str, role_config: Dict[str, Any]) -> None:
+        if role not in VALID_ROLES:
+            raise ValueError(f"Invalid role '{role}'. Must be one of: {', '.join(sorted(VALID_ROLES))}")
+        config = self.load()
+        config["roles"][role] = role_config
+        self.save(config)
+
+    def unregister_role(self, role: str) -> None:
+        config = self.load()
+        if role in config["roles"]:
+            del config["roles"][role]
+            self.save(config)
+
+    def get_role(self, role: str) -> Optional[Dict[str, Any]]:
+        config = self.load()
+        return config["roles"].get(role)
+
+    def get_all_roles(self) -> Dict[str, Any]:
+        config = self.load()
+        return config["roles"]
+
+    def get_base_port(self) -> int:
+        config = self.load()
+        return config["defaults"].get("base_port", 8080)
+
+    def get_models_dirs(self) -> list:
+        config = self.load()
+        return config["defaults"].get("models_dirs", DEFAULT_CONFIG["defaults"]["models_dirs"])
+
+    @staticmethod
+    def _default_config() -> Dict[str, Any]:
+        return json.loads(json.dumps(DEFAULT_CONFIG))
+
+    @property
+    def path(self) -> Path:
+        return self.config_path