@aladac/hu 0.1.0-a1 → 0.1.0-a2

package/plans/gleaming-crunching-bear.md

@@ -0,0 +1,179 @@
+# Implementation Plan: Local Model Integration
+
+## Overview
+
+Extend the existing `models/` module to support MLX LLM inference (text/vision), HuggingFace model discovery, and LM Studio cache scanning.
+
+**Key Principle**: Extend existing infrastructure, don't replace it.
+
+---
+
+## Phase 1: Dependencies & Cache Schema
+
+### 1.1 Update `pyproject.toml`
+Add new dependencies:
+```toml
+"mlx-lm>=0.20",           # Text LLM generation
+"huggingface_hub>=0.25",  # HuggingFace downloads
+```
+
+### 1.2 Extend `CachedModel` in `src/prompts/models/cache.py`
+Add fields (backward compatible):
+- `hf_repo_id: Optional[str]` - e.g., "mlx-community/Qwen2.5-VL-3B"
+- `hf_revision: Optional[str]` - git revision hash
+- `memory_gb: Optional[float]` - estimated VRAM usage
+
+Update `to_dict()` and `from_dict()` methods.
+
+### 1.3 Update `src/prompts/models/__init__.py`
+Add path constants:
+```python
+LMSTUDIO_MODELS_PATH = Path.home() / ".lmstudio" / "models" / "mlx-community"
+PROMPTS_MLX_CACHE = Path.home() / ".cache" / "prompts" / "mlx"
+```
+
+---
+
+## Phase 2: LM Studio Scanner
+
+### 2.1 Create `src/prompts/models/lmstudio.py`
+New file with:
+- `LMStudioModel` dataclass
+- `scan_lmstudio()` function - iterate MLX model directories
+- Parse `config.json` to detect text vs vision models
+- Calculate directory size for memory estimation
+
+### 2.2 Add `scan` command to `src/prompts/models/cli.py`
+```
+prompts models scan              # Scan all sources
+prompts models scan --lmstudio   # LM Studio only
+prompts models scan --sm         # StabilityMatrix only
+```
+
+---
+
+## Phase 3: HuggingFace Integration
+
+### 3.1 Create `src/prompts/models/huggingface.py`
+New file with:
+- `HFModelInfo` dataclass
+- `search_mlx_models(query)` - search mlx-community namespace
+- `download_mlx_model(repo_id)` - use `snapshot_download()`
+- `check_existing_caches(repo_id)` - check LM Studio/prompts cache first
+
+### 3.2 Extend `search` command in `src/prompts/models/cli.py`
+Add `--provider hf` option to existing search command.
+
+### 3.3 Add `download` command to `src/prompts/models/cli.py`
+```
+prompts models download mlx-community/Qwen2.5-VL-3B
+prompts models download 827184 --provider civitai
+```
+Auto-detect provider from ID format.
+
+---
+
+## Phase 4: MLX Runtime Classes
+
+### 4.1 Create `src/prompts/models/runtime/` directory
+New submodule structure:
+```
+runtime/
+├── __init__.py     # Convenience functions
+├── text.py         # TextLLM singleton class
+└── vision.py       # VisionLLM singleton class
+```
+
+### 4.2 Create `src/prompts/models/runtime/text.py`
+`TextLLM` class:
+- Singleton pattern with thread lock
+- `load(model_path)` - load via `mlx_lm.load()`
+- `generate(prompt)` - generate via `mlx_lm.generate()`
+- `unload()` - free memory with gc.collect()
+
+### 4.3 Create `src/prompts/models/runtime/vision.py`
+`VisionLLM` class:
+- Singleton pattern with thread lock
+- `load(model_path)` - load via `mlx_vlm.load()`
+- `describe(image_path, prompt)` - generate image description
+- `unload()` - free memory
+
+### 4.4 Create `src/prompts/models/runtime/__init__.py`
+Convenience functions:
+- `generate(prompt, model=None)` - text generation
+- `describe_image(image_path, prompt, model=None)` - vision
+- `_resolve_model_path(model_id)` - check caches
+
+---
+
+## Phase 5: Additional CLI Commands
+
+### 5.1 Add `convert` command
+```
+prompts models convert Qwen/Qwen2.5-7B-Instruct --quantize 4bit
+```
+Use `mlx_lm.convert` or `mlx_vlm.convert`.
+
+### 5.2 Add `remove` command
+```
+prompts models remove <model_id>           # Remove from manifest
+prompts models remove <model_id> --delete  # Also delete files
+```
+Skip deletion for external sources (lmstudio, sm).
+
+### 5.3 Add `nuke` command (top-level or under models)
+```
+prompts nuke --cache   # Delete ~/.cache/prompts/
+prompts nuke --config  # Delete ~/.config/prompts/
+prompts nuke --all     # Delete both
+```
+Require confirmation unless `--yes` flag.
+
+---
+
+## Files to Modify
+
+| File | Action |
+|------|--------|
+| `pyproject.toml` | Add mlx-lm, huggingface_hub deps |
+| `src/prompts/models/__init__.py` | Add path constants |
+| `src/prompts/models/cache.py` | Add hf_repo_id, hf_revision, memory_gb fields |
+| `src/prompts/models/cli.py` | Add scan, download, convert, remove commands |
+
+## Files to Create
+
+| File | Purpose |
+|------|---------|
+| `src/prompts/models/lmstudio.py` | LM Studio MLX scanner |
+| `src/prompts/models/huggingface.py` | HuggingFace search/download |
+| `src/prompts/models/runtime/__init__.py` | Convenience functions |
+| `src/prompts/models/runtime/text.py` | TextLLM singleton |
+| `src/prompts/models/runtime/vision.py` | VisionLLM singleton |
+
+---
+
+## Verification
+
+1. **Scan test**: `uv run prompts models scan` - should discover LM Studio models
+2. **Search test**: `uv run prompts models search "qwen" --provider hf` - HuggingFace results
+3. **Download test**: `uv run prompts models download mlx-community/Qwen2.5-0.5B-Instruct-4bit`
+4. **Runtime test**: Python import `from prompts.models.runtime import generate, describe_image`
+
+---
+
+## Deferred (Not in This Implementation)
+
+- **SD Runtime (Diffusers)**: Deferred since ComfyUI handles SD generation via StabilityMatrix
+- **MLX Conversion**: Lower priority, can add later
+- **Unit Tests**: Add after core implementation works
+
+---
+
+## Implementation Order
+
+1. Dependencies (pyproject.toml)
+2. Cache schema extension (cache.py, __init__.py)
+3. LM Studio scanner + scan command
+4. HuggingFace integration + download command
+5. MLX runtime classes
+6. Additional CLI commands (remove, nuke)