gpu-memory-guard 0.1.0__tar.gz

gpu_memory_guard-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dmytro Romanov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gpu_memory_guard-0.1.0/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.4
+Name: gpu-memory-guard
+Version: 0.1.0
+Summary: CLI tool to check GPU VRAM before loading AI models
+Author-email: Dmytro Romanov <casteldazur@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/CastelDazur/gpu-memory-guard
+Project-URL: Repository, https://github.com/CastelDazur/gpu-memory-guard
+Project-URL: Issues, https://github.com/CastelDazur/gpu-memory-guard/issues
+Keywords: gpu,vram,memory,ai,llm,cuda,nvidia,oom
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Hardware
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: pynvml
+Requires-Dist: pynvml>=11.0.0; extra == "pynvml"
+Dynamic: license-file
+
+# GPU Memory Guard
+
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![GitHub stars](https://img.shields.io/github/stars/CastelDazur/gpu-memory-guard?style=social)](https://github.com/CastelDazur/gpu-memory-guard/stargazers)
+
+A CLI utility that checks available GPU VRAM before you load AI models. Prevents OOM crashes that force a full system reboot.
+
+## Why?
+
+If you run local inference on consumer GPUs, you know the pain:
+
+| Without gpu-memory-guard | With gpu-memory-guard |
+|---|---|
+| Load 70B model on 24GB card | Check VRAM **before** loading |
+| System freezes, GPU hangs | Get a clear warning in terminal |
+| Force reboot, lose unsaved work | Pick a smaller model or free memory |
+| Repeat next week | Zero OOM crashes |
+
+One command saves you from constant reboots.
+
+## Quick Start
+
+```bash
+git clone https://github.com/CastelDazur/gpu-memory-guard.git
+cd gpu-memory-guard
+pip install -e .
+```
+
+```bash
+# Check current GPU status
+gpu-guard
+
+# Check if an 18GB model fits with 2GB safety buffer
+gpu-guard --model-size 18 --buffer 2
+```
+
+**Example output:**
+
+```
+GPU 0: NVIDIA GeForce RTX 5090
+  Total:     32.00 GB
+  Used:       4.12 GB
+  Available: 27.88 GB
+
+Model size: 18.00 GB (buffer: 2.00 GB)
+Status: OK - model fits with 7.88 GB to spare
+```
+
+## Installation
+
+### From source (recommended)
+
+```bash
+git clone https://github.com/CastelDazur/gpu-memory-guard.git
+cd gpu-memory-guard
+pip install -e .
+```
+
+### Requirements
+
+- Python 3.8+
+- NVIDIA GPU with `nvidia-smi` installed, OR
+- `pynvml` Python package (`pip install pynvml`)
+
+## Usage
+
+### CLI
+
+```bash
+# Basic VRAM check
+gpu-guard
+
+# Check if a model fits (size in GB)
+gpu-guard --model-size 13
+
+# Custom safety buffer (default: 1GB)
+gpu-guard --model-size 18 --buffer 2
+
+# JSON output for scripting
+gpu-guard --model-size 13 --json
+
+# Quiet mode: exit code only (0 = fits, 1 = doesn't)
+gpu-guard --model-size 7 --quiet
+```
+
+### As a Python library
+
+```python
+from gpu_guard import check_vram, can_load_model, get_gpu_info
+
+# Check current VRAM
+gpu_info = get_gpu_info()
+for gpu in gpu_info:
+    print(f"GPU {gpu.device_id}: {gpu.available_memory_gb:.2f}GB available")
+
+# Check if a model fits
+result = can_load_model(model_size_gb=13.0, buffer_gb=2.0)
+if result.fits:
+    print("Safe to load")
+else:
+    print(f"Need {result.shortage_gb:.2f}GB more VRAM")
+```
+
+### Scripting example
+
+```bash
+# Pre-check before launching inference
+if gpu-guard --model-size 13 --quiet; then
+    python run_inference.py --model llama-13b
+else
+    echo "Not enough VRAM, switching to 7B model"
+    python run_inference.py --model llama-7b
+fi
+```
+
+## Common model sizes (approximate VRAM)
+
+| Model | FP16 | Q4 (GGUF) |
+|---|---|---|
+| 7B params | ~14 GB | ~4 GB |
+| 13B params | ~26 GB | ~7 GB |
+| 33B params | ~66 GB | ~18 GB |
+| 70B params | ~140 GB | ~35 GB |
+
+## Roadmap
+
+- [ ] AMD ROCm support
+- [ ] Memory estimation by model architecture
+- [ ] Multi-GPU split recommendations
+- [ ] PyPI package (`pip install gpu-memory-guard`)
+- [ ] Integration with Ollama and vLLM
+
+## Contributing
+
+PRs welcome. If you want to add AMD ROCm support or model-specific memory estimation, open an issue first so we can discuss the approach.
+
+## License
+
+MIT

gpu_memory_guard-0.1.0/README.md

@@ -0,0 +1,140 @@
+# GPU Memory Guard
+
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![GitHub stars](https://img.shields.io/github/stars/CastelDazur/gpu-memory-guard?style=social)](https://github.com/CastelDazur/gpu-memory-guard/stargazers)
+
+A CLI utility that checks available GPU VRAM before you load AI models. Prevents OOM crashes that force a full system reboot.
+
+## Why?
+
+If you run local inference on consumer GPUs, you know the pain:
+
+| Without gpu-memory-guard | With gpu-memory-guard |
+|---|---|
+| Load 70B model on 24GB card | Check VRAM **before** loading |
+| System freezes, GPU hangs | Get a clear warning in terminal |
+| Force reboot, lose unsaved work | Pick a smaller model or free memory |
+| Repeat next week | Zero OOM crashes |
+
+One command saves you from constant reboots.
+
+## Quick Start
+
+```bash
+git clone https://github.com/CastelDazur/gpu-memory-guard.git
+cd gpu-memory-guard
+pip install -e .
+```
+
+```bash
+# Check current GPU status
+gpu-guard
+
+# Check if an 18GB model fits with 2GB safety buffer
+gpu-guard --model-size 18 --buffer 2
+```
+
+**Example output:**
+
+```
+GPU 0: NVIDIA GeForce RTX 5090
+  Total:     32.00 GB
+  Used:       4.12 GB
+  Available: 27.88 GB
+
+Model size: 18.00 GB (buffer: 2.00 GB)
+Status: OK - model fits with 7.88 GB to spare
+```
+
+## Installation
+
+### From source (recommended)
+
+```bash
+git clone https://github.com/CastelDazur/gpu-memory-guard.git
+cd gpu-memory-guard
+pip install -e .
+```
+
+### Requirements
+
+- Python 3.8+
+- NVIDIA GPU with `nvidia-smi` installed, OR
+- `pynvml` Python package (`pip install pynvml`)
+
+## Usage
+
+### CLI
+
+```bash
+# Basic VRAM check
+gpu-guard
+
+# Check if a model fits (size in GB)
+gpu-guard --model-size 13
+
+# Custom safety buffer (default: 1GB)
+gpu-guard --model-size 18 --buffer 2
+
+# JSON output for scripting
+gpu-guard --model-size 13 --json
+
+# Quiet mode: exit code only (0 = fits, 1 = doesn't)
+gpu-guard --model-size 7 --quiet
+```
+
+### As a Python library
+
+```python
+from gpu_guard import check_vram, can_load_model, get_gpu_info
+
+# Check current VRAM
+gpu_info = get_gpu_info()
+for gpu in gpu_info:
+    print(f"GPU {gpu.device_id}: {gpu.available_memory_gb:.2f}GB available")
+
+# Check if a model fits
+result = can_load_model(model_size_gb=13.0, buffer_gb=2.0)
+if result.fits:
+    print("Safe to load")
+else:
+    print(f"Need {result.shortage_gb:.2f}GB more VRAM")
+```
+
+### Scripting example
+
+```bash
+# Pre-check before launching inference
+if gpu-guard --model-size 13 --quiet; then
+    python run_inference.py --model llama-13b
+else
+    echo "Not enough VRAM, switching to 7B model"
+    python run_inference.py --model llama-7b
+fi
+```
+
+## Common model sizes (approximate VRAM)
+
+| Model | FP16 | Q4 (GGUF) |
+|---|---|---|
+| 7B params | ~14 GB | ~4 GB |
+| 13B params | ~26 GB | ~7 GB |
+| 33B params | ~66 GB | ~18 GB |
+| 70B params | ~140 GB | ~35 GB |
+
+## Roadmap
+
+- [ ] AMD ROCm support
+- [ ] Memory estimation by model architecture
+- [ ] Multi-GPU split recommendations
+- [ ] PyPI package (`pip install gpu-memory-guard`)
+- [ ] Integration with Ollama and vLLM
+
+## Contributing
+
+PRs welcome. If you want to add AMD ROCm support or model-specific memory estimation, open an issue first so we can discuss the approach.
+
+## License
+
+MIT

gpu_memory_guard-0.1.0/gpu_guard.py

@@ -0,0 +1,309 @@
+#!/usr/bin/env python3
+"""
+GPU Memory Guard - CLI utility to check VRAM before loading AI models.
+
+Prevents out-of-memory crashes by checking available GPU VRAM and
+estimating whether a model will fit with a safety buffer.
+"""
+
+import json
+import sys
+import argparse
+from dataclasses import dataclass, asdict
+from typing import Optional, List, Tuple
+
+
+@dataclass
+class GPUInfo:
+    """GPU memory information."""
+    device_id: int
+    name: str
+    total_memory_gb: float
+    used_memory_gb: float
+    available_memory_gb: float
+    utilization_percent: float
+
+
+def get_nvidia_smi_info() -> Optional[List[GPUInfo]]:
+    """Query GPU info using nvidia-smi."""
+    import subprocess
+
+    try:
+        cmd = [
+            "nvidia-smi",
+            "--query-gpu=index,name,memory.total,memory.used,memory.free,"
+            "utilization.gpu",
+            "--format=csv,nounits,noheader",
+        ]
+        result = subprocess.run(
+            cmd,
+            capture_output=True,
+            text=True,
+            timeout=5,
+        )
+
+        if result.returncode != 0:
+            return None
+
+        gpus = []
+        for line in result.stdout.strip().split("\n"):
+            if not line.strip():
+                continue
+
+            parts = [p.strip() for p in line.split(",")]
+            if len(parts) < 6:
+                continue
+
+            try:
+                device_id = int(parts[0])
+                name = parts[1]
+                total_mb = float(parts[2])
+                used_mb = float(parts[3])
+                free_mb = float(parts[4])
+                util = float(parts[5])
+
+                gpu = GPUInfo(
+                    device_id=device_id,
+                    name=name,
+                    total_memory_gb=total_mb / 1024.0,
+                    used_memory_gb=used_mb / 1024.0,
+                    available_memory_gb=free_mb / 1024.0,
+                    utilization_percent=util,
+                )
+                gpus.append(gpu)
+            except (ValueError, IndexError):
+                continue
+
+        return gpus if gpus else None
+
+    except (FileNotFoundError, subprocess.TimeoutExpired, Exception):
+        return None
+
+
+def get_pynvml_info() -> Optional[List[GPUInfo]]:
+    """Query GPU info using pynvml."""
+    try:
+        import pynvml
+
+        pynvml.nvmlInit()
+        device_count = pynvml.nvmlDeviceGetCount()
+
+        gpus = []
+        for i in range(device_count):
+            handle = pynvml.nvmlDeviceGetHandleByIndex(i)
+            name = pynvml.nvmlDeviceGetName(handle).decode("utf-8")
+            mem_info = pynvml.nvmlDeviceGetMemoryInfo(handle)
+            util = pynvml.nvmlDeviceGetUtilizationRates(handle)
+
+            gpu = GPUInfo(
+                device_id=i,
+                name=name,
+                total_memory_gb=mem_info.total / (1024**3),
+                used_memory_gb=mem_info.used / (1024**3),
+                available_memory_gb=mem_info.free / (1024**3),
+                utilization_percent=util.gpu,
+            )
+            gpus.append(gpu)
+
+        pynvml.nvmlShutdown()
+        return gpus
+
+    except (ImportError, Exception):
+        return None
+
+
+def get_gpu_info() -> Optional[List[GPUInfo]]:
+    """Get GPU information. Try pynvml first, fall back to nvidia-smi."""
+    info = get_pynvml_info()
+    if info is not None:
+        return info
+
+    info = get_nvidia_smi_info()
+    if info is not None:
+        return info
+
+    return None
+
+
+def check_vram(model_size_gb: float, buffer_gb: float = 0.5) -> Tuple[bool, str]:
+    """
+    Check if sufficient VRAM is available for a model.
+
+    Args:
+        model_size_gb: Model size in GB
+        buffer_gb: Safety buffer in GB (default 0.5)
+
+    Returns:
+        (can_fit, message) tuple
+    """
+    gpu_info = get_gpu_info()
+
+    if gpu_info is None:
+        msg = "Unable to detect GPU. Ensure nvidia-smi or pynvml is available."
+        return (False, msg)
+
+    if not gpu_info:
+        return (False, "No GPUs detected.")
+
+    total_available = sum(gpu.available_memory_gb for gpu in gpu_info)
+    required = model_size_gb + buffer_gb
+
+    can_fit = total_available >= required
+
+    message = f"Total available: {total_available:.2f}GB, required: {required:.2f}GB"
+
+    return (can_fit, message)
+
+
+def can_load_model(model_size_gb: float, buffer_gb: float = 0.5) -> bool:
+    """
+    Check if a model can be loaded without OOM.
+
+    Args:
+        model_size_gb: Model size in GB
+        buffer_gb: Safety buffer in GB
+
+    Returns:
+        True if the model fits, False otherwise
+    """
+    fits, _ = check_vram(model_size_gb, buffer_gb)
+    return fits
+
+
+def format_human_output(gpu_info, model_size_gb=None, buffer_gb=0.5):
+    """Format GPU info for human-readable output."""
+    lines = []
+    lines.append("GPU Memory Status")
+    lines.append("=" * 60)
+
+    total_available = 0
+    for gpu in gpu_info:
+        lines.append(f"\nGPU {gpu.device_id}: {gpu.name}")
+        lines.append(f"  Total:     {gpu.total_memory_gb:>7.2f}GB")
+        lines.append(f"  Used:      {gpu.used_memory_gb:>7.2f}GB")
+        lines.append(f"  Available: {gpu.available_memory_gb:>7.2f}GB")
+        lines.append(f"  Util:      {gpu.utilization_percent:>7.1f}%")
+        total_available += gpu.available_memory_gb
+
+    lines.append("\n" + "-" * 60)
+    lines.append(f"Total available across all GPUs: {total_available:.2f}GB")
+
+    if model_size_gb is not None:
+        required = model_size_gb + buffer_gb
+        lines.append(f"\nModel size:     {model_size_gb:.2f}GB")
+        lines.append(f"Safety buffer:  {buffer_gb:.2f}GB")
+        lines.append(f"Total required: {required:.2f}GB")
+        lines.append("-" * 60)
+
+        if total_available >= required:
+            margin = total_available - required
+            lines.append(f"\u2713 Model WILL fit ({margin:.2f}GB margin)")
+        else:
+            deficit = required - total_available
+            lines.append(f"\u2717 Model will NOT fit (need {deficit:.2f}GB more)")
+
+    return "\n".join(lines)
+
+
+def format_json_output(gpu_info, model_size_gb=None, buffer_gb=0.5):
+    """Format output as JSON."""
+    total_available = sum(gpu.available_memory_gb for gpu in gpu_info)
+
+    output = {
+        "gpus": [asdict(gpu) for gpu in gpu_info],
+        "total_available_gb": total_available,
+    }
+
+    if model_size_gb is not None:
+        required = model_size_gb + buffer_gb
+        output.update({
+            "model_size_gb": model_size_gb,
+            "buffer_gb": buffer_gb,
+            "total_required_gb": required,
+            "can_fit": total_available >= required,
+        })
+
+    return json.dumps(output, indent=2)
+
+
+def main():
+    """CLI entry point."""
+    parser = argparse.ArgumentParser(
+        description="GPU Memory Guard - Check VRAM before loading AI models",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  # Check current GPU status
+  gpu_guard.py
+
+  # Check if a 18GB model fits with 2GB buffer
+  gpu_guard.py --model-size 18 --buffer 2
+
+  # JSON output for scripting
+  gpu_guard.py --model-size 13 --json
+
+  # Minimal output for shell scripts
+  gpu_guard.py --model-size 7 --quiet
+        """,
+    )
+
+    parser.add_argument(
+        "--model-size",
+        type=float,
+        help="Model size in GB to check",
+    )
+    parser.add_argument(
+        "--buffer",
+        type=float,
+        default=0.5,
+        help="Safety buffer in GB (default: 0.5)",
+    )
+    parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output as JSON",
+    )
+    parser.add_argument(
+        "--quiet",
+        action="store_true",
+        help="Minimal output (exit code only)",
+    )
+
+    args = parser.parse_args()
+
+    gpu_info = get_gpu_info()
+
+    if gpu_info is None:
+        if not args.quiet:
+            print("ERROR: Unable to detect GPU.", file=sys.stderr)
+            print(
+                "Ensure nvidia-smi is installed or pynvml is available.",
+                file=sys.stderr,
+            )
+        sys.exit(2)
+
+    if not gpu_info:
+        if not args.quiet:
+            print("ERROR: No GPUs detected.", file=sys.stderr)
+        sys.exit(2)
+
+    if args.quiet and args.model_size is not None:
+        fits, _ = check_vram(args.model_size, args.buffer)
+        sys.exit(0 if fits else 1)
+
+    if args.json:
+        output = format_json_output(gpu_info, args.model_size, args.buffer)
+        print(output)
+    else:
+        output = format_human_output(gpu_info, args.model_size, args.buffer)
+        print(output)
+
+    if args.model_size is not None:
+        fits, _ = check_vram(args.model_size, args.buffer)
+        sys.exit(0 if fits else 1)
+
+    sys.exit(0)
+
+
+if __name__ == "__main__":
+    main()

gpu_memory_guard-0.1.0/gpu_memory_guard.egg-info/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.4
+Name: gpu-memory-guard
+Version: 0.1.0
+Summary: CLI tool to check GPU VRAM before loading AI models
+Author-email: Dmytro Romanov <casteldazur@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/CastelDazur/gpu-memory-guard
+Project-URL: Repository, https://github.com/CastelDazur/gpu-memory-guard
+Project-URL: Issues, https://github.com/CastelDazur/gpu-memory-guard/issues
+Keywords: gpu,vram,memory,ai,llm,cuda,nvidia,oom
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Hardware
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: pynvml
+Requires-Dist: pynvml>=11.0.0; extra == "pynvml"
+Dynamic: license-file
+
+# GPU Memory Guard
+
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![GitHub stars](https://img.shields.io/github/stars/CastelDazur/gpu-memory-guard?style=social)](https://github.com/CastelDazur/gpu-memory-guard/stargazers)
+
+A CLI utility that checks available GPU VRAM before you load AI models. Prevents OOM crashes that force a full system reboot.
+
+## Why?
+
+If you run local inference on consumer GPUs, you know the pain:
+
+| Without gpu-memory-guard | With gpu-memory-guard |
+|---|---|
+| Load 70B model on 24GB card | Check VRAM **before** loading |
+| System freezes, GPU hangs | Get a clear warning in terminal |
+| Force reboot, lose unsaved work | Pick a smaller model or free memory |
+| Repeat next week | Zero OOM crashes |
+
+One command saves you from constant reboots.
+
+## Quick Start
+
+```bash
+git clone https://github.com/CastelDazur/gpu-memory-guard.git
+cd gpu-memory-guard
+pip install -e .
+```
+
+```bash
+# Check current GPU status
+gpu-guard
+
+# Check if an 18GB model fits with 2GB safety buffer
+gpu-guard --model-size 18 --buffer 2
+```
+
+**Example output:**
+
+```
+GPU 0: NVIDIA GeForce RTX 5090
+  Total:     32.00 GB
+  Used:       4.12 GB
+  Available: 27.88 GB
+
+Model size: 18.00 GB (buffer: 2.00 GB)
+Status: OK - model fits with 7.88 GB to spare
+```
+
+## Installation
+
+### From source (recommended)
+
+```bash
+git clone https://github.com/CastelDazur/gpu-memory-guard.git
+cd gpu-memory-guard
+pip install -e .
+```
+
+### Requirements
+
+- Python 3.8+
+- NVIDIA GPU with `nvidia-smi` installed, OR
+- `pynvml` Python package (`pip install pynvml`)
+
+## Usage
+
+### CLI
+
+```bash
+# Basic VRAM check
+gpu-guard
+
+# Check if a model fits (size in GB)
+gpu-guard --model-size 13
+
+# Custom safety buffer (default: 1GB)
+gpu-guard --model-size 18 --buffer 2
+
+# JSON output for scripting
+gpu-guard --model-size 13 --json
+
+# Quiet mode: exit code only (0 = fits, 1 = doesn't)
+gpu-guard --model-size 7 --quiet
+```
+
+### As a Python library
+
+```python
+from gpu_guard import check_vram, can_load_model, get_gpu_info
+
+# Check current VRAM
+gpu_info = get_gpu_info()
+for gpu in gpu_info:
+    print(f"GPU {gpu.device_id}: {gpu.available_memory_gb:.2f}GB available")
+
+# Check if a model fits
+result = can_load_model(model_size_gb=13.0, buffer_gb=2.0)
+if result.fits:
+    print("Safe to load")
+else:
+    print(f"Need {result.shortage_gb:.2f}GB more VRAM")
+```
+
+### Scripting example
+
+```bash
+# Pre-check before launching inference
+if gpu-guard --model-size 13 --quiet; then
+    python run_inference.py --model llama-13b
+else
+    echo "Not enough VRAM, switching to 7B model"
+    python run_inference.py --model llama-7b
+fi
+```
+
+## Common model sizes (approximate VRAM)
+
+| Model | FP16 | Q4 (GGUF) |
+|---|---|---|
+| 7B params | ~14 GB | ~4 GB |
+| 13B params | ~26 GB | ~7 GB |
+| 33B params | ~66 GB | ~18 GB |
+| 70B params | ~140 GB | ~35 GB |
+
+## Roadmap
+
+- [ ] AMD ROCm support
+- [ ] Memory estimation by model architecture
+- [ ] Multi-GPU split recommendations
+- [ ] PyPI package (`pip install gpu-memory-guard`)
+- [ ] Integration with Ollama and vLLM
+
+## Contributing
+
+PRs welcome. If you want to add AMD ROCm support or model-specific memory estimation, open an issue first so we can discuss the approach.
+
+## License
+
+MIT

gpu_memory_guard-0.1.0/gpu_memory_guard.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+LICENSE
+README.md
+gpu_guard.py
+pyproject.toml
+gpu_memory_guard.egg-info/PKG-INFO
+gpu_memory_guard.egg-info/SOURCES.txt
+gpu_memory_guard.egg-info/dependency_links.txt
+gpu_memory_guard.egg-info/entry_points.txt
+gpu_memory_guard.egg-info/requires.txt
+gpu_memory_guard.egg-info/top_level.txt

gpu_memory_guard-0.1.0/gpu_memory_guard.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

gpu_memory_guard-0.1.0/gpu_memory_guard.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+gpu-guard = gpu_guard:main

gpu_memory_guard-0.1.0/gpu_memory_guard.egg-info/requires.txt

@@ -0,0 +1,3 @@
+
+[pynvml]
+pynvml>=11.0.0

gpu_memory_guard-0.1.0/gpu_memory_guard.egg-info/top_level.txt

@@ -0,0 +1 @@
+gpu_guard

gpu_memory_guard-0.1.0/pyproject.toml

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "gpu-memory-guard"
+version = "0.1.0"
+description = "CLI tool to check GPU VRAM before loading AI models"
+readme = "README.md"
+license = {text = "MIT"}
+requires-python = ">=3.8"
+authors = [
+    {name = "Dmytro Romanov", email = "casteldazur@gmail.com"}
+    ]
+    classifiers = [
+        "Development Status :: 3 - Alpha",
+            "Intended Audience :: Developers",
+                "License :: OSI Approved :: MIT License",
+                    "Programming Language :: Python :: 3",
+                        "Programming Language :: Python :: 3.8",
+                            "Programming Language :: Python :: 3.9",
+                                "Programming Language :: Python :: 3.10",
+                                    "Programming Language :: Python :: 3.11",
+                                        "Programming Language :: Python :: 3.12",
+                                            "Topic :: System :: Hardware",
+                                                "Topic :: Scientific/Engineering :: Artificial Intelligence",
+                                                ]
+                                                keywords = ["gpu", "vram", "memory", "ai", "llm", "cuda", "nvidia", "oom"]
+
+                                                [project.optional-dependencies]
+                                                pynvml = ["pynvml>=11.0.0"]
+
+                                                [project.scripts]
+                                                gpu-guard = "gpu_guard:main"
+
+                                                [project.urls]
+                                                Homepage = "https://github.com/CastelDazur/gpu-memory-guard"
+                                                Repository = "https://github.com/CastelDazur/gpu-memory-guard"
+                                                Issues = "https://github.com/CastelDazur/gpu-memory-guard/issues"

gpu_memory_guard-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+