modelinfo-cli 1.2.0__tar.gz → 1.3.0__tar.gz

modelinfo_cli-1.3.0/PKG-INFO

@@ -0,0 +1,171 @@
+Metadata-Version: 2.4
+Name: modelinfo-cli
+Version: 1.3.0
+Summary: A sub-100ms, zero-dependency CLI to inspect ML models (.safetensors, .gguf) locally or via Hugging Face, calculate exact VRAM footprints, and determine hardware fit.
+Author: ModelInfo Contributors
+License: MIT
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: rich>=13.0.0
+Provides-Extra: dev
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Dynamic: license-file
+
+# ModelInfo CLI
+
+![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)
+![Dependencies](https://img.shields.io/badge/dependencies-rich-green.svg)
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+
+ModelInfo is a terminal-native utility that inspects machine learning model checkpoints (`.safetensors`, `.gguf`, `.pt`) and calculates hardware requirements completely offline.
+
+It reads binary headers directly using the Python standard library. By bypassing full tensor payload loading and strictly excluding heavy ecosystems like PyTorch or HuggingFace, the tool executes in under 100 milliseconds.
+
+## Features
+
+- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Reads adjacent `config.json` for architecture fallback.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the checkpoint. Uses concurrent byte-range requests to read the binary headers directly off the CDN in under 2 seconds.
+- **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths.
+- **Hardware Fit Diagnostics**: Pass the `--gpu` flag (e.g. `--gpu RTX4090` or `--gpu auto`) to calculate if the model fits in your specific cluster. Defends against fragmentation OOMs using a 3-tier heuristic (Safe, Warning, Fail), calculates overhead across multi-GPU setups, and enforces Apple Silicon's 75% unified memory wire limit.
+- **Side-by-Side Comparison**: Pass multiple models to automatically trigger a comparison table. Compares parameters, data types, context lengths, and VRAM footprints side-by-side to evaluate trade-offs.
+- **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
+- **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
+
+> [!NOTE]
+> **A Note on Performance & Remote Fetching**
+> Local `.gguf` and `.safetensors` files are parsed in under 100ms. However, querying remote Hugging Face repositories takes **1 to 10 seconds**. This is an intentional trade-off. To remain zero-dependency, `modelinfo` negotiates raw TCP/TLS via Python `urllib` instead of loading PyTorch. For massive sharded models (e.g., 100+ shards), it must fetch every header individually, capped at an 8-worker thread pool to prevent Cloudflare IP bans. Waiting ~8 seconds to map a model is faster than downloading 400GB just to see if it fits your hardware.
+
+## Installation
+
+Install directly from PyPI:
+
+```bash
+pip install modelinfo-cli
+```
+
+### Development
+
+To install from source and run the test suite:
+
+```bash
+git clone https://github.com/pipe1os/modelinfo-cli.git
+cd modelinfo-cli
+python -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+## Testing
+
+The testing suite enforces cross-platform structural integrity and guards the zero-dependency latency constraint. Tests are isolated against custom binary mocks in `tests/fixtures/`.
+
+Run the test suite using pytest:
+
+```bash
+pytest tests/ -v
+```
+
+## Usage
+
+Inspect a local model checkpoint:
+
+```bash
+modelinfo mistral-7b.safetensors
+```
+
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
+For gated models (e.g., Llama 2), you must provide authentication by setting the `HF_TOKEN` environment variable. You can create a token in your [Hugging Face settings](https://huggingface.co/settings/tokens).
+
+```bash
+export HF_TOKEN="hf_your_token_here"
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
+Alternatively, the tool will automatically read tokens stored by the `hf auth login` command (located in `~/.cache/huggingface/token`).
+
+Calculate the memory footprint with a specific KV cache context window:
+
+```bash
+modelinfo mistral-7b.safetensors --context 8192
+```
+
+Adjust the VRAM heat-mapping thresholds for your specific hardware (e.g., an 80GB card):
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf --max-vram 80
+```
+
+Determine if a model fits your specific hardware:
+
+```bash
+modelinfo mistralai/Mistral-7B-v0.1 --gpu "RTX 4090"
+modelinfo mistralai/Mistral-7B-v0.1 --gpu auto
+```
+
+Compare multiple models side-by-side against a hardware target:
+
+```bash
+modelinfo mistralai/Mistral-7B-v0.1 Qwen/Qwen2.5-0.5B --gpu 12
+```
+
+### Example Output (Single Model)
+
+```text
+Format:          SafeTensors
+Architecture:    MistralForCausalLM (32 layers)
+Tensors:         291
+Parameters:      7.2B
+Dtype:           BF16
+Disk size:       13.49 GB
+VRAM (est):      ~15.07 GB Total Minimum Required
+                   ├─ Weights:    13.49 GB
+                   ├─ KV Cache:   1.0 GB (Default 8192 tokens. Native limit: 32,768)
+                   └─ Overhead:   600.0 MB (CUDA Context + Activations)
+Hardware Fit:    ✗ No (Requires 15.07 GB, Hardware has 12.0 GB)
+
+Top Tensors by Size:
+  model.embed_tokens.weight                     [32000 x 4096]   bf16   131.1M params
+  32x model.layers.[N].self_attn.q_proj.weight  [4096 x 4096]    bf16    16.8M params
+```
+
+### Example Output (Comparison)
+
+```text
+Model              Params    Dtype    Context    VRAM        Fits
+Mistral-7B-v0.1    7.2B      BF16     8K         15.07 GB    ✗
+Qwen2.5-0.5B       494.0M    BF16     8K         1.6 GB      ✓
+```
+
+## Command Reference
+
+| Argument | Example | Description |
+| :--- | :--- | :--- |
+| `[files...]` | `modelinfo model.safetensors` | Inspect a single model (local path or Hugging Face repo ID). |
+| `[files...]` | `modelinfo modelA modelB` | Pass multiple files/repos to automatically render a side-by-side comparison table instead of a deep-dive summary. |
+| `--gpu` | `--gpu rtx4090` | Check if the model fits. Accepts GPU names (`rtx4090`, `b200`, `rx7900xtx`), explicit VRAM limits in GB (`--gpu 24`), or local hardware auto-discovery (`--gpu auto`). |
+| `--context` | `--context 32768` | Adjust the target KV cache length. Essential for calculating the dynamic memory footprint of long-context models. Defaults to `8192`. |
+| `--max-vram` | `--max-vram 80` | Adjusts the color-coded heat mapping thresholds (Green/Yellow/Red) in the terminal output to match a specific hardware ceiling. |
+
+## Architecture
+
+The system operates across three modules:
+
+1. **Presentation (`cli.py`, `ui.py`)**: Parses arguments and formats tables via `rich`.
+2. **Parsing Engine (`parsers/`)**: Specialized binary readers (`safetensors.py`, `gguf.py`, `pytorch.py`) strictly confined to standard library operations.
+3. **Math Engine (`calculator.py`)**: Determines total parameter counts, maps data types to byte coefficients, and calculates dynamic memory allocations based on tensor shape heuristics.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

modelinfo_cli-1.3.0/README.md

@@ -0,0 +1,153 @@
+# ModelInfo CLI
+
+![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)
+![Dependencies](https://img.shields.io/badge/dependencies-rich-green.svg)
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+
+ModelInfo is a terminal-native utility that inspects machine learning model checkpoints (`.safetensors`, `.gguf`, `.pt`) and calculates hardware requirements completely offline.
+
+It reads binary headers directly using the Python standard library. By bypassing full tensor payload loading and strictly excluding heavy ecosystems like PyTorch or HuggingFace, the tool executes in under 100 milliseconds.
+
+## Features
+
+- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Reads adjacent `config.json` for architecture fallback.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the checkpoint. Uses concurrent byte-range requests to read the binary headers directly off the CDN in under 2 seconds.
+- **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths.
+- **Hardware Fit Diagnostics**: Pass the `--gpu` flag (e.g. `--gpu RTX4090` or `--gpu auto`) to calculate if the model fits in your specific cluster. Defends against fragmentation OOMs using a 3-tier heuristic (Safe, Warning, Fail), calculates overhead across multi-GPU setups, and enforces Apple Silicon's 75% unified memory wire limit.
+- **Side-by-Side Comparison**: Pass multiple models to automatically trigger a comparison table. Compares parameters, data types, context lengths, and VRAM footprints side-by-side to evaluate trade-offs.
+- **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
+- **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
+
+> [!NOTE]
+> **A Note on Performance & Remote Fetching**
+> Local `.gguf` and `.safetensors` files are parsed in under 100ms. However, querying remote Hugging Face repositories takes **1 to 10 seconds**. This is an intentional trade-off. To remain zero-dependency, `modelinfo` negotiates raw TCP/TLS via Python `urllib` instead of loading PyTorch. For massive sharded models (e.g., 100+ shards), it must fetch every header individually, capped at an 8-worker thread pool to prevent Cloudflare IP bans. Waiting ~8 seconds to map a model is faster than downloading 400GB just to see if it fits your hardware.
+
+## Installation
+
+Install directly from PyPI:
+
+```bash
+pip install modelinfo-cli
+```
+
+### Development
+
+To install from source and run the test suite:
+
+```bash
+git clone https://github.com/pipe1os/modelinfo-cli.git
+cd modelinfo-cli
+python -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+## Testing
+
+The testing suite enforces cross-platform structural integrity and guards the zero-dependency latency constraint. Tests are isolated against custom binary mocks in `tests/fixtures/`.
+
+Run the test suite using pytest:
+
+```bash
+pytest tests/ -v
+```
+
+## Usage
+
+Inspect a local model checkpoint:
+
+```bash
+modelinfo mistral-7b.safetensors
+```
+
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
+For gated models (e.g., Llama 2), you must provide authentication by setting the `HF_TOKEN` environment variable. You can create a token in your [Hugging Face settings](https://huggingface.co/settings/tokens).
+
+```bash
+export HF_TOKEN="hf_your_token_here"
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
+Alternatively, the tool will automatically read tokens stored by the `hf auth login` command (located in `~/.cache/huggingface/token`).
+
+Calculate the memory footprint with a specific KV cache context window:
+
+```bash
+modelinfo mistral-7b.safetensors --context 8192
+```
+
+Adjust the VRAM heat-mapping thresholds for your specific hardware (e.g., an 80GB card):
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf --max-vram 80
+```
+
+Determine if a model fits your specific hardware:
+
+```bash
+modelinfo mistralai/Mistral-7B-v0.1 --gpu "RTX 4090"
+modelinfo mistralai/Mistral-7B-v0.1 --gpu auto
+```
+
+Compare multiple models side-by-side against a hardware target:
+
+```bash
+modelinfo mistralai/Mistral-7B-v0.1 Qwen/Qwen2.5-0.5B --gpu 12
+```
+
+### Example Output (Single Model)
+
+```text
+Format:          SafeTensors
+Architecture:    MistralForCausalLM (32 layers)
+Tensors:         291
+Parameters:      7.2B
+Dtype:           BF16
+Disk size:       13.49 GB
+VRAM (est):      ~15.07 GB Total Minimum Required
+                   ├─ Weights:    13.49 GB
+                   ├─ KV Cache:   1.0 GB (Default 8192 tokens. Native limit: 32,768)
+                   └─ Overhead:   600.0 MB (CUDA Context + Activations)
+Hardware Fit:    ✗ No (Requires 15.07 GB, Hardware has 12.0 GB)
+
+Top Tensors by Size:
+  model.embed_tokens.weight                     [32000 x 4096]   bf16   131.1M params
+  32x model.layers.[N].self_attn.q_proj.weight  [4096 x 4096]    bf16    16.8M params
+```
+
+### Example Output (Comparison)
+
+```text
+Model              Params    Dtype    Context    VRAM        Fits
+Mistral-7B-v0.1    7.2B      BF16     8K         15.07 GB    ✗
+Qwen2.5-0.5B       494.0M    BF16     8K         1.6 GB      ✓
+```
+
+## Command Reference
+
+| Argument | Example | Description |
+| :--- | :--- | :--- |
+| `[files...]` | `modelinfo model.safetensors` | Inspect a single model (local path or Hugging Face repo ID). |
+| `[files...]` | `modelinfo modelA modelB` | Pass multiple files/repos to automatically render a side-by-side comparison table instead of a deep-dive summary. |
+| `--gpu` | `--gpu rtx4090` | Check if the model fits. Accepts GPU names (`rtx4090`, `b200`, `rx7900xtx`), explicit VRAM limits in GB (`--gpu 24`), or local hardware auto-discovery (`--gpu auto`). |
+| `--context` | `--context 32768` | Adjust the target KV cache length. Essential for calculating the dynamic memory footprint of long-context models. Defaults to `8192`. |
+| `--max-vram` | `--max-vram 80` | Adjusts the color-coded heat mapping thresholds (Green/Yellow/Red) in the terminal output to match a specific hardware ceiling. |
+
+## Architecture
+
+The system operates across three modules:
+
+1. **Presentation (`cli.py`, `ui.py`)**: Parses arguments and formats tables via `rich`.
+2. **Parsing Engine (`parsers/`)**: Specialized binary readers (`safetensors.py`, `gguf.py`, `pytorch.py`) strictly confined to standard library operations.
+3. **Math Engine (`calculator.py`)**: Determines total parameter counts, maps data types to byte coefficients, and calculates dynamic memory allocations based on tensor shape heuristics.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

{modelinfo_cli-1.2.0 → modelinfo_cli-1.3.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "modelinfo-cli"
-version = "1.2.0"
-description = "A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements."
+version = "1.3.0"
+description = "A sub-100ms, zero-dependency CLI to inspect ML models (.safetensors, .gguf) locally or via Hugging Face, calculate exact VRAM footprints, and determine hardware fit."
 readme = "README.md"
 requires-python = ">=3.10"
 license = { text = "MIT" }
@@ -30,4 +30,4 @@ modelinfo = "modelinfo.cli:main"
 
 [tool.ruff]
 line-length = 88
-target-version = "py310"
+target-version = "py310"

{modelinfo_cli-1.2.0 → modelinfo_cli-1.3.0}/src/modelinfo/__init__.py

@@ -2,4 +2,4 @@
 modelinfo - A high-performance CLI utility for inspecting ML model checkpoints.
 """
 
-__version__ = "1.2.0"
+__version__ = "1.3.0"

{modelinfo_cli-1.2.0 → modelinfo_cli-1.3.0}/src/modelinfo/calculator.py

@@ -29,7 +29,7 @@ def _get_bytes_per_param(dtype: str) -> float:
     """Return the size in bytes for a given data type."""
     return DTYPE_BYTES.get(dtype.upper(), 2.0)
 
-def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_size: int = 1, config: Dict[str, Any] = None) -> Dict[str, Any]:
+def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_size: int = 1, config: Dict[str, Any] = None, gpu_count: int = 1) -> Dict[str, Any]:
     """
     Calculate the memory footprint of a model based on its tensors and context length.
     """
@@ -62,7 +62,7 @@ def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_
     
     primary_dtype = max(dtype_counts.items(), key=lambda x: x[1])[0] if dtype_counts else "Unknown"
     
-    CUDA_CONTEXT_MB = 600
+    CUDA_CONTEXT_MB = 600 * gpu_count
     overhead_bytes = CUDA_CONTEXT_MB * 1024 * 1024
     
     return {

modelinfo_cli-1.3.0/src/modelinfo/cli.py

@@ -0,0 +1,156 @@
+import argparse
+import json
+import os
+import sys
+from typing import Sequence
+
+from modelinfo.architecture import identify_architecture_name
+from modelinfo.calculator import calculate_footprint
+from modelinfo.parsers.gguf import parse_gguf_header
+from modelinfo.parsers.pytorch import parse_pytorch_header
+from modelinfo.parsers.safetensors import parse_safetensors_header
+from modelinfo.ui import console, print_model_info, print_compare_info
+
+
+def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        prog="modelinfo",
+        description="High-performance CLI utility to inspect ML model checkpoints and calculate VRAM requirements.",
+    )
+    
+    parser.add_argument(
+        "file",
+        type=str,
+        nargs="+",
+        help="Path to the model checkpoint file(s) or Hugging Face repository IDs.",
+    )
+    parser.add_argument(
+        "--context",
+        type=int,
+        default=None,
+        help="Context length for dynamic KV cache footprint calculation.",
+    )
+    parser.add_argument(
+        "--max-vram",
+        type=float,
+        default=8.0,
+        help="Maximum VRAM in GB for color-coding thresholds.",
+    )
+    parser.add_argument(
+        "--gpu",
+        type=str,
+        default=None,
+        help="Target GPU hardware (e.g. 'RTX4090' or 'auto') to check if the model fits.",
+    )
+
+    return parser.parse_args(argv)
+
+
+def analyze_model(file_path: str, context_override: int | None, gpu_count: int = 1) -> dict:
+    tensors = {}
+    config = None
+    disk_size = 0.0
+    
+    file_path_lower = file_path.lower()
+    
+    if not os.path.exists(file_path) and not file_path_lower.endswith((".safetensors", ".gguf", ".pt", ".bin", ".index.json")):
+        from modelinfo.parsers.huggingface import fetch_huggingface_repo
+        tensors, config, format_name, disk_size = fetch_huggingface_repo(file_path)
+    elif file_path_lower.endswith(".safetensors") or file_path_lower.endswith(".index.json"):
+        tensors = parse_safetensors_header(file_path)
+        format_name = "SafeTensors"
+        
+        config_path = os.path.join(os.path.dirname(file_path), "config.json")
+        if os.path.exists(config_path):
+            try:
+                with open(config_path, "r", encoding="utf-8") as f:
+                    config = json.load(f)
+            except (json.JSONDecodeError, OSError):
+                pass
+                
+    elif file_path_lower.endswith(".gguf"):
+        tensors = parse_gguf_header(file_path)
+        format_name = "GGUF"
+    elif file_path_lower.endswith(".pt") or file_path_lower.endswith(".bin"):
+        tensors = parse_pytorch_header(file_path)
+        format_name = "PyTorch"
+    else:
+        raise ValueError(f"File '{file_path}' not found locally and does not appear to be a Hugging Face repository ID.")
+        
+    max_context = None
+    if config:
+        max_context = config.get("max_position_embeddings")
+    elif format_name == "GGUF":
+        metadata = tensors.get("__metadata__", {})
+        gen_arch = metadata.get("general.architecture")
+        if gen_arch:
+            max_context = metadata.get(f"{gen_arch}.context_length")
+            
+    is_default_context = False
+    context_length = context_override
+    if context_length is None:
+        context_length = min(8192, max_context) if max_context else 8192
+        is_default_context = True
+
+    footprint = calculate_footprint(tensors, context_length=context_length, config=config, gpu_count=gpu_count)
+    num_layers = footprint["num_layers"]
+    arch_name = identify_architecture_name(tensors, num_layers, config)
+
+    if format_name != "SafeTensors" or os.path.exists(file_path):
+        disk_size = os.path.getsize(file_path) if os.path.exists(file_path) else 0.0
+        
+    tensor_count = len([k for k in tensors.keys() if k != "__metadata__"])
+    
+    return {
+        "format_name": format_name,
+        "arch_name": arch_name,
+        "tensor_count": tensor_count,
+        "footprint": footprint,
+        "disk_size": disk_size,
+        "context_length": context_length,
+        "is_default_context": is_default_context,
+        "tensors": tensors,
+        "max_context": max_context
+    }
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    args = parse_args(argv)
+
+    gpu_name_display = None
+    gpu_count = 1
+    if args.gpu:
+        from modelinfo.hardware import resolve_gpu
+        try:
+            gpu_name_display, args.max_vram, gpu_count = resolve_gpu(args.gpu)
+        except Exception as e:
+            console.print(f"[red]{e}[/red]")
+            return 1
+
+    if len(args.file) > 1:
+        models = []
+        for model_path in args.file:
+            try:
+                info = analyze_model(model_path, args.context, gpu_count)
+                models.append((model_path.split("/")[-1], info))
+            except Exception as e:
+                console.print(f"[red]Error analyzing model '{model_path}': {e}[/red]")
+                return 1
+            
+        print_compare_info(models, args.max_vram, gpu_name=gpu_name_display)
+        return 0
+        
+    file_path = args.file[0]
+    
+    try:
+        info = analyze_model(file_path, args.context, gpu_count)
+    except Exception as e:
+        console.print(f"[red]Error: {e}[/red]")
+        return 1
+
+    print_model_info(**info, max_vram_gb=args.max_vram, gpu_name=gpu_name_display)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())