modelinfo-cli 1.1.0__tar.gz → 1.2.0__tar.gz

{modelinfo_cli-1.1.0/src/modelinfo_cli.egg-info → modelinfo_cli-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: modelinfo-cli
-Version: 1.1.0
+Version: 1.2.0
 Summary: A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements.
 Author: ModelInfo Contributors
 License: MIT
@@ -29,11 +29,12 @@ It reads binary headers directly using the Python standard library. By bypassing
 ## Features
 
 - **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Seamlessly reads adjacent `config.json` for robust fallback logic.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the 15GB checkpoint. Uses concurrent byte-range requests to pluck the binary headers directly off the CDN in under 2 seconds.
 - **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
-- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Actively warns users if requested context exceeds the model's native limit.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Defaults to 8192 tokens to prevent unrealistic VRAM calculations, while still warning users if the requested context exceeds the model's native limit. Estimates include a standard 600MB CUDA context overhead.
 - **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
 - **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
-- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
 
 ## Installation
 
@@ -67,12 +68,18 @@ pytest tests/ -v
 
 ## Usage
 
-Inspect a model checkpoint:
+Inspect a local model checkpoint:
 
 ```bash
 modelinfo mistral-7b.safetensors
 ```
 
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
 Calculate the memory footprint with a specific KV cache context window:
 
 ```bash

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/README.md

@@ -11,11 +11,12 @@ It reads binary headers directly using the Python standard library. By bypassing
 ## Features
 
 - **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Seamlessly reads adjacent `config.json` for robust fallback logic.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the 15GB checkpoint. Uses concurrent byte-range requests to pluck the binary headers directly off the CDN in under 2 seconds.
 - **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
-- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Actively warns users if requested context exceeds the model's native limit.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Defaults to 8192 tokens to prevent unrealistic VRAM calculations, while still warning users if the requested context exceeds the model's native limit. Estimates include a standard 600MB CUDA context overhead.
 - **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
 - **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
-- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
 
 ## Installation
 
@@ -49,12 +50,18 @@ pytest tests/ -v
 
 ## Usage
 
-Inspect a model checkpoint:
+Inspect a local model checkpoint:
 
 ```bash
 modelinfo mistral-7b.safetensors
 ```
 
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
 Calculate the memory footprint with a specific KV cache context window:
 
 ```bash

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "modelinfo-cli"
-version = "1.1.0"
+version = "1.2.0"
 description = "A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements."
 readme = "README.md"
 requires-python = ">=3.10"

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/src/modelinfo/__init__.py

@@ -2,4 +2,4 @@
 modelinfo - A high-performance CLI utility for inspecting ML model checkpoints.
 """
 
-__version__ = "0.1.0"
+__version__ = "1.2.0"

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/src/modelinfo/architecture.py

@@ -85,8 +85,12 @@ def extract_architecture(tensors: Dict[str, Any], config: Dict[str, Any] = None)
         
     return num_layers, kv_dim, is_estimate
 
-def identify_architecture_name(tensors: Dict[str, Any], num_layers: int) -> str:
-    """Attempt to identify the architecture family based on tensor names or metadata."""
+def identify_architecture_name(tensors: Dict[str, Any], num_layers: int, config: Dict[str, Any] = None) -> str:
+    """Attempt to identify the architecture family based on tensor names, metadata, or config.json."""
+    if config and "architectures" in config and config["architectures"]:
+        arch_title = config["architectures"][0]
+        return f"{arch_title} ({num_layers} layers)" if num_layers else arch_title
+        
     metadata = tensors.get("__metadata__", {})
     gen_arch = metadata.get("general.architecture")
     

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/src/modelinfo/calculator.py

@@ -62,11 +62,15 @@ def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_
     
     primary_dtype = max(dtype_counts.items(), key=lambda x: x[1])[0] if dtype_counts else "Unknown"
     
+    CUDA_CONTEXT_MB = 600
+    overhead_bytes = CUDA_CONTEXT_MB * 1024 * 1024
+    
     return {
         "total_params": total_params,
         "base_memory_bytes": base_memory_bytes,
         "kv_cache_bytes": kv_cache_bytes,
-        "total_memory_bytes": base_memory_bytes + kv_cache_bytes,
+        "overhead_bytes": overhead_bytes,
+        "total_memory_bytes": base_memory_bytes + kv_cache_bytes + overhead_bytes,
         "num_layers": num_layers,
         "kv_dim": kv_dim,
         "primary_dtype": primary_dtype,

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/src/modelinfo/cli.py

@@ -26,7 +26,7 @@ def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
     parser.add_argument(
         "--context",
         type=int,
-        default=0,
+        default=None,
         help="Context length for dynamic KV cache footprint calculation.",
     )
 
@@ -40,7 +40,14 @@ def main(argv: Sequence[str] | None = None) -> int:
     tensors = {}
     config = None
     
-    if file_path.endswith(".safetensors") or file_path.endswith(".index.json"):
+    if not os.path.exists(file_path) and not file_path.endswith((".safetensors", ".gguf", ".pt", ".bin", ".index.json")):
+        from modelinfo.parsers.huggingface import fetch_huggingface_repo
+        try:
+            tensors, config, format_name, disk_size = fetch_huggingface_repo(args.file)
+        except Exception as e:
+            console.print(f"[red]Error fetching from Hugging Face: {e}[/red]")
+            return 1
+    elif file_path.endswith(".safetensors") or file_path.endswith(".index.json"):
         tensors = parse_safetensors_header(args.file)
         format_name = "SafeTensors"
         
@@ -61,24 +68,33 @@ def main(argv: Sequence[str] | None = None) -> int:
         format_name = "PyTorch"
     else:
         console.print(
-            f"[red]Error: Unsupported file format '{args.file}'. Supported formats are .safetensors, .gguf, .pt[/red]"
+            f"[red]Error: File '{args.file}' not found locally and does not appear to be a Hugging Face repository ID.[/red]"
         )
         return 1
         
-    footprint = calculate_footprint(tensors, context_length=args.context, config=config)
-    num_layers = footprint["num_layers"]
-    arch_name = identify_architecture_name(tensors, num_layers)
-    
     max_context = None
     if config:
         max_context = config.get("max_position_embeddings")
-    else:
+    elif format_name == "GGUF":
         metadata = tensors.get("__metadata__", {})
         gen_arch = metadata.get("general.architecture")
         if gen_arch:
             max_context = metadata.get(f"{gen_arch}.context_length")
+            
+    # Determine the actual context length to use for calculation
+    is_default_context = False
+    context_length = args.context
+    if context_length is None:
+        context_length = min(8192, max_context) if max_context else 8192
+        is_default_context = True
 
-    disk_size = os.path.getsize(args.file) if os.path.exists(args.file) else 0.0
+    footprint = calculate_footprint(tensors, context_length=context_length, config=config)
+    num_layers = footprint["num_layers"]
+    arch_name = identify_architecture_name(tensors, num_layers, config)
+
+    if format_name != "SafeTensors" or os.path.exists(args.file):
+        disk_size = os.path.getsize(args.file) if os.path.exists(args.file) else 0.0
+        
     tensor_count = len([k for k in tensors.keys() if k != "__metadata__"])
     
     print_model_info(
@@ -87,7 +103,8 @@ def main(argv: Sequence[str] | None = None) -> int:
         tensor_count=tensor_count,
         footprint=footprint,
         disk_size=disk_size,
-        context_length=args.context,
+        context_length=context_length,
+        is_default_context=is_default_context,
         tensors=tensors,
         max_context=max_context
     )

modelinfo_cli-1.2.0/src/modelinfo/parsers/huggingface.py

@@ -0,0 +1,151 @@
+import concurrent.futures
+import json
+import os
+import struct
+import urllib.error
+import urllib.request
+from typing import Any, Dict, Tuple
+
+def _get_hf_token() -> str | None:
+    token = os.environ.get("HF_TOKEN")
+    if token:
+        return token
+        
+    cache_path = os.path.expanduser("~/.cache/huggingface/token")
+    if os.path.exists(cache_path):
+        try:
+            with open(cache_path, "r", encoding="utf-8") as f:
+                return f.read().strip()
+        except OSError:
+            pass
+            
+    legacy_path = os.path.expanduser("~/.huggingface/token")
+    if os.path.exists(legacy_path):
+        try:
+            with open(legacy_path, "r", encoding="utf-8") as f:
+                return f.read().strip()
+        except OSError:
+            pass
+            
+    return None
+
+def _make_request(url: str, headers: Dict[str, str] = None) -> bytes:
+    if headers is None:
+        headers = {}
+        
+    token = _get_hf_token()
+    if token:
+        headers["Authorization"] = f"Bearer {token}"
+        
+    req = urllib.request.Request(url, headers=headers)
+    try:
+        with urllib.request.urlopen(req, timeout=10) as response:
+            return response.read()
+    except urllib.error.HTTPError as e:
+        if e.code == 401:
+            raise PermissionError(f"🔒 Gated Model or Invalid Token: Please set HF_TOKEN environment variable to access {url}")
+        if e.code == 404:
+            raise FileNotFoundError(f"File not found on Hugging Face Hub: {url}")
+        raise
+
+def _fetch_safetensors_header(repo_id: str, filename: str) -> Dict[str, Any]:
+    url = f"https://huggingface.co/{repo_id}/resolve/main/{filename}"
+    
+    # 1. Fetch the first 500KB in a single roundtrip
+    headers = {"Range": "bytes=0-500000"}
+    try:
+        chunk = _make_request(url, headers=headers)
+    except urllib.error.HTTPError as e:
+        if e.code == 416: # Range Not Satisfiable (file is smaller than 500KB)
+            chunk = _make_request(url)
+        else:
+            raise
+            
+    if len(chunk) < 8:
+        raise ValueError(f"File {filename} is too small to contain a SafeTensors header.")
+        
+    header_size = struct.unpack("<Q", chunk[:8])[0]
+    
+    # 2. Slice locally if it fits
+    if 8 + header_size <= len(chunk):
+        json_bytes = chunk[8:8+header_size]
+    else:
+        # 3. Double-roundtrip only if the header is massive (>500KB)
+        headers = {"Range": f"bytes=8-{8+header_size-1}"}
+        json_bytes = _make_request(url, headers=headers)
+        
+    return json.loads(json_bytes)
+
+def fetch_huggingface_repo(repo_id: str) -> Tuple[Dict[str, Any], Dict[str, Any] | None, str, float]:
+    """
+    Fetches the metadata directly from the Hugging Face Hub over the network.
+    Returns: (tensors, config, format_name, disk_size)
+    """
+    api_url = f"https://huggingface.co/api/models/{repo_id}"
+    try:
+        api_data = json.loads(_make_request(api_url).decode("utf-8"))
+    except urllib.error.HTTPError as e:
+        if e.code == 401:
+            raise PermissionError(f"🔒 Gated Model: Please set HF_TOKEN environment variable to access {repo_id}")
+        raise
+        
+    siblings = api_data.get("siblings", [])
+    filenames = {s["rfilename"] for s in siblings}
+    
+    config = None
+    if "config.json" in filenames:
+        config_url = f"https://huggingface.co/{repo_id}/resolve/main/config.json"
+        config = json.loads(_make_request(config_url).decode("utf-8"))
+        
+    tensors = {}
+    total_size = 0.0
+    
+    if "model.safetensors.index.json" in filenames:
+        # Sharded SafeTensors
+        index_url = f"https://huggingface.co/{repo_id}/resolve/main/model.safetensors.index.json"
+        index_data = json.loads(_make_request(index_url).decode("utf-8"))
+        
+        weight_map = index_data.get("weight_map", {})
+        unique_shards = list(set(weight_map.values()))
+        
+        total_size = index_data.get("metadata", {}).get("total_size", 0.0)
+        
+        def fetch_shard(shard: str):
+            return shard, _fetch_safetensors_header(repo_id, shard)
+            
+        with concurrent.futures.ThreadPoolExecutor(max_workers=min(8, len(unique_shards))) as executor:
+            future_to_shard = {executor.submit(fetch_shard, shard): shard for shard in unique_shards}
+            for future in concurrent.futures.as_completed(future_to_shard):
+                shard, shard_header = future.result()
+                for k, v in shard_header.items():
+                    if k != "__metadata__":
+                        tensors[k] = v
+                        
+        tensors["__metadata__"] = {
+            "missing_shards": 0,
+            "total_shards": len(unique_shards),
+            "is_sharded": True
+        }
+        format_name = "SafeTensors"
+        
+    elif "model.safetensors" in filenames:
+        # Single SafeTensors
+        header = _fetch_safetensors_header(repo_id, "model.safetensors")
+        tensors = header
+        format_name = "SafeTensors"
+        
+        # We don't have total_size from index, so we could get it from Content-Length or just leave it 0
+        req = urllib.request.Request(f"https://huggingface.co/{repo_id}/resolve/main/model.safetensors", method="HEAD")
+        token = _get_hf_token()
+        if token:
+            req.add_header("Authorization", f"Bearer {token}")
+        try:
+            with urllib.request.urlopen(req) as response:
+                total_size = int(response.headers.get("Content-Length", 0))
+        except Exception:
+            pass
+            
+    else:
+        raise ValueError(f"Repository {repo_id} does not contain SafeTensors weights.")
+        
+    return tensors, config, format_name, float(total_size)

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/src/modelinfo/ui.py

@@ -43,6 +43,7 @@ def print_model_info(
     footprint: Dict[str, Any],
     disk_size: float,
     context_length: int,
+    is_default_context: bool,
     tensors: Dict[str, Any],
     max_context: int | None = None
 ) -> None:
@@ -58,21 +59,38 @@ def print_model_info(
         param_text = "[yellow]UNKNOWN (Missing Shards)[/yellow]"
         disk_text = "[yellow]UNKNOWN (Missing Shards)[/yellow]"
         vram_display = "[yellow]UNKNOWN (Missing Shards)[/yellow]"
+    elif footprint["total_memory_bytes"] == 0:
+        param_text = format_params(footprint["total_params"])
+        disk_text = format_bytes(disk_size)
+        vram_display = "[red]Unknown (Missing Tensor Shapes)[/red]"
     else:
         param_text = format_params(footprint["total_params"])
         disk_text = format_bytes(disk_size)
         vram_bytes = footprint["total_memory_bytes"]
         vram_color = "green" if vram_bytes < 8 * 1024**3 else "yellow" if vram_bytes < 16 * 1024**3 else "red"
         
-        vram_text = f"~{format_bytes(vram_bytes)}"
-        if context_length > 0:
-            if footprint.get("kv_is_estimate"):
-                vram_text += f" ({footprint['primary_dtype']}, Estimated KV Cache - Missing Config)"
+        vram_text = f"~{format_bytes(vram_bytes)} Total Minimum Required"
+        vram_display = f"[{vram_color}]{vram_text}[/{vram_color}]\n"
+        
+        weights_bytes = footprint["base_memory_bytes"]
+        vram_display += f"  ├─ Weights:    {format_bytes(weights_bytes)}\n"
+        
+        kv_cache_bytes = footprint["kv_cache_bytes"]
+        
+        if footprint.get("kv_is_estimate"):
+            kv_note = " (Estimated KV Cache - Missing Config)"
+        elif is_default_context:
+            if max_context and max_context > context_length:
+                kv_note = f" (Default {context_length} tokens. Native limit: {max_context:,})"
             else:
-                vram_text += f" ({footprint['primary_dtype']}, KV cache for {context_length} tokens)"
+                kv_note = f" (Default {context_length} tokens)"
         else:
-            vram_text += f" ({footprint['primary_dtype']}, no KV cache)"
-        vram_display = f"[{vram_color}]{vram_text}[/{vram_color}]"
+            kv_note = f" ({context_length} tokens)"
+            
+        vram_display += f"  ├─ KV Cache:   {format_bytes(kv_cache_bytes)}{kv_note}\n"
+        
+        overhead_bytes = footprint.get("overhead_bytes", 600 * 1024 * 1024)
+        vram_display += f"  └─ Overhead:   {format_bytes(overhead_bytes)} (CUDA Context + Activations)"
 
     summary.add_row("Format:", format_name)
     summary.add_row("Architecture:", arch_name)

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0/src/modelinfo_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: modelinfo-cli
-Version: 1.1.0
+Version: 1.2.0
 Summary: A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements.
 Author: ModelInfo Contributors
 License: MIT
@@ -29,11 +29,12 @@ It reads binary headers directly using the Python standard library. By bypassing
 ## Features
 
 - **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Seamlessly reads adjacent `config.json` for robust fallback logic.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the 15GB checkpoint. Uses concurrent byte-range requests to pluck the binary headers directly off the CDN in under 2 seconds.
 - **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
-- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Actively warns users if requested context exceeds the model's native limit.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Defaults to 8192 tokens to prevent unrealistic VRAM calculations, while still warning users if the requested context exceeds the model's native limit. Estimates include a standard 600MB CUDA context overhead.
 - **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
 - **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
-- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
 
 ## Installation
 
@@ -67,12 +68,18 @@ pytest tests/ -v
 
 ## Usage
 
-Inspect a model checkpoint:
+Inspect a local model checkpoint:
 
 ```bash
 modelinfo mistral-7b.safetensors
 ```
 
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
 Calculate the memory footprint with a specific KV cache context window:
 
 ```bash

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/src/modelinfo_cli.egg-info/SOURCES.txt

@@ -10,6 +10,7 @@ src/modelinfo/ui.py
 src/modelinfo/parsers/__init__.py
 src/modelinfo/parsers/base.py
 src/modelinfo/parsers/gguf.py
+src/modelinfo/parsers/huggingface.py
 src/modelinfo/parsers/pytorch.py
 src/modelinfo/parsers/safetensors.py
 src/modelinfo_cli.egg-info/PKG-INFO

{modelinfo_cli-1.1.0 → modelinfo_cli-1.2.0}/tests/test_calculator.py

@@ -99,3 +99,14 @@ def test_kv_cache_is_fp16():
     
     assert footprint["kv_cache_bytes"] == 33554432
     assert footprint["primary_dtype"] == "Q4"
+
+def test_framework_overhead_included():
+    """Verify that CUDA context and activation overhead is correctly included."""
+    tensors = {
+        "model.layers.0.attn.weight": {"shape": [1024, 1024], "dtype": "F16"}
+    }
+    footprint = calculate_footprint(tensors)
+    
+    assert "overhead_bytes" in footprint
+    assert footprint["overhead_bytes"] == 600 * 1024 * 1024
+    assert footprint["total_memory_bytes"] == footprint["base_memory_bytes"] + footprint["kv_cache_bytes"] + footprint["overhead_bytes"]