modelinfo-cli 1.0.0__tar.gz → 1.2.0__tar.gz

{modelinfo_cli-1.0.0/src/modelinfo_cli.egg-info → modelinfo_cli-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: modelinfo-cli
-Version: 1.0.0
+Version: 1.2.0
 Summary: A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements.
 Author: ModelInfo Contributors
 License: MIT
@@ -28,12 +28,13 @@ It reads binary headers directly using the Python standard library. By bypassing
 
 ## Features
 
-- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`.
+- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Seamlessly reads adjacent `config.json` for robust fallback logic.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the 15GB checkpoint. Uses concurrent byte-range requests to pluck the binary headers directly off the CDN in under 2 seconds.
 - **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
-- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Defaults to 8192 tokens to prevent unrealistic VRAM calculations, while still warning users if the requested context exceeds the model's native limit. Estimates include a standard 600MB CUDA context overhead.
 - **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
 - **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
-- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
 
 ## Installation
 
@@ -67,12 +68,18 @@ pytest tests/ -v
 
 ## Usage
 
-Inspect a model checkpoint:
+Inspect a local model checkpoint:
 
 ```bash
 modelinfo mistral-7b.safetensors
 ```
 
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
 Calculate the memory footprint with a specific KV cache context window:
 
 ```bash

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/README.md

@@ -10,12 +10,13 @@ It reads binary headers directly using the Python standard library. By bypassing
 
 ## Features
 
-- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`.
+- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Seamlessly reads adjacent `config.json` for robust fallback logic.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the 15GB checkpoint. Uses concurrent byte-range requests to pluck the binary headers directly off the CDN in under 2 seconds.
 - **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
-- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Defaults to 8192 tokens to prevent unrealistic VRAM calculations, while still warning users if the requested context exceeds the model's native limit. Estimates include a standard 600MB CUDA context overhead.
 - **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
 - **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
-- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
 
 ## Installation
 
@@ -49,12 +50,18 @@ pytest tests/ -v
 
 ## Usage
 
-Inspect a model checkpoint:
+Inspect a local model checkpoint:
 
 ```bash
 modelinfo mistral-7b.safetensors
 ```
 
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
 Calculate the memory footprint with a specific KV cache context window:
 
 ```bash

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "modelinfo-cli"
-version = "1.0.0"
+version = "1.2.0"
 description = "A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements."
 readme = "README.md"
 requires-python = ">=3.10"

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/src/modelinfo/__init__.py

@@ -2,4 +2,4 @@
 modelinfo - A high-performance CLI utility for inspecting ML model checkpoints.
 """
 
-__version__ = "0.1.0"
+__version__ = "1.2.0"

modelinfo_cli-1.2.0/src/modelinfo/architecture.py

@@ -0,0 +1,113 @@
+import os
+import json
+from typing import Any, Dict, Tuple
+
+def extract_architecture(tensors: Dict[str, Any], config: Dict[str, Any] = None) -> Tuple[int, int, bool]:
+    """
+    Extracts the number of layers and KV cache dimension (kv_heads * head_dim).
+    Returns (num_layers, kv_dim, is_estimate).
+    """
+    num_layers = 0
+    kv_dim = 0
+    is_estimate = False
+    
+    metadata = tensors.get("__metadata__", {})
+    gen_arch = metadata.get("general.architecture")
+    
+    # 1. Attempt explicit GGUF metadata
+    if gen_arch:
+        arch_str = str(gen_arch)
+        num_layers = metadata.get(f"{arch_str}.block_count", 0)
+        kv_heads = metadata.get(f"{arch_str}.attention.head_count_kv", 0)
+        
+        key_length = metadata.get(f"{arch_str}.attention.key_length")
+        if not key_length:
+            embed_len = metadata.get(f"{arch_str}.embedding_length", 0)
+            q_heads = metadata.get(f"{arch_str}.attention.head_count", 1)
+            if q_heads > 0:
+                key_length = embed_len // q_heads
+            else:
+                key_length = 0
+                
+        if kv_heads > 0 and key_length > 0:
+            kv_dim = kv_heads * key_length
+            if num_layers > 0:
+                return num_layers, kv_dim, False
+
+    # 2. Attempt explicit SafeTensors config.json
+    if config:
+        num_layers = config.get("num_hidden_layers", 0)
+        num_attention_heads = config.get("num_attention_heads", 1)
+        num_key_value_heads = config.get("num_key_value_heads", num_attention_heads)
+        hidden_size = config.get("hidden_size", 0)
+        
+        if num_attention_heads > 0:
+            head_dim = hidden_size // num_attention_heads
+            kv_dim = num_key_value_heads * head_dim
+            if num_layers > 0 and kv_dim > 0:
+                return num_layers, kv_dim, False
+
+    # 3. Fallback to shape guessing
+    layers_set = set()
+    found_fused = False
+    found_k_proj = False
+    
+    for name, meta in tensors.items():
+        if name == "__metadata__":
+            continue
+            
+        parts = name.split(".")
+        if "layers" in parts:
+            idx = parts.index("layers")
+            if len(parts) > idx + 1 and parts[idx+1].isdigit():
+                layers_set.add(int(parts[idx+1]))
+        elif "h" in parts:
+            idx = parts.index("h")
+            if len(parts) > idx + 1 and parts[idx+1].isdigit():
+                layers_set.add(int(parts[idx+1]))
+
+        if name.endswith("k_proj.weight") or name.endswith("attn.k.weight") or name.endswith("k_proj.w"):
+            found_k_proj = True
+            shape = meta.get("shape", [])
+            if len(shape) >= 2:
+                kv_dim = shape[0]
+                
+        if "qkv_proj.weight" in name or "c_attn.weight" in name:
+            found_fused = True
+            if not found_k_proj:
+                shape = meta.get("shape", [])
+                if len(shape) >= 2:
+                    kv_dim = shape[0] // 3
+
+    num_layers = len(layers_set)
+    if found_fused and not found_k_proj and kv_dim > 0:
+        is_estimate = True
+        
+    return num_layers, kv_dim, is_estimate
+
+def identify_architecture_name(tensors: Dict[str, Any], num_layers: int, config: Dict[str, Any] = None) -> str:
+    """Attempt to identify the architecture family based on tensor names, metadata, or config.json."""
+    if config and "architectures" in config and config["architectures"]:
+        arch_title = config["architectures"][0]
+        return f"{arch_title} ({num_layers} layers)" if num_layers else arch_title
+        
+    metadata = tensors.get("__metadata__", {})
+    gen_arch = metadata.get("general.architecture")
+    
+    if gen_arch:
+        arch_title = str(gen_arch).title()
+        return f"{arch_title} ({num_layers} transformer layers)" if num_layers else arch_title
+        
+    for name in tensors.keys():
+        if name == "__metadata__":
+            continue
+            
+        name_lower = name.lower()
+        if "llama" in name_lower:
+            return f"Llama ({num_layers} transformer layers)" if num_layers else "Llama"
+        if "mistral" in name_lower:
+            return f"Mistral ({num_layers} transformer layers)" if num_layers else "Mistral"
+        if "qwen" in name_lower:
+            return f"Qwen ({num_layers} transformer layers)" if num_layers else "Qwen"
+        
+    return f"Generic Transformer ({num_layers} layers)" if num_layers > 0 else "Unknown Architecture"

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/src/modelinfo/calculator.py

@@ -29,7 +29,7 @@ def _get_bytes_per_param(dtype: str) -> float:
     """Return the size in bytes for a given data type."""
     return DTYPE_BYTES.get(dtype.upper(), 2.0)
 
-def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_size: int = 1) -> Dict[str, Any]:
+def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_size: int = 1, config: Dict[str, Any] = None) -> Dict[str, Any]:
     """
     Calculate the memory footprint of a model based on its tensors and context length.
     """
@@ -54,7 +54,7 @@ def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_
         bytes_per_param = _get_bytes_per_param(dtype)
         base_memory_bytes += param_count * bytes_per_param
         
-    num_layers, kv_dim = extract_architecture(tensors)
+    num_layers, kv_dim, is_estimate = extract_architecture(tensors, config)
     
     # Formula: 2 * Layers * (KV_Heads * Head_Dim) * Context_Length * Batch_Size * Bytes_per_param
     # Assume FP16 (2 bytes) for KV cache
@@ -62,14 +62,19 @@ def calculate_footprint(tensors: Dict[str, Any], context_length: int = 0, batch_
     
     primary_dtype = max(dtype_counts.items(), key=lambda x: x[1])[0] if dtype_counts else "Unknown"
     
+    CUDA_CONTEXT_MB = 600
+    overhead_bytes = CUDA_CONTEXT_MB * 1024 * 1024
+    
     return {
         "total_params": total_params,
         "base_memory_bytes": base_memory_bytes,
         "kv_cache_bytes": kv_cache_bytes,
-        "total_memory_bytes": base_memory_bytes + kv_cache_bytes,
+        "overhead_bytes": overhead_bytes,
+        "total_memory_bytes": base_memory_bytes + kv_cache_bytes + overhead_bytes,
         "num_layers": num_layers,
         "kv_dim": kv_dim,
-        "primary_dtype": primary_dtype
+        "primary_dtype": primary_dtype,
+        "kv_is_estimate": is_estimate
     }
 
 def format_bytes(size_bytes: float) -> str:

modelinfo_cli-1.2.0/src/modelinfo/cli.py

@@ -0,0 +1,116 @@
+import argparse
+import json
+import os
+import sys
+from typing import Sequence
+
+from modelinfo.architecture import identify_architecture_name
+from modelinfo.calculator import calculate_footprint
+from modelinfo.parsers.gguf import parse_gguf_header
+from modelinfo.parsers.pytorch import parse_pytorch_header
+from modelinfo.parsers.safetensors import parse_safetensors_header
+from modelinfo.ui import console, print_model_info
+
+
+def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        prog="modelinfo",
+        description="High-performance CLI utility to inspect ML model checkpoints and calculate VRAM requirements.",
+    )
+    
+    parser.add_argument(
+        "file",
+        type=str,
+        help="Path to the model checkpoint file (.safetensors, .gguf, .pt)",
+    )
+    parser.add_argument(
+        "--context",
+        type=int,
+        default=None,
+        help="Context length for dynamic KV cache footprint calculation.",
+    )
+
+    return parser.parse_args(argv)
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    args = parse_args(argv)
+
+    file_path = args.file.lower()
+    tensors = {}
+    config = None
+    
+    if not os.path.exists(file_path) and not file_path.endswith((".safetensors", ".gguf", ".pt", ".bin", ".index.json")):
+        from modelinfo.parsers.huggingface import fetch_huggingface_repo
+        try:
+            tensors, config, format_name, disk_size = fetch_huggingface_repo(args.file)
+        except Exception as e:
+            console.print(f"[red]Error fetching from Hugging Face: {e}[/red]")
+            return 1
+    elif file_path.endswith(".safetensors") or file_path.endswith(".index.json"):
+        tensors = parse_safetensors_header(args.file)
+        format_name = "SafeTensors"
+        
+        # Read config.json to maintain pure math engines
+        config_path = os.path.join(os.path.dirname(args.file), "config.json")
+        if os.path.exists(config_path):
+            try:
+                with open(config_path, "r", encoding="utf-8") as f:
+                    config = json.load(f)
+            except (json.JSONDecodeError, OSError):
+                pass
+                
+    elif file_path.endswith(".gguf"):
+        tensors = parse_gguf_header(args.file)
+        format_name = "GGUF"
+    elif file_path.endswith(".pt") or file_path.endswith(".bin"):
+        tensors = parse_pytorch_header(args.file)
+        format_name = "PyTorch"
+    else:
+        console.print(
+            f"[red]Error: File '{args.file}' not found locally and does not appear to be a Hugging Face repository ID.[/red]"
+        )
+        return 1
+        
+    max_context = None
+    if config:
+        max_context = config.get("max_position_embeddings")
+    elif format_name == "GGUF":
+        metadata = tensors.get("__metadata__", {})
+        gen_arch = metadata.get("general.architecture")
+        if gen_arch:
+            max_context = metadata.get(f"{gen_arch}.context_length")
+            
+    # Determine the actual context length to use for calculation
+    is_default_context = False
+    context_length = args.context
+    if context_length is None:
+        context_length = min(8192, max_context) if max_context else 8192
+        is_default_context = True
+
+    footprint = calculate_footprint(tensors, context_length=context_length, config=config)
+    num_layers = footprint["num_layers"]
+    arch_name = identify_architecture_name(tensors, num_layers, config)
+
+    if format_name != "SafeTensors" or os.path.exists(args.file):
+        disk_size = os.path.getsize(args.file) if os.path.exists(args.file) else 0.0
+        
+    tensor_count = len([k for k in tensors.keys() if k != "__metadata__"])
+    
+    print_model_info(
+        format_name=format_name,
+        arch_name=arch_name,
+        tensor_count=tensor_count,
+        footprint=footprint,
+        disk_size=disk_size,
+        context_length=context_length,
+        is_default_context=is_default_context,
+        tensors=tensors,
+        max_context=max_context
+    )
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

modelinfo_cli-1.2.0/src/modelinfo/parsers/huggingface.py

@@ -0,0 +1,151 @@
+import concurrent.futures
+import json
+import os
+import struct
+import urllib.error
+import urllib.request
+from typing import Any, Dict, Tuple
+
+def _get_hf_token() -> str | None:
+    token = os.environ.get("HF_TOKEN")
+    if token:
+        return token
+        
+    cache_path = os.path.expanduser("~/.cache/huggingface/token")
+    if os.path.exists(cache_path):
+        try:
+            with open(cache_path, "r", encoding="utf-8") as f:
+                return f.read().strip()
+        except OSError:
+            pass
+            
+    legacy_path = os.path.expanduser("~/.huggingface/token")
+    if os.path.exists(legacy_path):
+        try:
+            with open(legacy_path, "r", encoding="utf-8") as f:
+                return f.read().strip()
+        except OSError:
+            pass
+            
+    return None
+
+def _make_request(url: str, headers: Dict[str, str] = None) -> bytes:
+    if headers is None:
+        headers = {}
+        
+    token = _get_hf_token()
+    if token:
+        headers["Authorization"] = f"Bearer {token}"
+        
+    req = urllib.request.Request(url, headers=headers)
+    try:
+        with urllib.request.urlopen(req, timeout=10) as response:
+            return response.read()
+    except urllib.error.HTTPError as e:
+        if e.code == 401:
+            raise PermissionError(f"🔒 Gated Model or Invalid Token: Please set HF_TOKEN environment variable to access {url}")
+        if e.code == 404:
+            raise FileNotFoundError(f"File not found on Hugging Face Hub: {url}")
+        raise
+
+def _fetch_safetensors_header(repo_id: str, filename: str) -> Dict[str, Any]:
+    url = f"https://huggingface.co/{repo_id}/resolve/main/{filename}"
+    
+    # 1. Fetch the first 500KB in a single roundtrip
+    headers = {"Range": "bytes=0-500000"}
+    try:
+        chunk = _make_request(url, headers=headers)
+    except urllib.error.HTTPError as e:
+        if e.code == 416: # Range Not Satisfiable (file is smaller than 500KB)
+            chunk = _make_request(url)
+        else:
+            raise
+            
+    if len(chunk) < 8:
+        raise ValueError(f"File {filename} is too small to contain a SafeTensors header.")
+        
+    header_size = struct.unpack("<Q", chunk[:8])[0]
+    
+    # 2. Slice locally if it fits
+    if 8 + header_size <= len(chunk):
+        json_bytes = chunk[8:8+header_size]
+    else:
+        # 3. Double-roundtrip only if the header is massive (>500KB)
+        headers = {"Range": f"bytes=8-{8+header_size-1}"}
+        json_bytes = _make_request(url, headers=headers)
+        
+    return json.loads(json_bytes)
+
+def fetch_huggingface_repo(repo_id: str) -> Tuple[Dict[str, Any], Dict[str, Any] | None, str, float]:
+    """
+    Fetches the metadata directly from the Hugging Face Hub over the network.
+    Returns: (tensors, config, format_name, disk_size)
+    """
+    api_url = f"https://huggingface.co/api/models/{repo_id}"
+    try:
+        api_data = json.loads(_make_request(api_url).decode("utf-8"))
+    except urllib.error.HTTPError as e:
+        if e.code == 401:
+            raise PermissionError(f"🔒 Gated Model: Please set HF_TOKEN environment variable to access {repo_id}")
+        raise
+        
+    siblings = api_data.get("siblings", [])
+    filenames = {s["rfilename"] for s in siblings}
+    
+    config = None
+    if "config.json" in filenames:
+        config_url = f"https://huggingface.co/{repo_id}/resolve/main/config.json"
+        config = json.loads(_make_request(config_url).decode("utf-8"))
+        
+    tensors = {}
+    total_size = 0.0
+    
+    if "model.safetensors.index.json" in filenames:
+        # Sharded SafeTensors
+        index_url = f"https://huggingface.co/{repo_id}/resolve/main/model.safetensors.index.json"
+        index_data = json.loads(_make_request(index_url).decode("utf-8"))
+        
+        weight_map = index_data.get("weight_map", {})
+        unique_shards = list(set(weight_map.values()))
+        
+        total_size = index_data.get("metadata", {}).get("total_size", 0.0)
+        
+        def fetch_shard(shard: str):
+            return shard, _fetch_safetensors_header(repo_id, shard)
+            
+        with concurrent.futures.ThreadPoolExecutor(max_workers=min(8, len(unique_shards))) as executor:
+            future_to_shard = {executor.submit(fetch_shard, shard): shard for shard in unique_shards}
+            for future in concurrent.futures.as_completed(future_to_shard):
+                shard, shard_header = future.result()
+                for k, v in shard_header.items():
+                    if k != "__metadata__":
+                        tensors[k] = v
+                        
+        tensors["__metadata__"] = {
+            "missing_shards": 0,
+            "total_shards": len(unique_shards),
+            "is_sharded": True
+        }
+        format_name = "SafeTensors"
+        
+    elif "model.safetensors" in filenames:
+        # Single SafeTensors
+        header = _fetch_safetensors_header(repo_id, "model.safetensors")
+        tensors = header
+        format_name = "SafeTensors"
+        
+        # We don't have total_size from index, so we could get it from Content-Length or just leave it 0
+        req = urllib.request.Request(f"https://huggingface.co/{repo_id}/resolve/main/model.safetensors", method="HEAD")
+        token = _get_hf_token()
+        if token:
+            req.add_header("Authorization", f"Bearer {token}")
+        try:
+            with urllib.request.urlopen(req) as response:
+                total_size = int(response.headers.get("Content-Length", 0))
+        except Exception:
+            pass
+            
+    else:
+        raise ValueError(f"Repository {repo_id} does not contain SafeTensors weights.")
+        
+    return tensors, config, format_name, float(total_size)

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/src/modelinfo/ui.py

@@ -43,7 +43,9 @@ def print_model_info(
     footprint: Dict[str, Any],
     disk_size: float,
     context_length: int,
-    tensors: Dict[str, Any]
+    is_default_context: bool,
+    tensors: Dict[str, Any],
+    max_context: int | None = None
 ) -> None:
     summary = Table(box=None, show_header=False, pad_edge=False, padding=(0, 2))
     summary.add_column("Property", style="bold")
@@ -57,18 +59,38 @@ def print_model_info(
         param_text = "[yellow]UNKNOWN (Missing Shards)[/yellow]"
         disk_text = "[yellow]UNKNOWN (Missing Shards)[/yellow]"
         vram_display = "[yellow]UNKNOWN (Missing Shards)[/yellow]"
+    elif footprint["total_memory_bytes"] == 0:
+        param_text = format_params(footprint["total_params"])
+        disk_text = format_bytes(disk_size)
+        vram_display = "[red]Unknown (Missing Tensor Shapes)[/red]"
     else:
         param_text = format_params(footprint["total_params"])
         disk_text = format_bytes(disk_size)
         vram_bytes = footprint["total_memory_bytes"]
         vram_color = "green" if vram_bytes < 8 * 1024**3 else "yellow" if vram_bytes < 16 * 1024**3 else "red"
         
-        vram_text = f"~{format_bytes(vram_bytes)}"
-        if context_length > 0:
-            vram_text += f" ({footprint['primary_dtype']}, KV cache for {context_length} tokens)"
+        vram_text = f"~{format_bytes(vram_bytes)} Total Minimum Required"
+        vram_display = f"[{vram_color}]{vram_text}[/{vram_color}]\n"
+        
+        weights_bytes = footprint["base_memory_bytes"]
+        vram_display += f"  ├─ Weights:    {format_bytes(weights_bytes)}\n"
+        
+        kv_cache_bytes = footprint["kv_cache_bytes"]
+        
+        if footprint.get("kv_is_estimate"):
+            kv_note = " (Estimated KV Cache - Missing Config)"
+        elif is_default_context:
+            if max_context and max_context > context_length:
+                kv_note = f" (Default {context_length} tokens. Native limit: {max_context:,})"
+            else:
+                kv_note = f" (Default {context_length} tokens)"
         else:
-            vram_text += f" ({footprint['primary_dtype']}, no KV cache)"
-        vram_display = f"[{vram_color}]{vram_text}[/{vram_color}]"
+            kv_note = f" ({context_length} tokens)"
+            
+        vram_display += f"  ├─ KV Cache:   {format_bytes(kv_cache_bytes)}{kv_note}\n"
+        
+        overhead_bytes = footprint.get("overhead_bytes", 600 * 1024 * 1024)
+        vram_display += f"  └─ Overhead:   {format_bytes(overhead_bytes)} (CUDA Context + Activations)"
 
     summary.add_row("Format:", format_name)
     summary.add_row("Architecture:", arch_name)
@@ -81,8 +103,11 @@ def print_model_info(
     console.print(summary)
 
     if missing_shards > 0:
-        console.print(f"[bold yellow]⚠️ Partial Model: Missing {missing_shards} of {total_shards} shards on disk. Totals are incomplete.[/bold yellow]")
+        console.print(f"[bold yellow]WARNING: Partial Model. Missing {missing_shards} of {total_shards} shards on disk. Totals are incomplete.[/bold yellow]")
         
+    if context_length > 0 and max_context is not None and context_length > max_context:
+        console.print(f"[bold yellow]WARNING: Requested context ({context_length:,}) exceeds model's native limit ({max_context:,}).[/bold yellow]")
+
     console.print()
     
     console.print("Top Tensors by Size:", style="bold")
@@ -90,7 +115,7 @@ def print_model_info(
     grouped_tensors = group_tensors_by_size(tensors)
     
     tensor_table = Table(box=None, show_header=False, pad_edge=False, padding=(0, 2))
-    tensor_table.add_column("Name")
+    tensor_table.add_column("Name", no_wrap=True, overflow="fold")
     tensor_table.add_column("Shape", justify="right")
     tensor_table.add_column("Dtype", justify="left")
     tensor_table.add_column("Params", justify="right")

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0/src/modelinfo_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: modelinfo-cli
-Version: 1.0.0
+Version: 1.2.0
 Summary: A sub-100ms, zero-dependency CLI tool to inspect ML model checkpoints and dynamically calculate VRAM requirements.
 Author: ModelInfo Contributors
 License: MIT
@@ -28,12 +28,13 @@ It reads binary headers directly using the Python standard library. By bypassing
 
 ## Features
 
-- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`.
+- **Zero-Dependency Parsing**: Reads the 8-byte JSON prefix of `.safetensors` files and the binary key-value metadata of `.gguf` directly via `struct` and `json`. Seamlessly reads adjacent `config.json` for robust fallback logic.
+- **Remote Hugging Face Hub Inspection**: Inspect any public or gated model directly via its repo ID (e.g., `modelinfo meta-llama/Llama-2-7b-hf`) without downloading the 15GB checkpoint. Uses concurrent byte-range requests to pluck the binary headers directly off the CDN in under 2 seconds.
 - **Sharded Model Support**: Transparently parses `model.safetensors.index.json` to detect multi-file checkpoint distributions, gracefully guarding against partial downloads without crashing.
-- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths.
+- **Dynamic VRAM Estimation**: Extracts underlying model architecture (layers, heads, dimensions) to calculate exact VRAM limits, including dynamic KV cache footprints based on user-specified context lengths. Defaults to 8192 tokens to prevent unrealistic VRAM calculations, while still warning users if the requested context exceeds the model's native limit. Estimates include a standard 600MB CUDA context overhead.
 - **Precise Block Quantization**: Factors in exact byte-scaling coefficients for GGUF formats (e.g., Q8, Q6, Q4) rather than naive averages, eliminating VRAM under-reporting.
 - **Secure Pickling**: Inspects legacy `.pt` files without executing arbitrary code by using a highly restricted `pickle.Unpickler`.
-- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`.
+- **Terminal UI**: Groups repetitive structural layers and color-codes VRAM heatmaps using `rich`. Breaks down memory footprints into Weights, KV Cache, and Overhead.
 
 ## Installation
 
@@ -67,12 +68,18 @@ pytest tests/ -v
 
 ## Usage
 
-Inspect a model checkpoint:
+Inspect a local model checkpoint:
 
 ```bash
 modelinfo mistral-7b.safetensors
 ```
 
+Inspect a remote model directly from the Hugging Face Hub:
+
+```bash
+modelinfo meta-llama/Llama-2-7b-hf
+```
+
 Calculate the memory footprint with a specific KV cache context window:
 
 ```bash

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/src/modelinfo_cli.egg-info/SOURCES.txt

@@ -10,6 +10,7 @@ src/modelinfo/ui.py
 src/modelinfo/parsers/__init__.py
 src/modelinfo/parsers/base.py
 src/modelinfo/parsers/gguf.py
+src/modelinfo/parsers/huggingface.py
 src/modelinfo/parsers/pytorch.py
 src/modelinfo/parsers/safetensors.py
 src/modelinfo_cli.egg-info/PKG-INFO

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/tests/test_calculator.py

@@ -63,3 +63,50 @@ def test_dynamic_kv_cache():
     assert footprint["num_layers"] == 2
     assert footprint["kv_dim"] == 1024
     assert footprint["kv_cache_bytes"] == 8192000
+
+def test_safetensors_config_fallback():
+    """Verify that architecture extraction correctly parses a config dictionary for SafeTensors."""
+    from modelinfo.architecture import extract_architecture
+    
+    tensors = {
+        "model.layers.0.qkv_proj.weight": {
+            "shape": [6144, 4096],
+            "dtype": "F16"
+        }
+    }
+    
+    config = {
+        "num_hidden_layers": 32,
+        "num_attention_heads": 32,
+        "num_key_value_heads": 8,
+        "hidden_size": 4096
+    }
+    
+    num_layers, kv_dim, is_estimate = extract_architecture(tensors, config=config)
+    
+    assert num_layers == 32
+    assert kv_dim == 1024
+    assert is_estimate is False
+
+def test_kv_cache_is_fp16():
+    """Verify that KV cache is always calculated using 2.0 bytes (FP16), even for Q4 base models."""
+    tensors = {
+        "model.layers.0.attn.weight": {"shape": [4096, 4096], "dtype": "Q4"},
+        "model.layers.0.attn.k.weight": {"shape": [1024, 4096], "dtype": "Q4"},
+    }
+    
+    footprint = calculate_footprint(tensors, context_length=8192)
+    
+    assert footprint["kv_cache_bytes"] == 33554432
+    assert footprint["primary_dtype"] == "Q4"
+
+def test_framework_overhead_included():
+    """Verify that CUDA context and activation overhead is correctly included."""
+    tensors = {
+        "model.layers.0.attn.weight": {"shape": [1024, 1024], "dtype": "F16"}
+    }
+    footprint = calculate_footprint(tensors)
+    
+    assert "overhead_bytes" in footprint
+    assert footprint["overhead_bytes"] == 600 * 1024 * 1024
+    assert footprint["total_memory_bytes"] == footprint["base_memory_bytes"] + footprint["kv_cache_bytes"] + footprint["overhead_bytes"]

{modelinfo_cli-1.0.0 → modelinfo_cli-1.2.0}/tests/test_parsers.py

@@ -30,3 +30,18 @@ def test_missing_shard_handling():
     # it fails safely when a file truly doesn't exist.
     with pytest.raises(FileNotFoundError):
         parse_safetensors_header(os.path.join(FIXTURES_DIR, "does_not_exist.safetensors"))
+
+def test_gguf_parser_metadata():
+    """Verify that the GGUF parser extracts the global metadata bypass."""
+    from modelinfo.parsers.gguf import parse_gguf_header
+    from modelinfo.architecture import identify_architecture_name
+    
+    mock_path = os.path.join(FIXTURES_DIR, "mock_model.gguf")
+    tensors = parse_gguf_header(mock_path)
+    
+    assert "__metadata__" in tensors
+    assert tensors["__metadata__"]["general.architecture"] == "qwen2"
+    
+    # Verify the architecture bypass parses it to titlecase and prevents "Unknown Architecture"
+    arch_name = identify_architecture_name(tensors, num_layers=1)
+    assert arch_name == "Qwen2 (1 transformer layers)"

modelinfo_cli-1.0.0/src/modelinfo/architecture.py

@@ -1,45 +0,0 @@
-from typing import Any, Dict, Tuple
-
-def extract_architecture(tensors: Dict[str, Any]) -> Tuple[int, int]:
-    """
-    Extracts the number of layers and KV cache dimension (kv_heads * head_dim)
-    from tensor metadata.
-    """
-    layers = set()
-    kv_dim = 0
-    
-    for name, metadata in tensors.items():
-        if name == "__metadata__":
-            continue
-            
-        parts = name.split(".")
-        
-        if "layers" in parts:
-            idx = parts.index("layers")
-            if len(parts) > idx + 1 and parts[idx+1].isdigit():
-                layers.add(int(parts[idx+1]))
-        elif "h" in parts:
-            idx = parts.index("h")
-            if len(parts) > idx + 1 and parts[idx+1].isdigit():
-                layers.add(int(parts[idx+1]))
-
-        if name.endswith("k_proj.weight") or name.endswith("attn.k.weight") or name.endswith("k_proj.w"):
-            shape = metadata.get("shape", [])
-            if len(shape) >= 2:
-                # Typically [out_features, in_features], so out_features is shape[0]
-                kv_dim = shape[0]
-
-    return len(layers), kv_dim
-
-def identify_architecture_name(tensors: Dict[str, Any], num_layers: int) -> str:
-    """Attempt to identify the architecture family based on tensor names."""
-    for name in tensors.keys():
-        name_lower = name.lower()
-        if "llama" in name_lower:
-            return f"Llama ({num_layers} transformer layers)" if num_layers else "Llama"
-        if "mistral" in name_lower:
-            return f"Mistral ({num_layers} transformer layers)" if num_layers else "Mistral"
-        if "qwen" in name_lower:
-            return f"Qwen ({num_layers} transformer layers)" if num_layers else "Qwen"
-        
-    return f"Generic Transformer ({num_layers} layers)" if num_layers > 0 else "Unknown Architecture"

modelinfo_cli-1.0.0/src/modelinfo/cli.py

@@ -1,77 +0,0 @@
-import argparse
-import os
-import sys
-from typing import Sequence
-
-from modelinfo.architecture import identify_architecture_name
-from modelinfo.calculator import calculate_footprint
-from modelinfo.parsers.gguf import parse_gguf_header
-from modelinfo.parsers.pytorch import parse_pytorch_header
-from modelinfo.parsers.safetensors import parse_safetensors_header
-from modelinfo.ui import console, print_model_info
-
-
-def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
-    parser = argparse.ArgumentParser(
-        prog="modelinfo",
-        description="High-performance CLI utility to inspect ML model checkpoints and calculate VRAM requirements.",
-    )
-    
-    parser.add_argument(
-        "file",
-        type=str,
-        help="Path to the model checkpoint file (.safetensors, .gguf, .pt)",
-    )
-    parser.add_argument(
-        "--context",
-        type=int,
-        default=0,
-        help="Context length for dynamic KV cache footprint calculation.",
-    )
-
-    return parser.parse_args(argv)
-
-
-def main(argv: Sequence[str] | None = None) -> int:
-    args = parse_args(argv)
-
-    file_path = args.file.lower()
-    tensors = {}
-    
-    if file_path.endswith(".safetensors") or file_path.endswith(".index.json"):
-        tensors = parse_safetensors_header(args.file)
-        format_name = "SafeTensors"
-    elif file_path.endswith(".gguf"):
-        tensors = parse_gguf_header(args.file)
-        format_name = "GGUF"
-    elif file_path.endswith(".pt") or file_path.endswith(".bin"):
-        tensors = parse_pytorch_header(args.file)
-        format_name = "PyTorch"
-    else:
-        console.print(
-            f"[red]Error: Unsupported file format '{args.file}'. Supported formats are .safetensors, .gguf, .pt[/red]"
-        )
-        return 1
-        
-    footprint = calculate_footprint(tensors, context_length=args.context)
-    num_layers = footprint["num_layers"]
-    arch_name = identify_architecture_name(tensors, num_layers)
-    
-    disk_size = os.path.getsize(args.file) if os.path.exists(args.file) else 0.0
-    tensor_count = len([k for k in tensors.keys() if k != "__metadata__"])
-    
-    print_model_info(
-        format_name=format_name,
-        arch_name=arch_name,
-        tensor_count=tensor_count,
-        footprint=footprint,
-        disk_size=disk_size,
-        context_length=args.context,
-        tensors=tensors
-    )
-
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())