hf-mem 0.5.6__tar.gz

hf_mem-0.5.6/LICENSE

@@ -0,0 +1,25 @@
+Copyright (c) 2026-present Alvaro Bartolome <alvarobartt@gmail.com>
+
+Permission is hereby granted, free of charge, to any
+person obtaining a copy of this software and associated
+documentation files (the "Software"), to deal in the
+Software without restriction, including without
+limitation the rights to use, copy, modify, merge,
+publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software
+is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice
+shall be included in all copies or substantial portions
+of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
+ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
+TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
+IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.

hf_mem-0.5.6/PKG-INFO

@@ -0,0 +1,131 @@
+Metadata-Version: 2.4
+Name: hf-mem
+Version: 0.5.6
+Summary: A CLI to estimate inference memory requirements for Hugging Face models, written in Python.
+Author: Alvaro Bartolome
+Author-email: Alvaro Bartolome <alvarobartt@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Requires-Dist: httpx2[http2]>=2.12.0
+Requires-Python: >=3.10
+Project-URL: Homepage, https://github.com/alvarobartt/hf-mem
+Project-URL: Repository, https://github.com/alvarobartt/hf-mem
+Project-URL: Issues, https://github.com/alvarobartt/hf-mem/issues
+Description-Content-Type: text/markdown
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/hf-mem.png" />
+
+---
+
+> [!WARNING]
+> `hf-mem` is still experimental and therefore subject to major changes across releases, so please keep in mind that breaking changes may occur until v1.0.0.
+
+`hf-mem` is a CLI to estimate inference memory requirements for Hugging Face models, written in Python. `hf-mem` is lightweight, only depends on [`httpx2`](https://github.com/pydantic/httpx2), as it pulls the [Safetensors](https://github.com/huggingface/safetensors) and / or [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) metadata via [HTTP Range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests). It's recommended to run with [`uv`](https://github.com/astral-sh/uv) for a better experience.
+
+`hf-mem` lets you estimate the inference requirements to run any model from the Hugging Face Hub, including [Transformers](https://github.com/huggingface/transformers), [Diffusers](https://github.com/huggingface/diffusers) and [Sentence Transformers](https://github.com/huggingface/sentence-transformers) models, or really any model as long as it contains any of [Safetensors](https://github.com/huggingface/safetensors) or [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) weights.
+
+Read more information about `hf-mem` in [this short-form post](https://alvarobartt.com/hf-mem), but note it's not up-to-date as it was written in January 2026.
+
+## Usage
+
+### CLI (Recommended)
+
+#### Transformers
+
+```bash
+uvx hf-mem --model-id MiniMaxAI/MiniMax-M2
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/transformers.png" />
+
+#### Diffusers
+
+```bash
+uvx hf-mem --model-id Qwen/Qwen-Image
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/diffusers.png" />
+
+#### Sentence Transformers
+
+```bash
+uvx hf-mem --model-id google/embeddinggemma-300m
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/sentence-transformers.png" />
+
+### Python
+
+You can also run it programmatically with Python as:
+
+```python
+from hf_mem import run
+
+result = run(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
+print(result)
+# Result(model_id='MiniMaxAI/MiniMax-M2', revision='main', filename=None, memory=230121630720, kv_cache=24964497408, total_memory=255086128128, details=False)
+```
+
+If you're already inside an async application, use `arun(...)` instead:
+
+```python
+from hf_mem import arun
+
+result = await arun(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
+print(result)
+# Result(model_id='MiniMaxAI/MiniMax-M2', revision='main', filename=None, memory=230121630720, kv_cache=24964497408, total_memory=255086128128, details=False)
+```
+
+## Experimental
+
+By enabling the `--experimental` flag, you can enable the KV Cache memory estimation for LLMs (`...ForCausalLM`) and VLMs (`...ForConditionalGeneration`), even including a custom `--max-model-len` (defaults to the `config.json` default), `--batch-size` (defaults to 1), and the `--kv-cache-dtype` (defaults to `auto` which means it uses the default data type set in `config.json` under `torch_dtype` or `dtype`, or rather from `quantization_config` when applicable).
+
+Additionally, for Mixture of Experts (MoEs), the `--experimental` flag also adds a weights-only breakdown of the base model and all the experts combined.
+
+```bash
+uvx hf-mem --model-id MiniMaxAI/MiniMax-M2 --experimental
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/experimental.png" />
+
+## GGUF
+
+If the repository contains GGUF model weights, those will be listed by default (only if there are no Safetensors weights, otherwise the GGUFs will be ignored) and the memory will be estimated for each one of those; whereas if a specific file is provided, then the memory estimation will be targeted for that given file instead.
+
+```bash
+uvx hf-mem --model-id TheBloke/deepseek-llm-7B-chat-GGUF --experimental
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/gguf.png" />
+
+Or if you want to only get the estimation on a given file:
+
+```bash
+uvx hf-mem --model-id TheBloke/deepseek-llm-7B-chat-GGUF --gguf-file deepseek-llm-7b-chat.Q2_K.gguf --experimental
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/gguf-file.png" />
+
+## Skills
+
+Optionally, you can add `hf-mem` as an agent skill, which allows the underlying coding agent to discover and use it when provided as a [`SKILL.md`](skills/hf-mem/SKILL.md), e.g., `.claude/skills/hf-mem/SKILL.md`.
+
+More information can be found at [Anthropic Agent Skills and how to use them](https://github.com/anthropics/skills).
+
+## Extensions
+
+Optionally, you can also add `hf-mem` as an extension to the Hugging Face Hub CLI under `hf` as `hf mem ...`. First you need to install `hf` as explained in [Hugging Face CLI - "Getting started"](https://huggingface.co/docs/huggingface_hub/en/guides/cli). To add `hf-mem` as an extension, all you need to do is run:
+
+```bash
+hf extensions add alvarobartt/hf-mem
+```
+More information can be found at [Hugging Face Hub CLI - "How Python extensions are installed"](https://huggingface.co/docs/huggingface_hub/en/guides/cli-extensions#how-python-extensions-are-installed).
+
+Once installed, you can use `hf-mem` via `hf` as `hf mem ...` along with the rest of Hugging Face extensions you have installed (which you can list via `hf extensions list`).
+
+## References
+
+- [Safetensors Metadata parsing](https://huggingface.co/docs/safetensors/en/metadata_parsing)
+- [GGUF File format](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md)
+- [GGUF on the Hugging Face Hub](https://huggingface.co/docs/hub/en/gguf)
+- [usgraphics - TR-100 Machine Report](https://github.com/usgraphics/usgc-machine-report)

hf_mem-0.5.6/README.md

@@ -0,0 +1,116 @@
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/hf-mem.png" />
+
+---
+
+> [!WARNING]
+> `hf-mem` is still experimental and therefore subject to major changes across releases, so please keep in mind that breaking changes may occur until v1.0.0.
+
+`hf-mem` is a CLI to estimate inference memory requirements for Hugging Face models, written in Python. `hf-mem` is lightweight, only depends on [`httpx2`](https://github.com/pydantic/httpx2), as it pulls the [Safetensors](https://github.com/huggingface/safetensors) and / or [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) metadata via [HTTP Range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests). It's recommended to run with [`uv`](https://github.com/astral-sh/uv) for a better experience.
+
+`hf-mem` lets you estimate the inference requirements to run any model from the Hugging Face Hub, including [Transformers](https://github.com/huggingface/transformers), [Diffusers](https://github.com/huggingface/diffusers) and [Sentence Transformers](https://github.com/huggingface/sentence-transformers) models, or really any model as long as it contains any of [Safetensors](https://github.com/huggingface/safetensors) or [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) weights.
+
+Read more information about `hf-mem` in [this short-form post](https://alvarobartt.com/hf-mem), but note it's not up-to-date as it was written in January 2026.
+
+## Usage
+
+### CLI (Recommended)
+
+#### Transformers
+
+```bash
+uvx hf-mem --model-id MiniMaxAI/MiniMax-M2
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/transformers.png" />
+
+#### Diffusers
+
+```bash
+uvx hf-mem --model-id Qwen/Qwen-Image
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/diffusers.png" />
+
+#### Sentence Transformers
+
+```bash
+uvx hf-mem --model-id google/embeddinggemma-300m
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/sentence-transformers.png" />
+
+### Python
+
+You can also run it programmatically with Python as:
+
+```python
+from hf_mem import run
+
+result = run(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
+print(result)
+# Result(model_id='MiniMaxAI/MiniMax-M2', revision='main', filename=None, memory=230121630720, kv_cache=24964497408, total_memory=255086128128, details=False)
+```
+
+If you're already inside an async application, use `arun(...)` instead:
+
+```python
+from hf_mem import arun
+
+result = await arun(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
+print(result)
+# Result(model_id='MiniMaxAI/MiniMax-M2', revision='main', filename=None, memory=230121630720, kv_cache=24964497408, total_memory=255086128128, details=False)
+```
+
+## Experimental
+
+By enabling the `--experimental` flag, you can enable the KV Cache memory estimation for LLMs (`...ForCausalLM`) and VLMs (`...ForConditionalGeneration`), even including a custom `--max-model-len` (defaults to the `config.json` default), `--batch-size` (defaults to 1), and the `--kv-cache-dtype` (defaults to `auto` which means it uses the default data type set in `config.json` under `torch_dtype` or `dtype`, or rather from `quantization_config` when applicable).
+
+Additionally, for Mixture of Experts (MoEs), the `--experimental` flag also adds a weights-only breakdown of the base model and all the experts combined.
+
+```bash
+uvx hf-mem --model-id MiniMaxAI/MiniMax-M2 --experimental
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/experimental.png" />
+
+## GGUF
+
+If the repository contains GGUF model weights, those will be listed by default (only if there are no Safetensors weights, otherwise the GGUFs will be ignored) and the memory will be estimated for each one of those; whereas if a specific file is provided, then the memory estimation will be targeted for that given file instead.
+
+```bash
+uvx hf-mem --model-id TheBloke/deepseek-llm-7B-chat-GGUF --experimental
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/gguf.png" />
+
+Or if you want to only get the estimation on a given file:
+
+```bash
+uvx hf-mem --model-id TheBloke/deepseek-llm-7B-chat-GGUF --gguf-file deepseek-llm-7b-chat.Q2_K.gguf --experimental
+```
+
+<img src="https://huggingface.co/datasets/alvarobartt/hf-mem/resolve/main/gguf-file.png" />
+
+## Skills
+
+Optionally, you can add `hf-mem` as an agent skill, which allows the underlying coding agent to discover and use it when provided as a [`SKILL.md`](skills/hf-mem/SKILL.md), e.g., `.claude/skills/hf-mem/SKILL.md`.
+
+More information can be found at [Anthropic Agent Skills and how to use them](https://github.com/anthropics/skills).
+
+## Extensions
+
+Optionally, you can also add `hf-mem` as an extension to the Hugging Face Hub CLI under `hf` as `hf mem ...`. First you need to install `hf` as explained in [Hugging Face CLI - "Getting started"](https://huggingface.co/docs/huggingface_hub/en/guides/cli). To add `hf-mem` as an extension, all you need to do is run:
+
+```bash
+hf extensions add alvarobartt/hf-mem
+```
+More information can be found at [Hugging Face Hub CLI - "How Python extensions are installed"](https://huggingface.co/docs/huggingface_hub/en/guides/cli-extensions#how-python-extensions-are-installed).
+
+Once installed, you can use `hf-mem` via `hf` as `hf mem ...` along with the rest of Hugging Face extensions you have installed (which you can list via `hf extensions list`).
+
+## References
+
+- [Safetensors Metadata parsing](https://huggingface.co/docs/safetensors/en/metadata_parsing)
+- [GGUF File format](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md)
+- [GGUF on the Hugging Face Hub](https://huggingface.co/docs/hub/en/gguf)
+- [usgraphics - TR-100 Machine Report](https://github.com/usgraphics/usgc-machine-report)

hf_mem-0.5.6/pyproject.toml

@@ -0,0 +1,40 @@
+[project]
+name = "hf-mem"
+version = "0.5.6"
+description = "A CLI to estimate inference memory requirements for Hugging Face models, written in Python."
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.10"
+dependencies = ["httpx2[http2]>=2.12.0"]
+
+[[project.authors]]
+name = "Alvaro Bartolome"
+email = "alvarobartt@gmail.com"
+
+[project.urls]
+Homepage = "https://github.com/alvarobartt/hf-mem"
+Repository = "https://github.com/alvarobartt/hf-mem"
+Issues = "https://github.com/alvarobartt/hf-mem/issues"
+
+[project.scripts]
+hf-mem = "hf_mem.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.9.5,<0.10.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+lint = [
+    "pre-commit>=4.5.1",
+    "ruff>=0.14.10",
+]
+
+[tool.ruff]
+line-length = 112
+
+[tool.ruff.lint]
+select = ["I"]
+
+[tool.ruff.lint.isort]
+force-single-line = false

hf_mem-0.5.6/pyproject.toml.orig

@@ -0,0 +1,41 @@
+[project]
+name = "hf-mem"
+version = "0.5.6"
+description = "A CLI to estimate inference memory requirements for Hugging Face models, written in Python."
+readme = "README.md"
+authors = [
+    { name = "Alvaro Bartolome", email = "alvarobartt@gmail.com" }
+]
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.10"
+dependencies = [
+    "httpx2[http2]>=2.12.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/alvarobartt/hf-mem"
+Repository = "https://github.com/alvarobartt/hf-mem"
+Issues = "https://github.com/alvarobartt/hf-mem/issues"
+
+[build-system]
+requires = ["uv_build>=0.9.5,<0.10.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+lint = [
+    "pre-commit>=4.5.1",
+    "ruff>=0.14.10",
+]
+
+[tool.ruff]
+line-length = 112
+
+[tool.ruff.lint]
+select = ["I"]
+
+[tool.ruff.lint.isort]
+force-single-line = false
+
+[project.scripts]
+hf-mem = "hf_mem.cli:main"

hf_mem-0.5.6/src/hf_mem/__init__.py

@@ -0,0 +1,4 @@
+from hf_mem._version import __version__
+from hf_mem.run import KvCache, Result, arun, run
+
+__all__ = ["__version__", "run", "arun", "Result", "KvCache"]

hf_mem-0.5.6/src/hf_mem/_fetch.py

@@ -0,0 +1,14 @@
+import os
+from typing import Any, Dict
+
+import httpx2
+
+# NOTE: Shared across all HTTP fetch operations; overridable via the `REQUEST_TIMEOUT` environment variable
+REQUEST_TIMEOUT = float(os.getenv("REQUEST_TIMEOUT", 30.0))
+
+
+# NOTE: Return type-hint set to `Any`, but it will only be a JSON-compatible object
+async def get_json_file(client: httpx2.AsyncClient, url: str, headers: Dict[str, str] | None = None) -> Any:
+    response = await client.get(url, headers=headers, timeout=REQUEST_TIMEOUT)
+    response.raise_for_status()
+    return response.json()

hf_mem-0.5.6/src/hf_mem/_print.py

@@ -0,0 +1,133 @@
+from typing import Literal
+
+MIN_NAME_LEN = 5
+MAX_NAME_LEN = 14
+MAX_DATA_LEN = 64
+BORDERS_AND_PADDING = 4
+
+BOX = {
+    "tl": "┌",
+    "tr": "┐",
+    "bl": "└",
+    "br": "┘",
+    "ht": "─",
+    "vt": "│",
+    "tsep": "┬",
+    "bsep": "┴",
+    "lm": "├",
+    "rm": "┤",
+    "mm": "┼",
+}
+
+
+def _print_with_color(content: str) -> None:
+    print(f"\x1b[38;2;244;183;63m{content}\x1b[0m")
+
+
+def _print_header(current_len: int, badge: str | None = None) -> None:
+    length = current_len + 2 * BORDERS_AND_PADDING + 2
+    if badge:
+        badge_text = f" {badge} "
+        fill_len = max(0, length - 2 - len(badge_text))
+        top = BOX["tl"] + (BOX["tsep"] * fill_len) + badge_text + BOX["tr"]
+    else:
+        top = BOX["tl"] + (BOX["tsep"] * (length - 2)) + BOX["tr"]
+    _print_with_color(top)
+
+    bottom = BOX["lm"] + (BOX["bsep"] * (length - 2)) + BOX["rm"]
+    _print_with_color(bottom)
+
+
+def _print_centered(text: str, current_len: int) -> None:
+    total_width = current_len + 2 * BORDERS_AND_PADDING
+    text_len = len(text)
+    pad_left = (total_width - text_len) // 2
+    pad_right = total_width - text_len - pad_left
+    _print_with_color(f"{BOX['vt']}{' ' * pad_left}{text}{' ' * pad_right}{BOX['vt']}")
+
+
+def _print_divider(
+    current_len: int,
+    side: Literal["top", "top-continue", "mid", "bottom", "bottom-continue"] | None = None,
+    name_len: int = MAX_NAME_LEN,
+) -> None:
+    match side:
+        case "top":
+            left, mid, right = BOX["lm"], BOX["tsep"], BOX["rm"]
+        case "top-continue":
+            left, mid, right = BOX["lm"], BOX["bsep"], BOX["rm"]
+        case "bottom":
+            left, mid, right = BOX["bl"], BOX["bsep"], BOX["br"]
+        case "bottom-continue":
+            left, mid, right = BOX["lm"], BOX["bsep"], BOX["rm"]
+        case "mid":
+            left, mid, right = BOX["lm"], BOX["ht"], BOX["rm"]
+        case _:
+            left, mid, right = BOX["lm"], BOX["mm"], BOX["rm"]
+
+    name_col_inner = name_len + 2
+    data_col_inner = current_len + 1
+
+    line = left
+    line += BOX["ht"] * name_col_inner
+    line += mid
+    line += BOX["ht"] * data_col_inner
+    line += right
+    _print_with_color(line)
+
+
+def _print_full_divider(
+    current_len: int,
+    side: Literal["top", "top-continue", "bottom", "bottom-continue", "mid"] | None = None,
+) -> None:
+    total_inner = current_len + 2 * BORDERS_AND_PADDING
+
+    match side:
+        case "top":
+            left, right = BOX["lm"], BOX["rm"]
+        case "top-continue":
+            left, right = BOX["lm"], BOX["rm"]
+        case "bottom":
+            left, right = BOX["bl"], BOX["br"]
+        case "bottom-continue":
+            left, right = BOX["lm"], BOX["rm"]
+        case _:
+            left, right = BOX["lm"], BOX["rm"]
+
+    _print_with_color(left + BOX["ht"] * total_inner + right)
+
+
+def _format_name(name: str, max_len: int = MAX_NAME_LEN) -> str:
+    if len(name) < MIN_NAME_LEN:
+        return f"{name:<{MIN_NAME_LEN}}"
+    if len(name) > max_len:
+        return name[: max_len - 3] + "..."
+    return f"{name:<{max_len}}"
+
+
+def _print_row(name: str, text: str, current_len: int, name_len: int = MAX_NAME_LEN) -> None:
+    name_fmt = _format_name(name, name_len)
+    data_fmt = f"{str(text):<{current_len}}"
+    _print_with_color(f"{BOX['vt']} {name_fmt} {BOX['vt']} {data_fmt} {BOX['vt']}")
+
+
+def _make_bar(used: float, total: float, width: int) -> str:
+    if total <= 0:
+        return "░" * width
+    frac = min(max(used / total, 0.0), 1.0)
+    filled = int(round(frac * width))
+    filled = max(0, min(width, filled))
+    return "█" * filled + "░" * (width - filled)
+
+
+def _format_short_number(n: float) -> str:
+    n = float(n)
+    for unit in ("", "K", "M", "B", "T"):
+        if abs(n) < 1000.0:
+            return f"{int(n)}" if unit == "" else f"{n:.2f}{unit}"
+        n /= 1000.0
+    return f"{n:.2f}P"
+
+
+def _bytes_to_gib(nbytes: int) -> float:
+    return nbytes / (1024**3)

hf_mem-0.5.6/src/hf_mem/_types.py

@@ -0,0 +1,9 @@
+from dataclasses import dataclass
+
+
+@dataclass
+class KvCache:
+    max_model_len: int
+    cache_size: int
+    batch_size: int
+    cache_dtype: str | None = None

hf_mem-0.5.6/src/hf_mem/_version.py

@@ -0,0 +1,5 @@
+from importlib.metadata import version
+
+# NOTE: Single source of truth for the package version, imported by all sub-modules that need it
+# to avoid circular imports through hf_mem.__init__
+__version__: str = version("hf-mem")

hf_mem-0.5.6/src/hf_mem/cli.py

@@ -0,0 +1,148 @@
+import argparse
+import asyncio
+import json
+import warnings
+
+from hf_mem.gguf.print import print_gguf_files_report, print_gguf_report
+from hf_mem.gguf.types import GGUFDtype
+from hf_mem.run import Result, arun
+from hf_mem.safetensors.kv_cache import KV_CACHE_DTYPE_CHOICES
+from hf_mem.safetensors.print import print_safetensors_report
+
+VALID_KV_CACHE_DTYPES = set(KV_CACHE_DTYPE_CHOICES) | set(GGUFDtype.__members__.keys())
+
+
+def _print_result(result: Result) -> None:
+    if result.safetensors is not None:
+        print_safetensors_report(
+            model_id=result.model_id,
+            revision=result.revision,
+            metadata=result.safetensors,
+            kv_cache=result.kv_cache_metadata,
+            moe=result.moe_metadata,
+        )
+        return
+
+    if result.gguf_files is not None:
+        if result.filename is not None:
+            print_gguf_report(
+                model_id=result.model_id,
+                filename=result.filename,
+                revision=result.revision,
+                metadata=result.gguf_files[result.filename],
+                kv_cache=result.gguf_files[result.filename].kv_cache,
+            )
+        else:
+            print_gguf_files_report(
+                model_id=result.model_id,
+                revision=result.revision,
+                gguf_files=result.gguf_files,
+                memory=result.memory,  # type: ignore[arg-type]
+                kv_cache=result.kv_cache,  # type: ignore[arg-type]
+            )
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--model-id", required=True, help="Model ID on the Hugging Face Hub")
+    parser.add_argument(
+        "--revision",
+        default="main",
+        help="Model revision on the Hugging Face Hub",
+    )
+    parser.add_argument(
+        "--hf-token",
+        type=str,
+        default=None,
+        required=False,
+        help="The Hugging Face Hub token to use when sending requests to the Hub. If not provided it will be retrieved from either the `HF_TOKEN` environment variable or from the `.cache/huggingface/token` file if applicable.",
+    )
+
+    parser.add_argument(
+        "--experimental",
+        action="store_true",
+        help=(
+            "Whether to enable the experimental KV Cache estimation or not. For Safetensors "
+            "MoE Transformers models it also includes a base-model-vs-experts weights "
+            "breakdown. Only applies to `...ForCausalLM` and `...ForConditionalGeneration` "
+            "models from Transformers."
+        ),
+    )
+    parser.add_argument(
+        "--max-model-len",
+        type=int,
+        default=None,
+        # NOTE: https://docs.vllm.ai/en/stable/configuration/engine_args/#-max-model-len
+        help="Model context length (prompt and output). If unspecified, will be automatically derived from the model config.",
+    )
+    parser.add_argument(
+        "--batch-size",
+        type=int,
+        default=1,
+        help="Batch size to help estimate the required RAM for caching when running the inference. Defaults to 1.",
+    )
+    parser.add_argument(
+        "--kv-cache-dtype",
+        type=str,
+        default="auto",
+        # NOTE: https://docs.vllm.ai/en/stable/cli/serve/#-kv-cache-dtype
+        help=f"Data type for the KV cache storage. If `auto` is specified, it will use the default model dtype specified in the `config.json` (if available) or F16 for GGUF files. Despite the FP8 data types having different formats, all those take 1 byte, meaning that the calculation would lead to the same results. Valid values are {KV_CACHE_DTYPE_CHOICES} for safetensors files and {['auto'] + list(GGUFDtype.__members__.keys())} for GGUF files. Defaults to `auto`.",
+    )
+
+    parser.add_argument(
+        "--json-output",
+        action="store_true",
+        help="Whether to provide the output as a JSON instead of printed as table.",
+    )
+    parser.add_argument(
+        "--ignore-table-width",
+        action="store_true",
+        help="Deprecated: table width now auto-expands when needed, so this flag is ignored.",
+    )
+    parser.add_argument(
+        "--gguf-file",
+        type=str,
+        default=None,
+        help="Specific GGUF file to estimate. If not provided, all GGUF files found in the repo will be estimated. The path within the repo is required (e.g. Q4_K_M/model-Q4_K_M.gguf).",
+    )
+    parser.add_argument(
+        "--details",
+        action="store_true",
+        help="Include per-component dtype and parameter breakdowns in JSON output (`--json-output` only).",
+    )
+    args = parser.parse_args()
+
+    if args.experimental:
+        warnings.warn(
+            "`--experimental` is set, which means that models with an architecture as `...ForCausalLM` and `...ForConditionalGeneration` will include estimations for the KV Cache as well. You can also provide the args `--max-model-len` and `--batch-size` as part of the estimation. Note that enabling `--experimental` means that the output will be different both when displayed and when dumped as JSON with `--json-output`, so bear that in mind."
+        )
+
+    if args.ignore_table_width:
+        warnings.warn(
+            "`--ignore-table-width` is deprecated and has no effect; table width auto-expands when needed by default."
+        )
+
+    if args.kv_cache_dtype not in VALID_KV_CACHE_DTYPES:
+        raise RuntimeError(
+            f"--kv-cache-dtype={args.kv_cache_dtype} not recognized. Valid options: {VALID_KV_CACHE_DTYPES}."
+        )
+
+    result = asyncio.run(
+        arun(
+            model_id=args.model_id,
+            revision=args.revision,
+            hf_token=args.hf_token,
+            experimental=args.experimental,
+            max_model_len=args.max_model_len,
+            batch_size=args.batch_size,
+            kv_cache_dtype=args.kv_cache_dtype,
+            gguf_file=args.gguf_file,
+            details=args.details,
+        )
+    )
+
+    if args.json_output:
+        print(json.dumps(result.to_json()))
+    else:
+        _print_result(result)