canirunllm 0.1.0__tar.gz

canirunllm-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 jashwanthsai678
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

canirunllm-0.1.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: canirunllm
+Version: 0.1.0
+Summary: Find out which open-source LLMs can run on your hardware.
+Author: jashwanthsai678
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/jashwanthsai678/CaniRunLLM
+Project-URL: Repository, https://github.com/jashwanthsai678/CaniRunLLM
+Project-URL: Issues, https://github.com/jashwanthsai678/CaniRunLLM/issues
+Keywords: llm,local-llm,hardware,gguf,llama.cpp,quantization
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Environment :: Web Environment
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: System :: Hardware
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastapi>=0.110
+Requires-Dist: uvicorn>=0.29
+Requires-Dist: pydantic>=2.0
+Requires-Dist: psutil>=5.9
+Requires-Dist: GPUtil>=1.4
+Dynamic: license-file
+
+# CanIRunLLM
+
+Find out which open-source LLMs your machine can actually run — and which
+one you should pick — with an automatic hardware scan and a local dashboard.
+
+CanIRunLLM inspects your real CPU, RAM, and GPU/VRAM, checks that against a
+registry of real open-weight models (Qwen, Llama, Mistral, Gemma, Phi,
+DeepSeek, and more), and gives you a plain-language verdict — not just a raw
+"fits/doesn't fit" table. It runs entirely on your machine: no account, no
+cloud calls, no telemetry.
+
+```
+$ canirunllm scan
+
+CanIRunLLM
+
+Checking your computer...
+  CPU detected
+  RAM detected
+  GPU detected
+  VRAM detected
+
+Analyzing local AI models...
+  58 models analyzed
+
+You can run 14 model(s) comfortably.
+7 more can run with CPU/RAM offload (slower).
+
+Dashboard:
+  http://127.0.0.1:8765
+
+Opening browser...
+```
+
+The browser dashboard shows a friendly "what can I run / what should I run"
+report, with the full technical breakdown (VRAM, RAM, KV cache, quantization,
+runtime, confidence) available behind a "Technical details" toggle for anyone
+who wants it.
+
+## Install
+
+Requires Python 3.10+.
+
+```bash
+git clone https://github.com/jashwanthsai678/CaniRunLLM.git
+cd CaniRunLLM
+pip install -e .
+```
+
+## Quick start
+
+```bash
+canirunllm scan               # scan hardware, evaluate models, open the dashboard
+canirunllm scan --no-browser  # same, but skip the dashboard (good for CI/headless)
+canirunllm scan --technical   # also print the full technical breakdown in the terminal
+
+canirunllm search qwen        # search the model registry
+canirunllm check Qwen3-8B     # check one model or a whole family
+canirunllm recommend          # ranked list of models for your hardware
+
+canirunllm web                # launch the dashboard on its own
+canirunllm web --port 9000    # on a custom port
+```
+
+## What it actually checks
+
+For every model + quantization pair, CanIRunLLM estimates:
+
+- **Weight memory** from parameter count and quantization (Q4_K_M, Q8_0, etc.)
+- **KV cache size**, which grows with context length — a model isn't just
+  "fits" or "doesn't," it fits *at a given context length*
+- **Runtime overhead and a safety margin**, not just the raw weight size
+- **Memory strategy**: does it fit on a single GPU, does it need to be split
+  across multiple GPUs, or does it need CPU/RAM offloading — each of these
+  is a real, different scenario, not a single generic "offload" verdict
+- **Runtime compatibility**: is the declared runtime (llama.cpp, etc.)
+  actually known to support that memory strategy
+
+The result is always a verdict *plus* a confidence level and a plain-English
+reason — never a bare "cannot run" with no explanation.
+
+## Architecture
+
+```
+CLI / Web Dashboard  (presentation only)
+        │
+Application Layer   (ScannerService, RecommendationEngine)
+        │
+   ┌────┴─────┬──────────────┬─────────────┐
+   ▼          ▼              ▼             ▼
+Hardware   Model Registry  Compatibility  Performance
+Scanner    + Resolver      Engine         Prediction
+```
+
+The CLI and the local web dashboard are two presentation layers over the
+same Python core — nothing about compatibility is calculated twice, and the
+frontend never re-derives a verdict on its own.
+
+## Current limitations (being upfront about them)
+
+- GPU detection currently only recognizes NVIDIA GPUs (via `GPUtil`/
+  `nvidia-smi`). On AMD/Intel/Apple Silicon machines it safely falls back to
+  CPU-only mode rather than crashing, but it won't report real GPU numbers yet.
+- The model registry is a curated set of well-known open-weight models, not
+  an exhaustive mirror of every model on Hugging Face — see
+  `src/canirunllm/registry/SOURCES.md` for exactly where every number in it
+  came from.
+- Performance numbers are a coarse, clearly-labeled *estimate* based on
+  parameter count and memory strategy — there is no real benchmarking yet,
+  and the tool never presents an estimate as a measurement.
+
+## Testing
+
+```bash
+pip install -e .
+pytest -v
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

canirunllm-0.1.0/README.md

@@ -0,0 +1,121 @@
+# CanIRunLLM
+
+Find out which open-source LLMs your machine can actually run — and which
+one you should pick — with an automatic hardware scan and a local dashboard.
+
+CanIRunLLM inspects your real CPU, RAM, and GPU/VRAM, checks that against a
+registry of real open-weight models (Qwen, Llama, Mistral, Gemma, Phi,
+DeepSeek, and more), and gives you a plain-language verdict — not just a raw
+"fits/doesn't fit" table. It runs entirely on your machine: no account, no
+cloud calls, no telemetry.
+
+```
+$ canirunllm scan
+
+CanIRunLLM
+
+Checking your computer...
+  CPU detected
+  RAM detected
+  GPU detected
+  VRAM detected
+
+Analyzing local AI models...
+  58 models analyzed
+
+You can run 14 model(s) comfortably.
+7 more can run with CPU/RAM offload (slower).
+
+Dashboard:
+  http://127.0.0.1:8765
+
+Opening browser...
+```
+
+The browser dashboard shows a friendly "what can I run / what should I run"
+report, with the full technical breakdown (VRAM, RAM, KV cache, quantization,
+runtime, confidence) available behind a "Technical details" toggle for anyone
+who wants it.
+
+## Install
+
+Requires Python 3.10+.
+
+```bash
+git clone https://github.com/jashwanthsai678/CaniRunLLM.git
+cd CaniRunLLM
+pip install -e .
+```
+
+## Quick start
+
+```bash
+canirunllm scan               # scan hardware, evaluate models, open the dashboard
+canirunllm scan --no-browser  # same, but skip the dashboard (good for CI/headless)
+canirunllm scan --technical   # also print the full technical breakdown in the terminal
+
+canirunllm search qwen        # search the model registry
+canirunllm check Qwen3-8B     # check one model or a whole family
+canirunllm recommend          # ranked list of models for your hardware
+
+canirunllm web                # launch the dashboard on its own
+canirunllm web --port 9000    # on a custom port
+```
+
+## What it actually checks
+
+For every model + quantization pair, CanIRunLLM estimates:
+
+- **Weight memory** from parameter count and quantization (Q4_K_M, Q8_0, etc.)
+- **KV cache size**, which grows with context length — a model isn't just
+  "fits" or "doesn't," it fits *at a given context length*
+- **Runtime overhead and a safety margin**, not just the raw weight size
+- **Memory strategy**: does it fit on a single GPU, does it need to be split
+  across multiple GPUs, or does it need CPU/RAM offloading — each of these
+  is a real, different scenario, not a single generic "offload" verdict
+- **Runtime compatibility**: is the declared runtime (llama.cpp, etc.)
+  actually known to support that memory strategy
+
+The result is always a verdict *plus* a confidence level and a plain-English
+reason — never a bare "cannot run" with no explanation.
+
+## Architecture
+
+```
+CLI / Web Dashboard  (presentation only)
+        │
+Application Layer   (ScannerService, RecommendationEngine)
+        │
+   ┌────┴─────┬──────────────┬─────────────┐
+   ▼          ▼              ▼             ▼
+Hardware   Model Registry  Compatibility  Performance
+Scanner    + Resolver      Engine         Prediction
+```
+
+The CLI and the local web dashboard are two presentation layers over the
+same Python core — nothing about compatibility is calculated twice, and the
+frontend never re-derives a verdict on its own.
+
+## Current limitations (being upfront about them)
+
+- GPU detection currently only recognizes NVIDIA GPUs (via `GPUtil`/
+  `nvidia-smi`). On AMD/Intel/Apple Silicon machines it safely falls back to
+  CPU-only mode rather than crashing, but it won't report real GPU numbers yet.
+- The model registry is a curated set of well-known open-weight models, not
+  an exhaustive mirror of every model on Hugging Face — see
+  `src/canirunllm/registry/SOURCES.md` for exactly where every number in it
+  came from.
+- Performance numbers are a coarse, clearly-labeled *estimate* based on
+  parameter count and memory strategy — there is no real benchmarking yet,
+  and the tool never presents an estimate as a measurement.
+
+## Testing
+
+```bash
+pip install -e .
+pytest -v
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

canirunllm-0.1.0/pyproject.toml

@@ -0,0 +1,51 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "canirunllm"
+version = "0.1.0"
+description = "Find out which open-source LLMs can run on your hardware."
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.10"
+authors = [
+    { name = "jashwanthsai678" },
+]
+keywords = ["llm", "local-llm", "hardware", "gguf", "llama.cpp", "quantization"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Environment :: Console",
+    "Environment :: Web Environment",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Libraries",
+    "Topic :: System :: Hardware",
+]
+dependencies = [
+    "fastapi>=0.110",
+    "uvicorn>=0.29",
+    "pydantic>=2.0",
+    "psutil>=5.9",
+    "GPUtil>=1.4",
+]
+
+[project.urls]
+Homepage = "https://github.com/jashwanthsai678/CaniRunLLM"
+Repository = "https://github.com/jashwanthsai678/CaniRunLLM"
+Issues = "https://github.com/jashwanthsai678/CaniRunLLM/issues"
+
+[project.scripts]
+canirunllm = "canirunllm.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+"canirunllm.registry" = ["*.json"]
+"canirunllm.web" = ["static/*", "templates/*"]

canirunllm-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

canirunllm-0.1.0/src/canirunllm/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

canirunllm-0.1.0/src/canirunllm/api/presentation.py

@@ -0,0 +1,192 @@
+"""Translates raw core objects into human-readable product language.
+
+This is deliberately kept server-side (Python), not JavaScript: the
+frontend must never re-derive compatibility meaning on its own. It
+only renders the strings and flags this module produces.
+
+Nothing here recalculates compatibility — it only interprets fields
+that CompatibilityResult/RankedModel already computed.
+"""
+
+from dataclasses import dataclass
+
+from canirunllm.compatibility.engine import CompatibilityResult
+from canirunllm.compatibility.decision import OverallVerdict
+from canirunllm.compatibility.runtime import RuntimeVerdict
+from canirunllm.models.model import ModelSpec
+from canirunllm.recommendation.engine import RankedModel
+from canirunllm.recommendation.tier import RecommendationTier
+
+
+GB = 1024 ** 3
+
+FAVORABLE_TIERS = {RecommendationTier.BEST_MATCH, RecommendationTier.GOOD}
+
+# (friendly label, icon) — icon is a UI-neutral token, not an emoji,
+# so the frontend controls the actual glyph/color.
+VERDICT_LABELS: dict[OverallVerdict, tuple[str, str]] = {
+    OverallVerdict.CAN_RUN: ("Runs comfortably", "check"),
+    OverallVerdict.CAN_RUN_WITH_OFFLOAD: ("Runs, but slower", "warn"),
+    OverallVerdict.NEEDS_VALIDATION: ("Needs more info to confirm", "unknown"),
+    OverallVerdict.CANNOT_RUN: ("Not enough resources", "cross"),
+}
+
+
+@dataclass
+class ReasonItem:
+    ok: bool
+    text: str
+
+
+@dataclass
+class RunCommandInfo:
+    runtime: str
+    command: str
+    note: str
+
+
+def friendly_verdict(overall_verdict: OverallVerdict) -> tuple[str, str]:
+    return VERDICT_LABELS[overall_verdict]
+
+
+def build_reasons(
+    model: ModelSpec,
+    compatibility: CompatibilityResult,
+) -> list[ReasonItem]:
+    """Human-readable checklist explaining a verdict — built only from
+    fields the compatibility engine already computed, never invented."""
+
+    reasons: list[ReasonItem] = []
+
+    required_gb = compatibility.required_memory_bytes / GB
+    vram_gb = compatibility.available_vram_bytes / GB
+    ram_gb = compatibility.available_ram_bytes / GB
+
+    strategy = compatibility.memory_strategy
+
+    if strategy == "SINGLE_GPU":
+        reasons.append(ReasonItem(
+            True,
+            f"Model weights fit within your available GPU memory "
+            f"(~{vram_gb:.1f} GB free).",
+        ))
+
+    elif strategy == "MULTI_GPU":
+        reasons.append(ReasonItem(
+            True,
+            "Model fits by splitting across your multiple GPUs.",
+        ))
+        reasons.append(ReasonItem(
+            False,
+            "Splitting a model across GPUs depends on the runtime "
+            "supporting it correctly, and can be slower than a single "
+            "GPU that fits the whole model.",
+        ))
+
+    elif strategy == "CPU_OFFLOAD":
+        reasons.append(ReasonItem(
+            True,
+            f"Model doesn't fully fit in GPU memory, but fits when "
+            f"combined with system RAM (~{ram_gb:.1f} GB available).",
+        ))
+        reasons.append(ReasonItem(
+            False,
+            "Running part of the model on CPU/RAM is typically much "
+            "slower than running fully on GPU.",
+        ))
+
+    else:
+        reasons.append(ReasonItem(
+            False,
+            f"This model needs about {required_gb:.1f} GB, but your "
+            f"machine currently has ~{vram_gb:.1f} GB free GPU memory "
+            f"and ~{ram_gb:.1f} GB free RAM.",
+        ))
+
+    if compatibility.runtime_verdict == RuntimeVerdict.SUPPORTED:
+        reasons.append(ReasonItem(
+            True,
+            f"Runtime '{model.runtime}' is supported.",
+        ))
+    elif compatibility.runtime_verdict == RuntimeVerdict.UNKNOWN:
+        reasons.append(ReasonItem(
+            False,
+            "This model doesn't specify a runtime, so compatibility "
+            "can't be fully confirmed.",
+        ))
+    else:
+        reasons.append(ReasonItem(
+            False,
+            f"Runtime '{model.runtime}' is not one of the runtimes "
+            "this tool currently recognizes.",
+        ))
+
+    return reasons
+
+
+# Command templates for runtimes this tool actually knows about. These
+# are illustrative examples, not verified working commands — this tool
+# does not download, store, or locate model files on disk.
+_RUNTIME_COMMAND_TEMPLATES: dict[str, tuple[str, str]] = {
+    "llama.cpp": (
+        "llama-cli -m /path/to/{filename} -c {context_length}",
+        "Point this at the GGUF file you've downloaded for this "
+        "model - this tool does not download or store model files.",
+    ),
+}
+
+
+def build_run_command(model: ModelSpec) -> RunCommandInfo | None:
+
+    if model.runtime is None:
+        return None
+
+    template = _RUNTIME_COMMAND_TEMPLATES.get(model.runtime.lower())
+
+    if template is None:
+        return None
+
+    command_template, note = template
+
+    command = command_template.format(
+        filename=f"{model.name}.gguf",
+        context_length=model.context_length,
+    )
+
+    return RunCommandInfo(
+        runtime=model.runtime,
+        command=command,
+        note=note,
+    )
+
+
+def pick_best_for_you(ranked: list[RankedModel]) -> RankedModel | None:
+
+    if not ranked:
+        return None
+
+    top = ranked[0]
+
+    if top.tier in FAVORABLE_TIERS:
+        return top
+
+    return None
+
+
+def pick_alternative(
+    ranked: list[RankedModel],
+    exclude_model_name: str,
+) -> ModelSpec | None:
+    """The best currently-recommendable model other than the one being
+    looked at — shown when that model can't run, so the user always
+    has a next step instead of a dead end."""
+
+    for entry in ranked:
+
+        if entry.model.name == exclude_model_name:
+            continue
+
+        if entry.tier in FAVORABLE_TIERS:
+            return entry.model
+
+    return None

canirunllm-0.1.0/src/canirunllm/api/schemas.py

@@ -0,0 +1,116 @@
+from pydantic import BaseModel
+
+
+class CPUResponse(BaseModel):
+    name: str
+    architecture: str
+    physical_cores: int | None
+    logical_cores: int | None
+    frequency_mhz: float | None
+
+
+class MemoryResponse(BaseModel):
+    total_bytes: int
+    available_bytes: int
+    used_bytes: int
+    usage_percent: float
+
+
+class GPUResponse(BaseModel):
+    name: str
+    memory_total_bytes: int
+    memory_used_bytes: int
+    memory_free_bytes: int
+    utilization_percent: float
+
+
+class OSResponse(BaseModel):
+    system: str
+    release: str
+    version: str
+    machine: str
+
+
+class HardwareResponse(BaseModel):
+    cpu: CPUResponse
+    memory: MemoryResponse
+    os: OSResponse
+    gpus: list[GPUResponse]
+
+
+class ModelResponse(BaseModel):
+    name: str
+    family: str
+    architecture: str
+    parameters: int
+    quantization: str
+    context_length: int
+    runtime: str | None
+    file_size_bytes: int | None
+
+
+class ReasonItemResponse(BaseModel):
+    ok: bool
+    text: str
+
+
+class CompatibilityResponse(BaseModel):
+    memory_verdict: str
+    runtime_verdict: str
+    overall_verdict: str
+    confidence: str
+    memory_strategy: str
+    required_memory_bytes: int
+    available_vram_bytes: int
+    available_ram_bytes: int
+    reason: str
+
+    # Human-facing interpretation of the fields above — generated
+    # server-side (see api/presentation.py), never in the frontend.
+    friendly_verdict: str
+    friendly_icon: str
+    reasons: list[ReasonItemResponse]
+
+
+class ModelResultResponse(BaseModel):
+    model: ModelResponse
+    compatibility: CompatibilityResponse
+
+
+class SummaryResponse(BaseModel):
+    total: int
+    can_run: int
+    can_run_with_offload: int
+    needs_validation: int
+    cannot_run: int
+
+
+class ScanResponse(BaseModel):
+    hardware: HardwareResponse
+    summary: SummaryResponse
+    results: list[ModelResultResponse]
+    recommended: list[ModelResultResponse]
+    best_for_you: ModelResultResponse | None
+    scanned_at: str
+
+
+class MemoryBreakdownResponse(BaseModel):
+    weight_memory_bytes: int
+    kv_cache_bytes: int
+    runtime_overhead_bytes: int
+    safety_margin_bytes: int
+    total_required_bytes: int
+
+
+class RunCommandResponse(BaseModel):
+    runtime: str
+    command: str
+    note: str
+
+
+class ModelDetailResponse(BaseModel):
+    model: ModelResponse
+    compatibility: CompatibilityResponse
+    memory_breakdown: MemoryBreakdownResponse
+    run_command: RunCommandResponse | None
+    alternative: ModelResponse | None