bitsandbytes-cpu-fork 0.50.2.dev0__tar.gz → 0.50.2.dev1__tar.gz

{bitsandbytes_cpu_fork-0.50.2.dev0 → bitsandbytes_cpu_fork-0.50.2.dev1}/MANIFEST.in

@@ -17,6 +17,17 @@ graft csrc
 graft build_manual
 graft docs_cpu
 graft examples
+# tools/ holds gen_api_reference.py and verify_help_examples.py, which regenerate and
+# check docs_cpu/API_REFERENCE*.md from the tables in bitsandbytes_cpu_cli.py. The
+# generated documents ARE shipped, so leaving the generators out would hand users the
+# output of scripts they cannot run.
+graft tools
+# ...but not the Windows binaries inside it. `graft tools` takes the whole directory,
+# and the four sector_*.exe disaster-recovery tools were 0.65 MB of a 1.36 MB sdist --
+# nearly half its weight, on every platform, for binaries that cannot run on two of the
+# three. Their .c sources stay in the sdist and in the repository.
+global-exclude *.exe
+global-exclude *.exe.manifest
 include build_linux.sh
 include build_termux.sh
 include selftest_cpu.c

{bitsandbytes_cpu_fork-0.50.2.dev0/bitsandbytes_cpu_fork.egg-info → bitsandbytes_cpu_fork-0.50.2.dev1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bitsandbytes-cpu-fork
-Version: 0.50.2.dev0
+Version: 0.50.2.dev1
 Summary: bitsandbytes with a pure-CPU training backend (AVX2 / NEON): 8-bit optimizers, fused GDN, gemm_8bit, disk balancer. No GPU required.
 License-Expression: MIT
 Project-URL: repository, https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU
@@ -22,14 +22,93 @@ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: torch<3,>=2.4
 Requires-Dist: numpy>=1.17
 Requires-Dist: packaging>=20.9
+Requires-Dist: psutil>=5.9
 Provides-Extra: test
 Requires-Dist: pytest~=8.3; extra == "test"
-Requires-Dist: psutil>=5.9; extra == "test"
 Dynamic: license-file
 
+> **This is the CPU fork** — `bitsandbytes-cpu-fork`, not upstream `bitsandbytes`.
+> It replaces the CUDA kernels with a pure-CPU backend (AVX2 on x86-64, NEON on ARM64):
+> 8-bit optimizers, fused Gated DeltaNet, `gemm_8bit`, block-wise 4/8-bit quantisation.
+> No GPU, no CUDA toolkit, no NVIDIA driver. Full guide:
+> [`docs_cpu/QUICKSTART.md`](docs_cpu/QUICKSTART.md) (中文) ·
+> [`docs_cpu/QUICKSTART_EN.md`](docs_cpu/QUICKSTART_EN.md).
+
+## Install
+
+**`torch` is deliberately not a dependency.** On PyPI the name `torch` resolves to the
+CUDA build, so declaring it made this CPU-only library download roughly 2.5 GB of
+`nvidia-*` wheels — cublas, cudnn, nccl, nvshmem and thirteen more — onto machines with
+no NVIDIA device at all. Measured on a clean Ubuntu install. `install_requires` cannot
+express "the CPU build", because that lives on a different index, so install it first:
+
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cpu
+pip install bitsandbytes-cpu-fork
+```
+
+### Windows
+
+A wheel is published with the CPU library already built in, so no compiler and no
+Visual C++ Redistributable are needed:
+
+```bat
+pip install bitsandbytes-cpu-fork
+```
+
+### Linux
+
+Only the source distribution is published. `pip` builds it, and `setup.py` runs
+`build_linux.sh` for you, so the library is compiled against your own glibc rather than
+shipped prebuilt. That needs a C++ compiler with OpenMP:
+
+```bash
+sudo apt install g++ libomp-dev        # Debian / Ubuntu
+sudo dnf install gcc-c++ libomp-devel  # Fedora / RHEL
+sudo pacman -S gcc openmp              # Arch
+```
+
+If the toolchain is missing the build **stops and names that command**, rather than
+producing a wheel that installs cleanly and then fails at `import bitsandbytes`.
+
+To build by hand instead, or to check the kernels without torch:
+
+```bash
+bash build_linux.sh --selftest
+```
+
+### Verify the install
+
+```bash
+python -m bitsandbytes
+```
+
+### Then ask it what it can do
+
+`pip` installs a `bitsandbytes-cpu` command (also available as `bnb-cpu`). It is the
+reference for this fork: what the hardware is, whether the kernels load, and **how to
+call every feature it adds** — each entry carries the exact call, and every Python
+example in it is executed before release.
+
+```bash
+bitsandbytes-cpu detect          # CPU model, cores, SIMD, RAM, recommended thread count
+bitsandbytes-cpu selftest        # runs a 4-bit layer, an 8-bit optimizer step, the GDN kernel
+bitsandbytes-cpu help            # the whole reference
+bitsandbytes-cpu help 8bitopt    # one section: the 8-bit optimizer
+bitsandbytes-cpu help 4bit       # one section: 4-bit layers on a CPU
+bitsandbytes-cpu help kernels    # one section: the fused kernels, called directly
+bitsandbytes-cpu doctor          # which .dll/.so will load, and which symbols it has
+```
+
+Section keys: `intro start 4bit qlora 8bitopt kernels toolkit threads memory gdn disks
+layers optim functional notes`. The same tables are published as
+[`docs_cpu/API_REFERENCE.md`](docs_cpu/API_REFERENCE.md)
+([English](docs_cpu/API_REFERENCE_EN.md)).
+
+---
+
 <p align="center"><img src="https://avatars.githubusercontent.com/u/175231607?s=200&v=4" alt=""></p>
 <h1 align="center">bitsandbytes</h1>
 <p align="center">
@@ -238,20 +317,18 @@ extensions make pure-CPU LoRA / quantization / diffusion training practical.
 
 ### Install (Windows, no compiler needed)
 
-The wheel bundles the prebuilt CPU kernel and its OpenMP runtime, so nothing has to
-be built after installing:
-
 ```bash
-pip install https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU/releases/download/v0.50.2.dev0-cpu/bitsandbytes_cpu_fork-0.50.2.dev0-py3-none-win_amd64.whl
+pip install bitsandbytes-cpu-fork
 ```
 
-One wheel covers every Python 3 version on 64-bit Windows. `pip` pulls `torch` and
-`numpy` itself. The import name is unchanged, so existing code and the
-Transformers / PEFT / Diffusers integrations keep working as-is:
+The wheel bundles the prebuilt CPU kernel and its OpenMP runtime, so nothing has to
+be built after installing; `pip` pulls `torch` and `numpy` itself. One wheel covers
+every Python 3 version on 64-bit Windows. The import name is unchanged, so existing
+code and the Transformers / PEFT / Diffusers integrations keep working as-is:
 
 ```python
 import bitsandbytes as bnb          # the fork, not upstream
-print(bnb.__version__)              # 0.50.2.dev0
+print(bnb.__version__)              # 0.50.2.dev1
 ```
 
 Building from source instead (any platform) is described in

{bitsandbytes_cpu_fork-0.50.2.dev0 → bitsandbytes_cpu_fork-0.50.2.dev1}/README.md

@@ -1,3 +1,83 @@
+> **This is the CPU fork** — `bitsandbytes-cpu-fork`, not upstream `bitsandbytes`.
+> It replaces the CUDA kernels with a pure-CPU backend (AVX2 on x86-64, NEON on ARM64):
+> 8-bit optimizers, fused Gated DeltaNet, `gemm_8bit`, block-wise 4/8-bit quantisation.
+> No GPU, no CUDA toolkit, no NVIDIA driver. Full guide:
+> [`docs_cpu/QUICKSTART.md`](docs_cpu/QUICKSTART.md) (中文) ·
+> [`docs_cpu/QUICKSTART_EN.md`](docs_cpu/QUICKSTART_EN.md).
+
+## Install
+
+**`torch` is deliberately not a dependency.** On PyPI the name `torch` resolves to the
+CUDA build, so declaring it made this CPU-only library download roughly 2.5 GB of
+`nvidia-*` wheels — cublas, cudnn, nccl, nvshmem and thirteen more — onto machines with
+no NVIDIA device at all. Measured on a clean Ubuntu install. `install_requires` cannot
+express "the CPU build", because that lives on a different index, so install it first:
+
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cpu
+pip install bitsandbytes-cpu-fork
+```
+
+### Windows
+
+A wheel is published with the CPU library already built in, so no compiler and no
+Visual C++ Redistributable are needed:
+
+```bat
+pip install bitsandbytes-cpu-fork
+```
+
+### Linux
+
+Only the source distribution is published. `pip` builds it, and `setup.py` runs
+`build_linux.sh` for you, so the library is compiled against your own glibc rather than
+shipped prebuilt. That needs a C++ compiler with OpenMP:
+
+```bash
+sudo apt install g++ libomp-dev        # Debian / Ubuntu
+sudo dnf install gcc-c++ libomp-devel  # Fedora / RHEL
+sudo pacman -S gcc openmp              # Arch
+```
+
+If the toolchain is missing the build **stops and names that command**, rather than
+producing a wheel that installs cleanly and then fails at `import bitsandbytes`.
+
+To build by hand instead, or to check the kernels without torch:
+
+```bash
+bash build_linux.sh --selftest
+```
+
+### Verify the install
+
+```bash
+python -m bitsandbytes
+```
+
+### Then ask it what it can do
+
+`pip` installs a `bitsandbytes-cpu` command (also available as `bnb-cpu`). It is the
+reference for this fork: what the hardware is, whether the kernels load, and **how to
+call every feature it adds** — each entry carries the exact call, and every Python
+example in it is executed before release.
+
+```bash
+bitsandbytes-cpu detect          # CPU model, cores, SIMD, RAM, recommended thread count
+bitsandbytes-cpu selftest        # runs a 4-bit layer, an 8-bit optimizer step, the GDN kernel
+bitsandbytes-cpu help            # the whole reference
+bitsandbytes-cpu help 8bitopt    # one section: the 8-bit optimizer
+bitsandbytes-cpu help 4bit       # one section: 4-bit layers on a CPU
+bitsandbytes-cpu help kernels    # one section: the fused kernels, called directly
+bitsandbytes-cpu doctor          # which .dll/.so will load, and which symbols it has
+```
+
+Section keys: `intro start 4bit qlora 8bitopt kernels toolkit threads memory gdn disks
+layers optim functional notes`. The same tables are published as
+[`docs_cpu/API_REFERENCE.md`](docs_cpu/API_REFERENCE.md)
+([English](docs_cpu/API_REFERENCE_EN.md)).
+
+---
+
 <p align="center"><img src="https://avatars.githubusercontent.com/u/175231607?s=200&v=4" alt=""></p>
 <h1 align="center">bitsandbytes</h1>
 <p align="center">
@@ -206,20 +286,18 @@ extensions make pure-CPU LoRA / quantization / diffusion training practical.
 
 ### Install (Windows, no compiler needed)
 
-The wheel bundles the prebuilt CPU kernel and its OpenMP runtime, so nothing has to
-be built after installing:
-
 ```bash
-pip install https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU/releases/download/v0.50.2.dev0-cpu/bitsandbytes_cpu_fork-0.50.2.dev0-py3-none-win_amd64.whl
+pip install bitsandbytes-cpu-fork
 ```
 
-One wheel covers every Python 3 version on 64-bit Windows. `pip` pulls `torch` and
-`numpy` itself. The import name is unchanged, so existing code and the
-Transformers / PEFT / Diffusers integrations keep working as-is:
+The wheel bundles the prebuilt CPU kernel and its OpenMP runtime, so nothing has to
+be built after installing; `pip` pulls `torch` and `numpy` itself. One wheel covers
+every Python 3 version on 64-bit Windows. The import name is unchanged, so existing
+code and the Transformers / PEFT / Diffusers integrations keep working as-is:
 
 ```python
 import bitsandbytes as bnb          # the fork, not upstream
-print(bnb.__version__)              # 0.50.2.dev0
+print(bnb.__version__)              # 0.50.2.dev1
 ```
 
 Building from source instead (any platform) is described in

{bitsandbytes_cpu_fork-0.50.2.dev0 → bitsandbytes_cpu_fork-0.50.2.dev1}/bitsandbytes/__init__.py

@@ -75,7 +75,7 @@ __pdoc__ = {
     "optim.optimizer.MockArgs": False,
 }
 
-__version__ = "0.50.2.dev0"
+__version__ = "0.50.2.dev1"
 
 # 挂载 gdn_cpu（GDN 融合内核 / patch_transformers 入口），让 bnb.gdn_cpu 可直接访问
 from . import gdn_cpu  # noqa: E402,F401

bitsandbytes_cpu_fork-0.50.2.dev1/bitsandbytes/cli.py

@@ -0,0 +1,50 @@
+"""Compatibility shim. The reference itself lives in `bitsandbytes_cpu_cli`.
+
+Why it is not here: `bitsandbytes/__init__.py` imports torch, and torch is deliberately
+not a dependency of this distribution. So this module can only be imported on a machine
+that already has torch -- which is not the machine whose user needs the help. The console
+script therefore points at the top-level `bitsandbytes_cpu_cli`, which uses nothing but
+the standard library.
+
+This file is kept so that `python -m bitsandbytes.cli help` and code that imports the
+tables from here keep working once torch is installed.
+"""
+from __future__ import annotations
+
+import sys
+
+from bitsandbytes_cpu_cli import (  # noqa: F401
+    EXAMPLES,
+    PY,
+    SECTIONS,
+    SH,
+    USAGE,
+    WIDTH,
+    _row,
+    _select,
+    _wrap,
+    cmd_detect,
+    cmd_doctor,
+    cmd_help,
+    cmd_selftest,
+    cmd_version,
+    main,
+    render_markdown,
+    render_text,
+)
+
+__all__ = [
+    "EXAMPLES",
+    "PY",
+    "SECTIONS",
+    "SH",
+    "USAGE",
+    "WIDTH",
+    "main",
+    "render_markdown",
+    "render_text",
+]
+
+
+if __name__ == "__main__":
+    sys.exit(main())

bitsandbytes_cpu_fork-0.50.2.dev1/bitsandbytes/detect_cpu.py

@@ -0,0 +1,189 @@
+# -*- coding: utf-8 -*-
+"""detect_cpu.py — 一键检测 CPU 能力 + 推荐配置（开源小白友好）。
+
+自动检测并推荐：
+  - CPU 型号 / 物理核 / 逻辑核 / AVX2 / AVX-512
+  - 内存总量/可用
+  - 推荐线程数（按 GEMM/conv 任务类型）
+  - 是否建议用 bnb 8-bit 优化器 / 量化基座 / disk_balancer
+输出一行可用作配置的推荐摘要，可被训练脚本直接使用。
+
+用法：
+    python detect_cpu.py                 # 打印全部检测 + 推荐
+    python detect_cpu.py --json          # 输出 JSON（供脚本解析）
+"""
+import argparse
+import json
+import os
+import platform
+
+
+def _cpu_model() -> str:
+    """The CPU's name, not the first line of whatever tool we asked.
+
+    `lscpu` was run and its FIRST line taken, which on every Linux is
+    "Architecture: x86_64" -- the report said the CPU was a machine architecture. The
+    model name is on the "Model name:" line; /proc/cpuinfo carries it too, and does not
+    depend on util-linux being installed.
+
+    Windows keeps the wmic path but parses the value out rather than the first line,
+    since `wmic ... /value` emits "Key=Value" lines whose order is not guaranteed.
+    """
+    try:
+        import subprocess
+        if os.name == "nt":
+            # Registry first. `wmic` has been removed from current Windows builds, and
+            # when it is missing the old code fell through to platform.processor(),
+            # which reports "AMD64 Family 23 Model 96 Stepping 1, AuthenticAMD" -- a
+            # description of the CPU's family, not its name. The registry key is where
+            # the name actually lives and needs no subprocess.
+            try:
+                import winreg
+                k = winreg.OpenKey(
+                    winreg.HKEY_LOCAL_MACHINE,
+                    r"HARDWARE\DESCRIPTION\System\CentralProcessor\0")
+                return winreg.QueryValueEx(k, "ProcessorNameString")[0].strip()
+            except Exception:
+                pass
+            out = subprocess.run(
+                ["wmic", "cpu", "get", "Name", "/value"],
+                capture_output=True, text=True, timeout=15).stdout
+            for line in out.splitlines():
+                if line.strip().lower().startswith("name="):
+                    return line.split("=", 1)[1].strip()
+        else:
+            # /proc/cpuinfo first: no dependency on lscpu being present.
+            try:
+                with open("/proc/cpuinfo", encoding="utf-8", errors="replace") as fh:
+                    for line in fh:
+                        if line.lower().startswith("model name"):
+                            return line.split(":", 1)[1].strip()
+            except OSError:
+                pass
+            out = subprocess.run(["lscpu"], capture_output=True, text=True,
+                                 timeout=15).stdout
+            for line in out.splitlines():
+                if line.lower().startswith("model name"):
+                    return line.split(":", 1)[1].strip()
+    except Exception:
+        pass
+    return platform.processor() or "unknown-cpu"
+
+
+def _physical_cores() -> int:
+    """Physical cores, not logical ones: an SMT machine reports twice as many threads
+    and the recommendation below depends on the difference.
+
+    Tries the package-relative import first, because this module now ships inside
+    `bitsandbytes`. The bare import is kept for the checkout layout, where the two files
+    sit next to each other at the repository root and there is no package to be relative
+    to. Falling back to os.cpu_count() reports LOGICAL cores, which on an SMT machine
+    makes the recommendation wrong -- so that is the last resort, not the first.
+    """
+    try:
+        from . import torch_cpu_kit as tck
+        return tck.physical_cores()
+    except Exception:
+        pass
+    try:
+        import torch_cpu_kit as tck
+        return tck.physical_cores()
+    except Exception:
+        return os.cpu_count() or 1
+
+
+def _mem() -> tuple[float, float]:
+    try:
+        import psutil
+        vm = psutil.virtual_memory()
+        return vm.total / 1e9, vm.available / 1e9
+    except Exception:
+        return 0.0, 0.0
+
+
+def _cpu_capability() -> str:
+    try:
+        import torch
+        return torch.backends.cpu.get_cpu_capability()
+    except Exception:
+        return "unknown"
+
+
+def _recommend_threads(logical: int, physical: int) -> dict:
+    """Recommend a thread count per task, from measurements rather than from core count.
+
+    The previous version returned `logical` for diffusion on any SMT machine, on the
+    stated basis that "conv benefits from hyperthreading". Measured on the i5-10400
+    (6 physical / 12 logical), that is wrong for the workload this project actually
+    runs. The Qwen-Image-2.1 step is 82% dense GEMM, and on the four shapes that model
+    uses, six threads beat twelve on every one:
+
+        16384x4096x4096    6T 1.316 s    12T 1.449 s
+        16384x4096x12288   6T 4.048 s    12T 4.325 s
+         4096x4096x4096    6T 0.335 s    12T 0.390 s
+        GFLOPS              6T 407-418    12T 352-381
+
+    A whole 1024x1024 render agreed: 225 s/step at 12 threads against 203 at 6. The
+    second thread on a physical core shares its FP units and halves its cache, which a
+    compute-bound GEMM cannot absorb.
+
+    So: physical cores everywhere. The `logical` value is still returned under a
+    separate key for callers that know their workload is bandwidth- or latency-bound
+    rather than arithmetic-bound.
+    """
+    return {"lm": physical,
+            "diffusion": physical,
+            "image": physical,
+            "logical_if_bandwidth_bound": logical}
+
+
+def detect() -> dict:
+    logical = os.cpu_count() or 1
+    physical = _physical_cores()
+    total_gb, avail_gb = _mem()
+    cap = _cpu_capability()
+    r = _recommend_threads(logical, physical)
+    return {
+        "cpu_model": _cpu_model(),
+        "physical_cores": physical,
+        "logical_cores": logical,
+        "cpu_capability": cap,           # 如 AVX2 / AVX512
+        "has_avx2": "AVX2" in cap or "AVX512" in cap or "AVXNEON" in cap,
+        "has_avx512": "AVX512" in cap,
+        "ram_total_gb": round(total_gb, 1),
+        "ram_available_gb": round(avail_gb, 1),
+        "recommend_threads": r,
+        "recommend_bf16": "BF16" in cap,   # AVX512-BF16/AMX 才建议 bf16
+        "recommend_8bit_optimizer": True,  # 纯 CPU 存储紧张时 8bit 优化器省内存
+        # 内存读不到时（没装 psutil）不要给 True：avail_gb 是 0，< 8 会成立，
+        # 于是这个推荐就变成了一个由缺失依赖制造的假警报。
+        "recommend_disk_balancer": bool(avail_gb) and avail_gb < 8,
+    }
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--json", action="store_true")
+    a = ap.parse_args()
+    d = detect()
+    if a.json:
+        print(json.dumps(d, ensure_ascii=False, indent=2))
+        return
+    print("=== CPU 能力检测 ===")
+    print(f"  CPU: {d['cpu_model']}")
+    print(f"  物理核/逻辑核: {d['physical_cores']}/{d['logical_cores']}")
+    print(f"  SIMD 能力: {d['cpu_capability']}  (AVX2={d['has_avx2']}, AVX-512={d['has_avx512']})")
+    print(f"  内存: 总 {d['ram_total_gb']}GB / 可用 {d['ram_available_gb']}GB")
+    print("\n=== 推荐配置（实测依据）===")
+    t = d["recommend_threads"]
+    print(f"  LLM 训练线程数 : {t['lm']}（GEMM 密集，超线程只会抢同一个核的 FP 单元）")
+    print(f"  生图线程数     : {t['diffusion']}（Qwen-Image 一步 82% 是 GEMM；i5 实测 "
+          f"6 线程 203 s/step vs 12 线程 225）")
+    print(f"  带宽受限时可用 : {t['logical_if_bandwidth_bound']}（仅当负载不是算术受限）")
+    print(f"  推荐 bf16      : {d['recommend_bf16']}（AVX2-only 机器应用 fp32）")
+    print(f"  推荐 8bit 优化器: {d['recommend_8bit_optimizer']}（内存紧张时省内存）")
+    print(f"  推荐 disk_balancer: {d['recommend_disk_balancer']}（内存<8GB 建议开 --flash）")
+
+
+if __name__ == "__main__":
+    main()