bitsandbytes-cpu-fork 0.50.2.dev5__tar.gz → 0.50.2.dev6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {bitsandbytes_cpu_fork-0.50.2.dev5/bitsandbytes_cpu_fork.egg-info → bitsandbytes_cpu_fork-0.50.2.dev6}/PKG-INFO +2 -2
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/README.md +1 -1
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/__init__.py +1 -1
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6/bitsandbytes_cpu_fork.egg-info}/PKG-INFO +2 -2
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/SOURCES.txt +1 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_cache.h +3 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_features.h +6 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_ops.cpp +376 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_ops.h +34 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/pythonInterface.cpp +49 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/EPYC_VALIDATION.md +8 -0
- bitsandbytes_cpu_fork-0.50.2.dev6/docs_cpu/ISA_MATRIX.md +79 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/WHEEL_PLATFORM_NOTES.md +26 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/selftest_cpu.c +448 -269
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/LICENSE +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/MANIFEST.in +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/__main__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/_cpu_caps.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/_ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/autograd/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/autograd/_functions.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cpu/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cpu/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cuda/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cuda/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/default/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/default/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/hpu/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/hpu/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/mps/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/mps/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/kernels_4bit.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/kernels_8bit_quant.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/kernels_optim.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/utils.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/xpu/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/xpu/ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/cextension.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/cli.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/consts.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/cuda_specs.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/detect_cpu.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/cuda.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/main.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/utils.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/disk_balancer.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/efst.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/functional.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/gdn_cpu.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/libbitsandbytes_cpu.dll +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/nn/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/nn/modules.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/nn/parametrize.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/__init__.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/adagrad.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/adam.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/adamw.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/ademamix.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/lamb.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/lars.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/lion.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/optimizer.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/rmsprop.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/sgd.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/py.typed +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/torch_cpu_kit.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/utils.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/vcomp140.dll +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_cli.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/dependency_links.txt +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/entry_points.txt +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/requires.txt +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/top_level.txt +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_linux.sh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/build_manual.bat +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/export.def +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/export_avx512.def +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/export_generic.def +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/selftest_win.bat +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_termux.sh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/avx2_gemv_4bit.h +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/common.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/common.h +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/compat.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/compat_device.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_gdn.cpp +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit.cu +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_common.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_mma.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_simt.cu +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_simt.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm75.cu +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm75.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm80.cu +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm80.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/kernels.cu +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/kernels.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/ops.cu +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/ops.cuh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/test_avx2_gemv_4bit.cpp +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/test_cpu_cache.cpp +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_kernels.cpp +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_kernels.h +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_ops.cpp +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_ops.h +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/API_REFERENCE.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/API_REFERENCE_EN.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/MOTION_BOUNDARY.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/MOTION_BOUNDARY_EN.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/QUICKSTART.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/QUICKSTART_EN.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECHNICAL_GUIDE.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECHNICAL_GUIDE_EN.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECH_REPORT.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECH_REPORT_EN.md +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/check_docs.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/compile_inference.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/cpu/cpu_training.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/int8_inference_huggingface.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/xpu/benchmark_paged_memory.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/xpu/paged_xpu_training.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/pyproject.toml +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/setup.cfg +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/setup.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_autograd.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_cpu_e2e.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_cuda_setup_evaluator.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_functional.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_generation.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_linear4bit.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_linear8bitlt.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_modules.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_ops.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_optim.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_parametrize.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_rmsnorm_fwd_cpu.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/create_github_release.ps1 +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/dump_signatures.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/gen_api_reference.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/publish.ps1 +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_carve.c +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_carve_gui.c +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_mirror.c +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_mirror_gui.c +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/train_llm_sft.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/verify_help_examples.py +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/verify_linux_cli.sh +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/verify_train_release.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: bitsandbytes-cpu-fork
|
|
3
|
-
Version: 0.50.2.
|
|
3
|
+
Version: 0.50.2.dev6
|
|
4
4
|
Summary: bitsandbytes with a pure-CPU training backend (AVX2 / NEON): 8-bit optimizers, fused GDN, gemm_8bit, disk balancer. No GPU required.
|
|
5
5
|
License-Expression: MIT
|
|
6
6
|
Project-URL: repository, https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU
|
|
@@ -336,7 +336,7 @@ code and the Transformers / PEFT / Diffusers integrations keep working as-is:
|
|
|
336
336
|
|
|
337
337
|
```python
|
|
338
338
|
import bitsandbytes as bnb # the fork, not upstream
|
|
339
|
-
print(bnb.__version__) # 0.50.2.
|
|
339
|
+
print(bnb.__version__) # 0.50.2.dev6
|
|
340
340
|
```
|
|
341
341
|
|
|
342
342
|
Building from source instead (any platform) is described in
|
|
@@ -305,7 +305,7 @@ code and the Transformers / PEFT / Diffusers integrations keep working as-is:
|
|
|
305
305
|
|
|
306
306
|
```python
|
|
307
307
|
import bitsandbytes as bnb # the fork, not upstream
|
|
308
|
-
print(bnb.__version__) # 0.50.2.
|
|
308
|
+
print(bnb.__version__) # 0.50.2.dev6
|
|
309
309
|
```
|
|
310
310
|
|
|
311
311
|
Building from source instead (any platform) is described in
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: bitsandbytes-cpu-fork
|
|
3
|
-
Version: 0.50.2.
|
|
3
|
+
Version: 0.50.2.dev6
|
|
4
4
|
Summary: bitsandbytes with a pure-CPU training backend (AVX2 / NEON): 8-bit optimizers, fused GDN, gemm_8bit, disk balancer. No GPU required.
|
|
5
5
|
License-Expression: MIT
|
|
6
6
|
Project-URL: repository, https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU
|
|
@@ -336,7 +336,7 @@ code and the Transformers / PEFT / Diffusers integrations keep working as-is:
|
|
|
336
336
|
|
|
337
337
|
```python
|
|
338
338
|
import bitsandbytes as bnb # the fork, not upstream
|
|
339
|
-
print(bnb.__version__) # 0.50.2.
|
|
339
|
+
print(bnb.__version__) # 0.50.2.dev6
|
|
340
340
|
```
|
|
341
341
|
|
|
342
342
|
Building from source instead (any platform) is described in
|
|
@@ -59,8 +59,11 @@
|
|
|
59
59
|
#include <intrin.h>
|
|
60
60
|
#include <windows.h>
|
|
61
61
|
#elif defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))
|
|
62
|
+
#ifndef BNB_CPUID_H_INCLUDED
|
|
63
|
+
#define BNB_CPUID_H_INCLUDED
|
|
62
64
|
#include <cpuid.h>
|
|
63
65
|
#endif
|
|
66
|
+
#endif
|
|
64
67
|
|
|
65
68
|
namespace bnb_cache_detail {
|
|
66
69
|
|
|
@@ -13,7 +13,13 @@
|
|
|
13
13
|
|
|
14
14
|
#if (defined(__x86_64__) || defined(__i386__)) && !defined(_MSC_VER)
|
|
15
15
|
|
|
16
|
+
#ifndef BNB_CPUID_H_INCLUDED
|
|
17
|
+
#define BNB_CPUID_H_INCLUDED
|
|
18
|
+
// GCC 8's <cpuid.h> has no include guard, so include it exactly once across the build
|
|
19
|
+
// (cpu_features.h and cpu_cache.h both need it; without this sentinel old GCC errors
|
|
20
|
+
// with "redefinition of __get_cpuid_count").
|
|
16
21
|
#include <cpuid.h>
|
|
22
|
+
#endif
|
|
17
23
|
// clang's <cpuid.h> defines __cpuid/__cpuid_count as function-like macros. Windows
|
|
18
24
|
// headers (intrin.h -> psdk_inc/intrin-impl.h, pulled in later via cpu_cache.h) declare
|
|
19
25
|
// void __cpuid(int[4], int), which those macros then break ("too few arguments provided
|
|
@@ -4036,3 +4036,379 @@ long long fused_add_scale_rmsnorm_cpu(
|
|
|
4036
4036
|
// 第二步:对 x 做 RMSNorm(x 现在就是 h),res 传 NULL
|
|
4037
4037
|
return fused_rmsnorm_fwd_cpu(x, nullptr, weight, out, xs, M, D, eps);
|
|
4038
4038
|
}
|
|
4039
|
+
|
|
4040
|
+
// ---------------------------------------------------------------------------
|
|
4041
|
+
// 2-/3-bit blockwise dequantization (research prototype, see q3bit_proto.py).
|
|
4042
|
+
// The scalar reference is cross-checked against the vector paths by selftest_cpu.
|
|
4043
|
+
// ---------------------------------------------------------------------------
|
|
4044
|
+
static inline float bnb_q3bit_value(int code, float scale, float amax) {
|
|
4045
|
+
return (float)code * scale - amax;
|
|
4046
|
+
}
|
|
4047
|
+
|
|
4048
|
+
void dequantize_3bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
4049
|
+
long long n_blocks, long long block) {
|
|
4050
|
+
for (long long b = 0; b < n_blocks; ++b) {
|
|
4051
|
+
const unsigned char* p = packed + b * (block / 8) * 3;
|
|
4052
|
+
const float amax = absmax[b];
|
|
4053
|
+
const float scale = amax / 3.5f;
|
|
4054
|
+
float* dst = out + b * block;
|
|
4055
|
+
for (long long g = 0; g < block; g += 8) {
|
|
4056
|
+
const unsigned char* q = p + (g / 8) * 3;
|
|
4057
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
|
|
4058
|
+
for (int j = 0; j < 8; ++j) {
|
|
4059
|
+
dst[g + j] = bnb_q3bit_value((int)((v >> (3 * j)) & 7), scale, amax);
|
|
4060
|
+
}
|
|
4061
|
+
}
|
|
4062
|
+
}
|
|
4063
|
+
}
|
|
4064
|
+
|
|
4065
|
+
void dequantize_2bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
4066
|
+
long long n_blocks, long long block) {
|
|
4067
|
+
for (long long b = 0; b < n_blocks; ++b) {
|
|
4068
|
+
const unsigned char* p = packed + b * (block / 4);
|
|
4069
|
+
const float amax = absmax[b];
|
|
4070
|
+
const float scale = amax / 1.5f;
|
|
4071
|
+
float* dst = out + b * block;
|
|
4072
|
+
for (long long g = 0; g < block; g += 8) {
|
|
4073
|
+
const unsigned char* q = p + (g / 4);
|
|
4074
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8);
|
|
4075
|
+
for (int j = 0; j < 8; ++j) {
|
|
4076
|
+
dst[g + j] = bnb_q3bit_value((int)((v >> (2 * j)) & 3), scale, amax);
|
|
4077
|
+
}
|
|
4078
|
+
}
|
|
4079
|
+
}
|
|
4080
|
+
}
|
|
4081
|
+
|
|
4082
|
+
#if BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
|
|
4083
|
+
static inline void bnb_dequant_3bit_avx2(const unsigned char* packed, const float* absmax,
|
|
4084
|
+
float* out, long long n_blocks, long long block) {
|
|
4085
|
+
for (long long b = 0; b < n_blocks; ++b) {
|
|
4086
|
+
const unsigned char* p = packed + b * (block / 8) * 3;
|
|
4087
|
+
const float amax = absmax[b];
|
|
4088
|
+
const float scale = amax / 3.5f;
|
|
4089
|
+
const __m256 vs = _mm256_set1_ps(scale);
|
|
4090
|
+
const __m256 va = _mm256_set1_ps(amax);
|
|
4091
|
+
float* dst = out + b * block;
|
|
4092
|
+
for (long long g = 0; g < block; g += 8) {
|
|
4093
|
+
const unsigned char* q = p + (g / 8) * 3;
|
|
4094
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
|
|
4095
|
+
__m256i codes = _mm256_setr_epi32(
|
|
4096
|
+
(int)(v & 7), (int)((v >> 3) & 7), (int)((v >> 6) & 7), (int)((v >> 9) & 7),
|
|
4097
|
+
(int)((v >> 12) & 7), (int)((v >> 15) & 7), (int)((v >> 18) & 7),
|
|
4098
|
+
(int)((v >> 21) & 7));
|
|
4099
|
+
__m256 f = _mm256_sub_ps(_mm256_mul_ps(_mm256_cvtepi32_ps(codes), vs), va);
|
|
4100
|
+
_mm256_storeu_ps(dst + g, f);
|
|
4101
|
+
}
|
|
4102
|
+
}
|
|
4103
|
+
}
|
|
4104
|
+
|
|
4105
|
+
static inline void bnb_dequant_2bit_avx2(const unsigned char* packed, const float* absmax,
|
|
4106
|
+
float* out, long long n_blocks, long long block) {
|
|
4107
|
+
for (long long b = 0; b < n_blocks; ++b) {
|
|
4108
|
+
const unsigned char* p = packed + b * (block / 4);
|
|
4109
|
+
const float amax = absmax[b];
|
|
4110
|
+
const float scale = amax / 1.5f;
|
|
4111
|
+
const __m256 vs = _mm256_set1_ps(scale);
|
|
4112
|
+
const __m256 va = _mm256_set1_ps(amax);
|
|
4113
|
+
float* dst = out + b * block;
|
|
4114
|
+
for (long long g = 0; g < block; g += 8) {
|
|
4115
|
+
const unsigned char* q = p + (g / 4);
|
|
4116
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8);
|
|
4117
|
+
__m256i codes = _mm256_setr_epi32(
|
|
4118
|
+
(int)(v & 3), (int)((v >> 2) & 3), (int)((v >> 4) & 3), (int)((v >> 6) & 3),
|
|
4119
|
+
(int)((v >> 8) & 3), (int)((v >> 10) & 3), (int)((v >> 12) & 3),
|
|
4120
|
+
(int)((v >> 14) & 3));
|
|
4121
|
+
__m256 f = _mm256_sub_ps(_mm256_mul_ps(_mm256_cvtepi32_ps(codes), vs), va);
|
|
4122
|
+
_mm256_storeu_ps(dst + g, f);
|
|
4123
|
+
}
|
|
4124
|
+
}
|
|
4125
|
+
}
|
|
4126
|
+
#endif
|
|
4127
|
+
|
|
4128
|
+
#if defined(_M_ARM64) || defined(__aarch64__)
|
|
4129
|
+
static inline void bnb_dequant_3bit_neon(const unsigned char* packed, const float* absmax,
|
|
4130
|
+
float* out, long long n_blocks, long long block) {
|
|
4131
|
+
for (long long b = 0; b < n_blocks; ++b) {
|
|
4132
|
+
const unsigned char* p = packed + b * (block / 8) * 3;
|
|
4133
|
+
const float amax = absmax[b];
|
|
4134
|
+
const float scale = amax / 3.5f;
|
|
4135
|
+
const float32x4_t vs = vdupq_n_f32(scale);
|
|
4136
|
+
const float32x4_t va = vdupq_n_f32(amax);
|
|
4137
|
+
float* dst = out + b * block;
|
|
4138
|
+
for (long long g = 0; g < block; g += 8) {
|
|
4139
|
+
const unsigned char* q = p + (g / 8) * 3;
|
|
4140
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
|
|
4141
|
+
uint32_t codes[8];
|
|
4142
|
+
for (int j = 0; j < 8; ++j) codes[j] = (v >> (3 * j)) & 7;
|
|
4143
|
+
float32x4_t c0 = vcvtq_f32_u32(vld1q_u32(codes));
|
|
4144
|
+
float32x4_t c1 = vcvtq_f32_u32(vld1q_u32(codes + 4));
|
|
4145
|
+
vst1q_f32(dst + g, vsubq_f32(vmulq_f32(c0, vs), va));
|
|
4146
|
+
vst1q_f32(dst + g + 4, vsubq_f32(vmulq_f32(c1, vs), va));
|
|
4147
|
+
}
|
|
4148
|
+
}
|
|
4149
|
+
}
|
|
4150
|
+
|
|
4151
|
+
static inline void bnb_dequant_2bit_neon(const unsigned char* packed, const float* absmax,
|
|
4152
|
+
float* out, long long n_blocks, long long block) {
|
|
4153
|
+
for (long long b = 0; b < n_blocks; ++b) {
|
|
4154
|
+
const unsigned char* p = packed + b * (block / 4);
|
|
4155
|
+
const float amax = absmax[b];
|
|
4156
|
+
const float scale = amax / 1.5f;
|
|
4157
|
+
const float32x4_t vs = vdupq_n_f32(scale);
|
|
4158
|
+
const float32x4_t va = vdupq_n_f32(amax);
|
|
4159
|
+
float* dst = out + b * block;
|
|
4160
|
+
for (long long g = 0; g < block; g += 8) {
|
|
4161
|
+
const unsigned char* q = p + (g / 4);
|
|
4162
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8);
|
|
4163
|
+
uint32_t codes[8];
|
|
4164
|
+
for (int j = 0; j < 8; ++j) codes[j] = (v >> (2 * j)) & 3;
|
|
4165
|
+
float32x4_t c0 = vcvtq_f32_u32(vld1q_u32(codes));
|
|
4166
|
+
float32x4_t c1 = vcvtq_f32_u32(vld1q_u32(codes + 4));
|
|
4167
|
+
vst1q_f32(dst + g, vsubq_f32(vmulq_f32(c0, vs), va));
|
|
4168
|
+
vst1q_f32(dst + g + 4, vsubq_f32(vmulq_f32(c1, vs), va));
|
|
4169
|
+
}
|
|
4170
|
+
}
|
|
4171
|
+
}
|
|
4172
|
+
#endif
|
|
4173
|
+
|
|
4174
|
+
void dequantize_3bit_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
4175
|
+
long long n_blocks, long long block) {
|
|
4176
|
+
#if defined(_M_ARM64) || defined(__aarch64__)
|
|
4177
|
+
bnb_dequant_3bit_neon(packed, absmax, out, n_blocks, block);
|
|
4178
|
+
#elif BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
|
|
4179
|
+
bnb_dequant_3bit_avx2(packed, absmax, out, n_blocks, block);
|
|
4180
|
+
#else
|
|
4181
|
+
dequantize_3bit_scalar_cpu(packed, absmax, out, n_blocks, block);
|
|
4182
|
+
#endif
|
|
4183
|
+
}
|
|
4184
|
+
|
|
4185
|
+
void dequantize_2bit_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
4186
|
+
long long n_blocks, long long block) {
|
|
4187
|
+
#if defined(_M_ARM64) || defined(__aarch64__)
|
|
4188
|
+
bnb_dequant_2bit_neon(packed, absmax, out, n_blocks, block);
|
|
4189
|
+
#elif BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
|
|
4190
|
+
bnb_dequant_2bit_avx2(packed, absmax, out, n_blocks, block);
|
|
4191
|
+
#else
|
|
4192
|
+
dequantize_2bit_scalar_cpu(packed, absmax, out, n_blocks, block);
|
|
4193
|
+
#endif
|
|
4194
|
+
}
|
|
4195
|
+
|
|
4196
|
+
// ---------------------------------------------------------------------------
|
|
4197
|
+
// Fused 3-bit dequant + GEMM (research prototype promoted to csrc).
|
|
4198
|
+
// C[M,N] = A[M,K] @ dequant(B)[N,K]^T. B is packed per row: nblocks blocks of
|
|
4199
|
+
// `block` elements (block % 8 == 0), each block block/8*3 bytes plus one fp32 absmax.
|
|
4200
|
+
// A B-row tile is dequantized into a small buffer (L2-resident) and the dot products run
|
|
4201
|
+
// from there, so the full fp32 table (e.g. 0.95 GB lm_head) is never materialized.
|
|
4202
|
+
// ---------------------------------------------------------------------------
|
|
4203
|
+
static inline void bnb_q3bit_dot(const float* a, const float* b, int K, float* out) {
|
|
4204
|
+
#if BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
|
|
4205
|
+
__m256 acc = _mm256_setzero_ps();
|
|
4206
|
+
int k = 0;
|
|
4207
|
+
for (; k + 8 <= K; k += 8) {
|
|
4208
|
+
acc = _mm256_fmadd_ps(_mm256_loadu_ps(a + k), _mm256_loadu_ps(b + k), acc);
|
|
4209
|
+
}
|
|
4210
|
+
__m128 lo = _mm256_castps256_ps128(acc);
|
|
4211
|
+
__m128 hi = _mm256_extractf128_ps(acc, 1);
|
|
4212
|
+
lo = _mm_add_ps(lo, hi);
|
|
4213
|
+
__m128 shuf = _mm_movehdup_ps(lo);
|
|
4214
|
+
__m128 sums = _mm_add_ps(lo, shuf);
|
|
4215
|
+
shuf = _mm_movehl_ps(shuf, sums);
|
|
4216
|
+
sums = _mm_add_ss(sums, shuf);
|
|
4217
|
+
float total = _mm_cvtss_f32(sums);
|
|
4218
|
+
for (; k < K; ++k) total += a[k] * b[k];
|
|
4219
|
+
*out = total;
|
|
4220
|
+
#else
|
|
4221
|
+
float total = 0.0f;
|
|
4222
|
+
for (int k = 0; k < K; ++k) total += a[k] * b[k];
|
|
4223
|
+
*out = total;
|
|
4224
|
+
#endif
|
|
4225
|
+
}
|
|
4226
|
+
|
|
4227
|
+
static void bnb_q3bit_gemm_worker(const float* A, int M, const unsigned char* packed,
|
|
4228
|
+
const float* absmax, float* C, int N, int K, int block,
|
|
4229
|
+
int ntile, int row_start, int row_end) {
|
|
4230
|
+
const int nblocks = K / block;
|
|
4231
|
+
const int row_stride = (block / 8) * 3;
|
|
4232
|
+
std::vector<float> buf((size_t)ntile * K);
|
|
4233
|
+
for (int n0 = row_start; n0 < row_end; n0 += ntile) {
|
|
4234
|
+
const int nt = std::min(ntile, row_end - n0);
|
|
4235
|
+
for (int t = 0; t < nt; ++t) {
|
|
4236
|
+
const unsigned char* prow = packed + (size_t)(n0 + t) * nblocks * row_stride;
|
|
4237
|
+
const float* amax_row = absmax + (size_t)(n0 + t) * nblocks;
|
|
4238
|
+
float* dst = buf.data() + (size_t)t * K;
|
|
4239
|
+
for (int b = 0; b < nblocks; ++b) {
|
|
4240
|
+
dequantize_3bit_cpu(prow + (size_t)b * row_stride, amax_row + b,
|
|
4241
|
+
dst + (size_t)b * block, 1, block);
|
|
4242
|
+
}
|
|
4243
|
+
}
|
|
4244
|
+
for (int m = 0; m < M; ++m) {
|
|
4245
|
+
const float* arow = A + (size_t)m * K;
|
|
4246
|
+
float* crow = C + (size_t)m * N + n0;
|
|
4247
|
+
for (int t = 0; t < nt; ++t) {
|
|
4248
|
+
bnb_q3bit_dot(arow, buf.data() + (size_t)t * K, K, crow + t);
|
|
4249
|
+
}
|
|
4250
|
+
}
|
|
4251
|
+
}
|
|
4252
|
+
}
|
|
4253
|
+
|
|
4254
|
+
void q3bit_gemm_cpu(const float* A, int M, const unsigned char* packed, const float* absmax,
|
|
4255
|
+
float* C, int N, int K, int block, int nthreads, int ntile) {
|
|
4256
|
+
if (!A || !packed || !absmax || !C || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
|
|
4257
|
+
(block % 8) != 0 || (K % block) != 0) {
|
|
4258
|
+
return;
|
|
4259
|
+
}
|
|
4260
|
+
if (ntile <= 0) ntile = 64;
|
|
4261
|
+
if (nthreads <= 0) {
|
|
4262
|
+
nthreads = (int)std::thread::hardware_concurrency();
|
|
4263
|
+
if (nthreads <= 0) nthreads = 1;
|
|
4264
|
+
}
|
|
4265
|
+
std::vector<std::thread> threads;
|
|
4266
|
+
threads.reserve((size_t)nthreads);
|
|
4267
|
+
for (int th = 0; th < nthreads; ++th) {
|
|
4268
|
+
const int row_start = (int)((long long)N * th / nthreads);
|
|
4269
|
+
const int row_end = (int)((long long)N * (th + 1) / nthreads);
|
|
4270
|
+
if (row_start >= row_end) continue;
|
|
4271
|
+
threads.emplace_back(bnb_q3bit_gemm_worker, A, M, packed, absmax, C, N, K, block,
|
|
4272
|
+
ntile, row_start, row_end);
|
|
4273
|
+
}
|
|
4274
|
+
for (auto& th : threads) th.join();
|
|
4275
|
+
}
|
|
4276
|
+
|
|
4277
|
+
// ---------------------------------------------------------------------------
|
|
4278
|
+
// Transposed variant for training backward: dX[M,K] = dY[M,N] @ dequant(B)[N,K].
|
|
4279
|
+
// Each thread accumulates a private dX (M*K floats, small) over its B-row range; the
|
|
4280
|
+
// partials are summed at the end, so there is no locking in the hot loop.
|
|
4281
|
+
// ---------------------------------------------------------------------------
|
|
4282
|
+
struct BnbQ3GemmTCtx {
|
|
4283
|
+
const float* dY;
|
|
4284
|
+
const unsigned char* packed;
|
|
4285
|
+
const float* absmax;
|
|
4286
|
+
int M, N, K, block, ntile, row_start, row_end, row_stride;
|
|
4287
|
+
};
|
|
4288
|
+
|
|
4289
|
+
static void bnb_q3bit_gemm_t_worker(const BnbQ3GemmTCtx* ctx, float* part) {
|
|
4290
|
+
const int nblocks = ctx->K / ctx->block;
|
|
4291
|
+
std::vector<float> buf((size_t)ctx->ntile * ctx->K);
|
|
4292
|
+
for (size_t i = 0; i < (size_t)ctx->M * ctx->K; ++i) part[i] = 0.0f;
|
|
4293
|
+
for (int n0 = ctx->row_start; n0 < ctx->row_end; n0 += ctx->ntile) {
|
|
4294
|
+
const int nt = std::min(ctx->ntile, ctx->row_end - n0);
|
|
4295
|
+
for (int t = 0; t < nt; ++t) {
|
|
4296
|
+
const unsigned char* prow = ctx->packed +
|
|
4297
|
+
(size_t)(n0 + t) * nblocks * ctx->row_stride;
|
|
4298
|
+
const float* amax_row = ctx->absmax + (size_t)(n0 + t) * nblocks;
|
|
4299
|
+
float* dst = buf.data() + (size_t)t * ctx->K;
|
|
4300
|
+
for (int b = 0; b < nblocks; ++b) {
|
|
4301
|
+
dequantize_3bit_cpu(prow + (size_t)b * ctx->row_stride, amax_row + b,
|
|
4302
|
+
dst + (size_t)b * ctx->block, 1, ctx->block);
|
|
4303
|
+
}
|
|
4304
|
+
}
|
|
4305
|
+
for (int m = 0; m < ctx->M; ++m) {
|
|
4306
|
+
float* dxrow = part + (size_t)m * ctx->K;
|
|
4307
|
+
const float* dyrow = ctx->dY + (size_t)m * ctx->N + n0;
|
|
4308
|
+
for (int t = 0; t < nt; ++t) {
|
|
4309
|
+
const float s = dyrow[t];
|
|
4310
|
+
const float* brow = buf.data() + (size_t)t * ctx->K;
|
|
4311
|
+
#if BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
|
|
4312
|
+
int k = 0;
|
|
4313
|
+
const __m256 vs = _mm256_set1_ps(s);
|
|
4314
|
+
for (; k + 8 <= ctx->K; k += 8) {
|
|
4315
|
+
__m256 acc = _mm256_loadu_ps(dxrow + k);
|
|
4316
|
+
acc = _mm256_fmadd_ps(vs, _mm256_loadu_ps(brow + k), acc);
|
|
4317
|
+
_mm256_storeu_ps(dxrow + k, acc);
|
|
4318
|
+
}
|
|
4319
|
+
for (; k < ctx->K; ++k) dxrow[k] += s * brow[k];
|
|
4320
|
+
#else
|
|
4321
|
+
for (int k = 0; k < ctx->K; ++k) dxrow[k] += s * brow[k];
|
|
4322
|
+
#endif
|
|
4323
|
+
}
|
|
4324
|
+
}
|
|
4325
|
+
}
|
|
4326
|
+
}
|
|
4327
|
+
|
|
4328
|
+
void q3bit_gemm_t_cpu(const float* dY, int M, const unsigned char* packed,
|
|
4329
|
+
const float* absmax, float* dX, int N, int K, int block,
|
|
4330
|
+
int nthreads, int ntile) {
|
|
4331
|
+
if (!dY || !packed || !absmax || !dX || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
|
|
4332
|
+
(block % 8) != 0 || (K % block) != 0) {
|
|
4333
|
+
return;
|
|
4334
|
+
}
|
|
4335
|
+
if (ntile <= 0) ntile = 64;
|
|
4336
|
+
if (nthreads <= 0) {
|
|
4337
|
+
nthreads = (int)std::thread::hardware_concurrency();
|
|
4338
|
+
if (nthreads <= 0) nthreads = 1;
|
|
4339
|
+
}
|
|
4340
|
+
const int row_stride = (block / 8) * 3;
|
|
4341
|
+
std::vector<std::vector<float>> partials((size_t)nthreads);
|
|
4342
|
+
std::vector<BnbQ3GemmTCtx> ctxs((size_t)nthreads);
|
|
4343
|
+
std::vector<std::thread> threads;
|
|
4344
|
+
threads.reserve((size_t)nthreads);
|
|
4345
|
+
for (int t = 0; t < nthreads; ++t) {
|
|
4346
|
+
const int row_start = (int)((long long)N * t / nthreads);
|
|
4347
|
+
const int row_end = (int)((long long)N * (t + 1) / nthreads);
|
|
4348
|
+
if (row_start >= row_end) continue;
|
|
4349
|
+
partials[t].assign((size_t)M * K, 0.0f);
|
|
4350
|
+
ctxs[t] = BnbQ3GemmTCtx{dY, packed, absmax, M, N, K, block, ntile, row_start,
|
|
4351
|
+
row_end, row_stride};
|
|
4352
|
+
threads.emplace_back(bnb_q3bit_gemm_t_worker, &ctxs[t], partials[t].data());
|
|
4353
|
+
}
|
|
4354
|
+
for (auto& th : threads) th.join();
|
|
4355
|
+
for (size_t i = 0; i < (size_t)M * K; ++i) dX[i] = 0.0f;
|
|
4356
|
+
for (auto& part : partials) {
|
|
4357
|
+
for (size_t i = 0; i < (size_t)M * K; ++i) dX[i] += part[i];
|
|
4358
|
+
}
|
|
4359
|
+
}
|
|
4360
|
+
|
|
4361
|
+
void q3bit_gemm_t_ref_cpu(const float* dY, int M, const unsigned char* packed,
|
|
4362
|
+
const float* absmax, float* dX, int N, int K, int block) {
|
|
4363
|
+
const int nblocks = K / block;
|
|
4364
|
+
const int row_stride = (block / 8) * 3;
|
|
4365
|
+
for (int i = 0; i < M * K; ++i) dX[i] = 0.0f;
|
|
4366
|
+
for (int n = 0; n < N; ++n) {
|
|
4367
|
+
const unsigned char* prow = packed + (size_t)n * nblocks * row_stride;
|
|
4368
|
+
const float* amax_row = absmax + (size_t)n * nblocks;
|
|
4369
|
+
for (int m = 0; m < M; ++m) {
|
|
4370
|
+
const float s = dY[(size_t)m * N + n];
|
|
4371
|
+
float* dxrow = dX + (size_t)m * K;
|
|
4372
|
+
for (int b = 0; b < nblocks; ++b) {
|
|
4373
|
+
const float amax = amax_row[b];
|
|
4374
|
+
const float scale = amax / 3.5f;
|
|
4375
|
+
const unsigned char* p = prow + (size_t)b * row_stride;
|
|
4376
|
+
for (int i = 0; i < block; i += 8) {
|
|
4377
|
+
const unsigned char* q = p + (i / 8) * 3;
|
|
4378
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
|
|
4379
|
+
for (int j = 0; j < 8; ++j) {
|
|
4380
|
+
int code = (int)((v >> (3 * j)) & 7);
|
|
4381
|
+
dxrow[b * block + i + j] += s * ((float)code * scale - amax);
|
|
4382
|
+
}
|
|
4383
|
+
}
|
|
4384
|
+
}
|
|
4385
|
+
}
|
|
4386
|
+
}
|
|
4387
|
+
}
|
|
4388
|
+
|
|
4389
|
+
void q3bit_gemm_ref_cpu(const float* A, int M, const unsigned char* packed,
|
|
4390
|
+
const float* absmax, float* C, int N, int K, int block) {
|
|
4391
|
+
const int nblocks = K / block;
|
|
4392
|
+
const int row_stride = (block / 8) * 3;
|
|
4393
|
+
for (int n = 0; n < N; ++n) {
|
|
4394
|
+
for (int m = 0; m < M; ++m) {
|
|
4395
|
+
float sum = 0.0f;
|
|
4396
|
+
for (int b = 0; b < nblocks; ++b) {
|
|
4397
|
+
const unsigned char* prow = packed + (size_t)n * nblocks * row_stride +
|
|
4398
|
+
(size_t)b * row_stride;
|
|
4399
|
+
const float amax = absmax[(size_t)n * nblocks + b];
|
|
4400
|
+
const float scale = amax / 3.5f;
|
|
4401
|
+
for (int i = 0; i < block; i += 8) {
|
|
4402
|
+
const unsigned char* q = prow + (i / 8) * 3;
|
|
4403
|
+
unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
|
|
4404
|
+
for (int j = 0; j < 8; ++j) {
|
|
4405
|
+
int code = (int)((v >> (3 * j)) & 7);
|
|
4406
|
+
float bv = (float)code * scale - amax;
|
|
4407
|
+
sum += A[(size_t)m * K + b * block + i + j] * bv;
|
|
4408
|
+
}
|
|
4409
|
+
}
|
|
4410
|
+
}
|
|
4411
|
+
C[(size_t)m * N + n] = sum;
|
|
4412
|
+
}
|
|
4413
|
+
}
|
|
4414
|
+
}
|
|
@@ -595,6 +595,40 @@ long long fused_add_scale_rmsnorm_cpu(
|
|
|
595
595
|
float* x, const float* y, const float* scale,
|
|
596
596
|
const float* weight, float* out, float* xs, long long M, long long D, float eps
|
|
597
597
|
);
|
|
598
|
+
|
|
599
|
+
// ---------------------------------------------------------------------------
|
|
600
|
+
// 2-/3-bit blockwise dequantization (from the 2bit/3bit CPU training research,
|
|
601
|
+
// see q3bit_proto.py / report 7.139-7.141). Layout matches the prototype:
|
|
602
|
+
// blocks of `block` elements; 3-bit block = block/8*3 bytes, 2-bit = block/4;
|
|
603
|
+
// one float absmax per block; value(code) = code*(absmax/center) - absmax,
|
|
604
|
+
// center = (2^bits - 1) / 2.
|
|
605
|
+
// `dequantize_*_cpu` dispatch to the best available path (NEON on ARM, AVX2 on
|
|
606
|
+
// x86, scalar otherwise); the `*_scalar_cpu` variants are the reference used by
|
|
607
|
+
// the C selftest to cross-check the vector paths.
|
|
608
|
+
// ---------------------------------------------------------------------------
|
|
609
|
+
void dequantize_3bit_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
610
|
+
long long n_blocks, long long block);
|
|
611
|
+
void dequantize_2bit_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
612
|
+
long long n_blocks, long long block);
|
|
613
|
+
void dequantize_3bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
614
|
+
long long n_blocks, long long block);
|
|
615
|
+
void dequantize_2bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
|
|
616
|
+
long long n_blocks, long long block);
|
|
617
|
+
|
|
618
|
+
// Fused 3-bit dequant + GEMM: C[M,N] = A[M,K] @ dequant_3bit(B)[N,K]^T, computed directly
|
|
619
|
+
// from the packed table without materializing the fp32 weight matrix (lm_head / large-vocab
|
|
620
|
+
// use; benchmarked faster than fp32 matmul on this CPU, see report 7.143).
|
|
621
|
+
void q3bit_gemm_cpu(const float* A, int M, const unsigned char* packed, const float* absmax,
|
|
622
|
+
float* C, int N, int K, int block, int nthreads, int ntile);
|
|
623
|
+
// Scalar reference for tests.
|
|
624
|
+
void q3bit_gemm_ref_cpu(const float* A, int M, const unsigned char* packed,
|
|
625
|
+
const float* absmax, float* C, int N, int K, int block);
|
|
626
|
+
// Transposed variant for training backward: dX[M,K] = dY[M,N] @ dequant(B)[N,K].
|
|
627
|
+
void q3bit_gemm_t_cpu(const float* dY, int M, const unsigned char* packed,
|
|
628
|
+
const float* absmax, float* dX, int N, int K, int block,
|
|
629
|
+
int nthreads, int ntile);
|
|
630
|
+
void q3bit_gemm_t_ref_cpu(const float* dY, int M, const unsigned char* packed,
|
|
631
|
+
const float* absmax, float* dX, int N, int K, int block);
|
|
598
632
|
#if defined(__cplusplus)
|
|
599
633
|
}
|
|
600
634
|
#endif
|
{bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/pythonInterface.cpp
RENAMED
|
@@ -1013,4 +1013,53 @@ long long cfused_add_scale_rmsnorm_cpu(
|
|
|
1013
1013
|
return fused_add_scale_rmsnorm_cpu(x, y, scale, weight, out, xs, M, D, eps);
|
|
1014
1014
|
}
|
|
1015
1015
|
|
|
1016
|
+
// 2-/3-bit blockwise dequantization (research prototype; see cpu_ops.h).
|
|
1017
|
+
long long cbdequantize_3bit_cpu(
|
|
1018
|
+
const unsigned char* packed, const float* absmax, float* out,
|
|
1019
|
+
long long n_blocks, long long block
|
|
1020
|
+
) {
|
|
1021
|
+
if (!packed || !absmax || !out || n_blocks <= 0 || block <= 0 || (block % 8) != 0) return -1;
|
|
1022
|
+
dequantize_3bit_cpu(packed, absmax, out, n_blocks, block);
|
|
1023
|
+
return 0;
|
|
1024
|
+
}
|
|
1025
|
+
|
|
1026
|
+
long long cbdequantize_2bit_cpu(
|
|
1027
|
+
const unsigned char* packed, const float* absmax, float* out,
|
|
1028
|
+
long long n_blocks, long long block
|
|
1029
|
+
) {
|
|
1030
|
+
if (!packed || !absmax || !out || n_blocks <= 0 || block <= 0 || (block % 4) != 0) return -1;
|
|
1031
|
+
dequantize_2bit_cpu(packed, absmax, out, n_blocks, block);
|
|
1032
|
+
return 0;
|
|
1033
|
+
}
|
|
1034
|
+
|
|
1035
|
+
// Transposed fused 3-bit dequant+GEMM: dX[M,K] = dY[M,N] @ dequant(B)[N,K].
|
|
1036
|
+
long long cbq3bit_gemm_t_cpu(
|
|
1037
|
+
const float* dY, long long M, const unsigned char* packed, const float* absmax,
|
|
1038
|
+
float* dX, long long N, long long K, long long block, long long nthreads, long long ntile
|
|
1039
|
+
) {
|
|
1040
|
+
if (!dY || !packed || !absmax || !dX || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
|
|
1041
|
+
(block % 8) != 0 || (K % block) != 0 || M > 2147483647LL || N > 2147483647LL ||
|
|
1042
|
+
K > 2147483647LL) {
|
|
1043
|
+
return -1;
|
|
1044
|
+
}
|
|
1045
|
+
q3bit_gemm_t_cpu(dY, (int)M, packed, absmax, dX, (int)N, (int)K, (int)block,
|
|
1046
|
+
(int)nthreads, (int)ntile);
|
|
1047
|
+
return 0;
|
|
1048
|
+
}
|
|
1049
|
+
|
|
1050
|
+
// Fused 3-bit dequant + GEMM (C[M,N] = A[M,K] @ dequant(B)[N,K]^T).
|
|
1051
|
+
long long cbq3bit_gemm_cpu(
|
|
1052
|
+
const float* A, long long M, const unsigned char* packed, const float* absmax,
|
|
1053
|
+
float* C, long long N, long long K, long long block, long long nthreads, long long ntile
|
|
1054
|
+
) {
|
|
1055
|
+
if (!A || !packed || !absmax || !C || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
|
|
1056
|
+
(block % 8) != 0 || (K % block) != 0 || M > 2147483647LL || N > 2147483647LL ||
|
|
1057
|
+
K > 2147483647LL) {
|
|
1058
|
+
return -1;
|
|
1059
|
+
}
|
|
1060
|
+
q3bit_gemm_cpu(A, (int)M, packed, absmax, C, (int)N, (int)K, (int)block,
|
|
1061
|
+
(int)nthreads, (int)ntile);
|
|
1062
|
+
return 0;
|
|
1063
|
+
}
|
|
1064
|
+
|
|
1016
1065
|
}
|
{bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/EPYC_VALIDATION.md
RENAMED
|
@@ -59,3 +59,11 @@ cd /tmp
|
|
|
59
59
|
bash container_isa_report.sh 2>&1 | tee /tmp/cpu_forge_isa_report.txt
|
|
60
60
|
bash container_isa_runtime_check.sh 2>&1 | tee /tmp/cpu_forge_runtime_report.txt
|
|
61
61
|
```
|
|
62
|
+
|
|
63
|
+
## 2026-10-19 补充:manylinux_2_17 并行 wheel 已可构建
|
|
64
|
+
|
|
65
|
+
之前记录"EPYC 上 pip 选到 manylinux_2_17 serial wheel"的缺口已解决:`build_oldgomp_linux.sh`
|
|
66
|
+
用 CentOS7 devtoolset-8 产出带 OpenMP 的 manylinux_2_17 wheel(C 自检 9/9,优化器内核比 serial
|
|
67
|
+
快 4×)。注意该 wheel 的 avx512 .so 仍是 zig serial;在 AVX-512 机器上比较 AVX2 OpenMP 与
|
|
68
|
+
AVX-512 serial 时可用 `BNB_CPU_DISABLE_AVX512=1` 切换。详见 `bench_results/oldglibc_openmp_wheel.md`。
|
|
69
|
+
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# ISA 兼容矩阵(2026-10-19 回归)
|
|
2
|
+
|
|
3
|
+
本文件由 `isa_matrix.sh` 的实测输出整理;原始日志见 `bench_results/isa_matrix_raw.log`,
|
|
4
|
+
构建脚本 `research/isa_build.py`(zig 交叉编译)。复现(x86 开发机 + WSL):
|
|
5
|
+
|
|
6
|
+
```bash
|
|
7
|
+
python D:\work\research\isa_build.py # 编译全部变体(Windows,zig)
|
|
8
|
+
wsl -d Ubuntu -- bash /mnt/d/work/bitsandbytes-CPU/isa_matrix.sh # 审计 + 运行时自检
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
## 构建矩阵
|
|
12
|
+
|
|
13
|
+
| 目标 | 工具链 | 产物 | glibc 需求 | 用途 |
|
|
14
|
+
|---|---|---|---|---|
|
|
15
|
+
| x86_64 AVX2 | zig 0.14 clang | `libbitsandbytes_cpu.so` | ≤2.16 | manylinux_2_17 wheel(serial,无 OpenMP) |
|
|
16
|
+
| x86_64 generic | zig 0.14 clang | `libbitsandbytes_cpu_generic.so` | ≤2.16 | 无 AVX2 x86 的 portable 回退 |
|
|
17
|
+
| x86_64 AVX-512 | zig 0.14 clang | `libbitsandbytes_cpu_avx512.so` | ≤2.16 | AVX-512/BF16 机器加速(EPYC 实测) |
|
|
18
|
+
| x86_64 原生 OpenMP | 本机 g++ 15 | manylinux_2_39 wheel | ≤2.39 | 新 glibc 的并行版(auditwheel 打包 libgomp) |
|
|
19
|
+
| aarch64 gnu | zig 0.14 clang | `build_arm64/libbitsandbytes_cpu.so` | ≤2.27 | 源码/自编 Linux ARM |
|
|
20
|
+
| aarch64 musl | zig 0.14 clang | `build_arm64/libbitsandbytes_cpu_musl.so` | 无 | Termux/静态 musl |
|
|
21
|
+
| aarch64 静态自检 | zig 0.14 clang | `build_arm64/selftest_arm64_musl` | 无 | qemu-aarch64 运行时验证 |
|
|
22
|
+
|
|
23
|
+
## 静态指令审计
|
|
24
|
+
|
|
25
|
+
| 变体 | AVX 类指令 | zmm(AVX-512) | BF16 指令 | 导出 `has_avx512bf16_cpu` |
|
|
26
|
+
|---|---|---|---|---|
|
|
27
|
+
| x86_64 AVX2 | 8,058 | 0 | 0 | 否 |
|
|
28
|
+
| x86_64 generic | **0** | 0 | 0 | 否 |
|
|
29
|
+
| x86_64 AVX-512 | 9,890 | 2,947 | **182** | 是 |
|
|
30
|
+
| aarch64 gnu/musl | —(ARM) | — | — | 否(NEON 类指令 674 条(含新增的 2/3-bit dequant NEON 路径),AArch64 机器码) |
|
|
31
|
+
|
|
32
|
+
结论:generic 库**零 AVX 指令**(可在任何 x86_64 上跑);AVX2 库无 AVX-512/zmm;
|
|
33
|
+
AVX-512 库的 182 条 BF16 指令与 `_cpu_caps` 的 `avx512_bf16` 门控一致(不会在无 BF16
|
|
34
|
+
的机器上误载入)。
|
|
35
|
+
|
|
36
|
+
## 运行时自检(C 内核独立自检,7 项:quantize 往返 / gemm_8bit / gemv_4bit /
|
|
37
|
+
AdamW8bit / 向量-标量叉验证 / 3-bit dequant / 2-bit dequant)
|
|
38
|
+
|
|
39
|
+
| 变体 | 运行方式 | 结果 |
|
|
40
|
+
|---|---|---|
|
|
41
|
+
| x86_64 AVX2 | 本机(R5-4500U,AVX2) | **7/7 PASS** |
|
|
42
|
+
| x86_64 generic | 本机 | **7/7 PASS** |
|
|
43
|
+
| x86_64 AVX-512 | 本机无 AVX-512 → 静态审计;EPYC 9Y24 容器真机 | 静态 182 BF16 指令;EPYC 上 **5/5 PASS**、GDN 5/5(见 `EPYC_VALIDATION.md`) |
|
|
44
|
+
| aarch64 musl | qemu-aarch64(x86 开发机) | **7/7 PASS** |
|
|
45
|
+
|
|
46
|
+
## 边界
|
|
47
|
+
|
|
48
|
+
- aarch64 是**架构级**验证(qemu + 静态 musl),不是 Android/bionic 真机;Termux 真机
|
|
49
|
+
历史结论仍有效。
|
|
50
|
+
- aarch64 gnu 库当前 zig 默认 glibc 目标为 2.27(若将来发 aarch64 manylinux wheel,
|
|
51
|
+
需显式 `-target aarch64-linux-gnu.2.17`)。
|
|
52
|
+
- AVX-512 运行时验证依赖真实硬件(EPYC 容器),本机只能做静态/编译检查。
|
|
53
|
+
|
|
54
|
+
## 2/3-bit dequant 内核(2026-10-19 加入 csrc)
|
|
55
|
+
|
|
56
|
+
从 2bit/3bit 训练原型移植进正式 `csrc`:
|
|
57
|
+
- `cpu_ops.h` / `cpu_ops.cpp`:`dequantize_3bit_cpu` / `dequantize_2bit_cpu`(按架构分派:
|
|
58
|
+
ARM NEON / x86 AVX2 / 标量回退),另有 `*_scalar_cpu` 参考实现;
|
|
59
|
+
- `pythonInterface.cpp`:导出 `cbdequantize_3bit_cpu` / `cbdequantize_2bit_cpu`(供 ctypes/未来
|
|
60
|
+
Python API 使用);
|
|
61
|
+
- `selftest_cpu.c`:新增第 6/7 项——C 里量化后分别用标量参考与 AVX2/NEON 路径反量化,
|
|
62
|
+
要求 maxdiff < 1e-6。x86 AVX2、generic、aarch64 qemu 全部 7/7 PASS。
|
|
63
|
+
|
|
64
|
+
aarch64 库的 NEON 类指令从 570 条增加到 **674 条**(新增 dequant NEON 路径的直接证据)。
|
|
65
|
+
|
|
66
|
+
## 8. csrc fused 3-bit GEMM(第 8 项自检)
|
|
67
|
+
|
|
68
|
+
`q3bit_gemm_cpu` / `cbq3bit_gemm_cpu`(`cpu_ops.cpp` + `pythonInterface.cpp`):
|
|
69
|
+
`C[M,N] = A[M,K] @ dequant_3bit(B)[N,K]^T`,按 B 行 tile 现场解包到 L2 缓冲,
|
|
70
|
+
不物化 fp32 权重矩阵。自检第 8 项要求它与标量参考的相对误差 < 1e-4;x86 AVX2/generic、
|
|
71
|
+
aarch64 qemu 全部 **8/8 PASS**。性能见 `bench_results/2bit3bit_proto.md` §7
|
|
72
|
+
(lm_head 形状:比 fp32 matmul 快 14–22%,内存 -89%)。
|
|
73
|
+
|
|
74
|
+
## 9. 训练反向的转置融合 GEMM(第 9 项自检)
|
|
75
|
+
|
|
76
|
+
`q3bit_gemm_t_cpu` / `cbq3bit_gemm_t_cpu`:`dX[M,K] = dY[M,N] @ dequant(B)[N,K]`,线程私有
|
|
77
|
+
dX 部分和再归约。自检第 9 项(transposed fused GEMM vs 标量参考,rel < 1e-4):x86 AVX2 原生、
|
|
78
|
+
x86 generic、aarch64 qemu 全部 **9/9 PASS**。性能见 `bench_results/2bit3bit_proto.md` §8
|
|
79
|
+
(反向比 oneDNN fp32 慢 2.4×,暂不替换;前向融合仍严格更优)。
|