bitsandbytes-cpu-fork 0.50.2.dev5__tar.gz → 0.50.2.dev6__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (152) hide show
  1. {bitsandbytes_cpu_fork-0.50.2.dev5/bitsandbytes_cpu_fork.egg-info → bitsandbytes_cpu_fork-0.50.2.dev6}/PKG-INFO +2 -2
  2. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/README.md +1 -1
  3. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/__init__.py +1 -1
  4. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6/bitsandbytes_cpu_fork.egg-info}/PKG-INFO +2 -2
  5. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/SOURCES.txt +1 -0
  6. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_cache.h +3 -0
  7. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_features.h +6 -0
  8. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_ops.cpp +376 -0
  9. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_ops.h +34 -0
  10. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/pythonInterface.cpp +49 -0
  11. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/EPYC_VALIDATION.md +8 -0
  12. bitsandbytes_cpu_fork-0.50.2.dev6/docs_cpu/ISA_MATRIX.md +79 -0
  13. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/WHEEL_PLATFORM_NOTES.md +26 -0
  14. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/selftest_cpu.c +448 -269
  15. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/LICENSE +0 -0
  16. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/MANIFEST.in +0 -0
  17. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/__main__.py +0 -0
  18. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/_cpu_caps.py +0 -0
  19. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/_ops.py +0 -0
  20. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/autograd/__init__.py +0 -0
  21. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/autograd/_functions.py +0 -0
  22. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/__init__.py +0 -0
  23. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cpu/__init__.py +0 -0
  24. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cpu/ops.py +0 -0
  25. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cuda/__init__.py +0 -0
  26. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/cuda/ops.py +0 -0
  27. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/default/__init__.py +0 -0
  28. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/default/ops.py +0 -0
  29. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/hpu/__init__.py +0 -0
  30. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/hpu/ops.py +0 -0
  31. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/mps/__init__.py +0 -0
  32. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/mps/ops.py +0 -0
  33. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/__init__.py +0 -0
  34. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/kernels_4bit.py +0 -0
  35. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/kernels_8bit_quant.py +0 -0
  36. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/kernels_optim.py +0 -0
  37. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/triton/ops.py +0 -0
  38. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/utils.py +0 -0
  39. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/xpu/__init__.py +0 -0
  40. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/backends/xpu/ops.py +0 -0
  41. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/cextension.py +0 -0
  42. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/cli.py +0 -0
  43. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/consts.py +0 -0
  44. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/cuda_specs.py +0 -0
  45. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/detect_cpu.py +0 -0
  46. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/__init__.py +0 -0
  47. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/cuda.py +0 -0
  48. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/main.py +0 -0
  49. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/diagnostics/utils.py +0 -0
  50. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/disk_balancer.py +0 -0
  51. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/efst.py +0 -0
  52. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/functional.py +0 -0
  53. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/gdn_cpu.py +0 -0
  54. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/libbitsandbytes_cpu.dll +0 -0
  55. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/nn/__init__.py +0 -0
  56. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/nn/modules.py +0 -0
  57. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/nn/parametrize.py +0 -0
  58. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/__init__.py +0 -0
  59. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/adagrad.py +0 -0
  60. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/adam.py +0 -0
  61. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/adamw.py +0 -0
  62. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/ademamix.py +0 -0
  63. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/lamb.py +0 -0
  64. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/lars.py +0 -0
  65. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/lion.py +0 -0
  66. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/optimizer.py +0 -0
  67. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/rmsprop.py +0 -0
  68. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/optim/sgd.py +0 -0
  69. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/py.typed +0 -0
  70. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/torch_cpu_kit.py +0 -0
  71. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/utils.py +0 -0
  72. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes/vcomp140.dll +0 -0
  73. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_cli.py +0 -0
  74. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/dependency_links.txt +0 -0
  75. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/entry_points.txt +0 -0
  76. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/requires.txt +0 -0
  77. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/bitsandbytes_cpu_fork.egg-info/top_level.txt +0 -0
  78. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_linux.sh +0 -0
  79. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/build_manual.bat +0 -0
  80. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/export.def +0 -0
  81. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/export_avx512.def +0 -0
  82. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/export_generic.def +0 -0
  83. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_manual/selftest_win.bat +0 -0
  84. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/build_termux.sh +0 -0
  85. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/avx2_gemv_4bit.h +0 -0
  86. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/common.cuh +0 -0
  87. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/common.h +0 -0
  88. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/compat.cuh +0 -0
  89. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/compat_device.cuh +0 -0
  90. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/cpu_gdn.cpp +0 -0
  91. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit.cu +0 -0
  92. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_common.cuh +0 -0
  93. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_mma.cuh +0 -0
  94. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_simt.cu +0 -0
  95. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_simt.cuh +0 -0
  96. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm75.cu +0 -0
  97. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm75.cuh +0 -0
  98. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm80.cu +0 -0
  99. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/gemm_4bit_sm80.cuh +0 -0
  100. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/kernels.cu +0 -0
  101. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/kernels.cuh +0 -0
  102. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/ops.cu +0 -0
  103. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/ops.cuh +0 -0
  104. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/test_avx2_gemv_4bit.cpp +0 -0
  105. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/test_cpu_cache.cpp +0 -0
  106. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_kernels.cpp +0 -0
  107. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_kernels.h +0 -0
  108. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_ops.cpp +0 -0
  109. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/csrc/xpu_ops.h +0 -0
  110. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/API_REFERENCE.md +0 -0
  111. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/API_REFERENCE_EN.md +0 -0
  112. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/MOTION_BOUNDARY.md +0 -0
  113. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/MOTION_BOUNDARY_EN.md +0 -0
  114. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/QUICKSTART.md +0 -0
  115. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/QUICKSTART_EN.md +0 -0
  116. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECHNICAL_GUIDE.md +0 -0
  117. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECHNICAL_GUIDE_EN.md +0 -0
  118. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECH_REPORT.md +0 -0
  119. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/TECH_REPORT_EN.md +0 -0
  120. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/docs_cpu/check_docs.py +0 -0
  121. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/compile_inference.py +0 -0
  122. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/cpu/cpu_training.py +0 -0
  123. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/int8_inference_huggingface.py +0 -0
  124. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/xpu/benchmark_paged_memory.py +0 -0
  125. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/examples/xpu/paged_xpu_training.py +0 -0
  126. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/pyproject.toml +0 -0
  127. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/setup.cfg +0 -0
  128. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/setup.py +0 -0
  129. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_autograd.py +0 -0
  130. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_cpu_e2e.py +0 -0
  131. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_cuda_setup_evaluator.py +0 -0
  132. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_functional.py +0 -0
  133. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_generation.py +0 -0
  134. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_linear4bit.py +0 -0
  135. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_linear8bitlt.py +0 -0
  136. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_modules.py +0 -0
  137. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_ops.py +0 -0
  138. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_optim.py +0 -0
  139. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_parametrize.py +0 -0
  140. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tests/test_rmsnorm_fwd_cpu.py +0 -0
  141. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/create_github_release.ps1 +0 -0
  142. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/dump_signatures.py +0 -0
  143. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/gen_api_reference.py +0 -0
  144. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/publish.ps1 +0 -0
  145. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_carve.c +0 -0
  146. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_carve_gui.c +0 -0
  147. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_mirror.c +0 -0
  148. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/sector_mirror_gui.c +0 -0
  149. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/train_llm_sft.py +0 -0
  150. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/verify_help_examples.py +0 -0
  151. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/verify_linux_cli.sh +0 -0
  152. {bitsandbytes_cpu_fork-0.50.2.dev5 → bitsandbytes_cpu_fork-0.50.2.dev6}/tools/verify_train_release.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: bitsandbytes-cpu-fork
3
- Version: 0.50.2.dev5
3
+ Version: 0.50.2.dev6
4
4
  Summary: bitsandbytes with a pure-CPU training backend (AVX2 / NEON): 8-bit optimizers, fused GDN, gemm_8bit, disk balancer. No GPU required.
5
5
  License-Expression: MIT
6
6
  Project-URL: repository, https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU
@@ -336,7 +336,7 @@ code and the Transformers / PEFT / Diffusers integrations keep working as-is:
336
336
 
337
337
  ```python
338
338
  import bitsandbytes as bnb # the fork, not upstream
339
- print(bnb.__version__) # 0.50.2.dev5
339
+ print(bnb.__version__) # 0.50.2.dev6
340
340
  ```
341
341
 
342
342
  Building from source instead (any platform) is described in
@@ -305,7 +305,7 @@ code and the Transformers / PEFT / Diffusers integrations keep working as-is:
305
305
 
306
306
  ```python
307
307
  import bitsandbytes as bnb # the fork, not upstream
308
- print(bnb.__version__) # 0.50.2.dev5
308
+ print(bnb.__version__) # 0.50.2.dev6
309
309
  ```
310
310
 
311
311
  Building from source instead (any platform) is described in
@@ -75,7 +75,7 @@ __pdoc__ = {
75
75
  "optim.optimizer.MockArgs": False,
76
76
  }
77
77
 
78
- __version__ = "0.50.2.dev5"
78
+ __version__ = "0.50.2.dev6"
79
79
 
80
80
  # 挂载 gdn_cpu(GDN 融合内核 / patch_transformers 入口),让 bnb.gdn_cpu 可直接访问
81
81
  from . import gdn_cpu # noqa: E402,F401
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: bitsandbytes-cpu-fork
3
- Version: 0.50.2.dev5
3
+ Version: 0.50.2.dev6
4
4
  Summary: bitsandbytes with a pure-CPU training backend (AVX2 / NEON): 8-bit optimizers, fused GDN, gemm_8bit, disk balancer. No GPU required.
5
5
  License-Expression: MIT
6
6
  Project-URL: repository, https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU
@@ -336,7 +336,7 @@ code and the Transformers / PEFT / Diffusers integrations keep working as-is:
336
336
 
337
337
  ```python
338
338
  import bitsandbytes as bnb # the fork, not upstream
339
- print(bnb.__version__) # 0.50.2.dev5
339
+ print(bnb.__version__) # 0.50.2.dev6
340
340
  ```
341
341
 
342
342
  Building from source instead (any platform) is described in
@@ -108,6 +108,7 @@ csrc/xpu_ops.h
108
108
  docs_cpu/API_REFERENCE.md
109
109
  docs_cpu/API_REFERENCE_EN.md
110
110
  docs_cpu/EPYC_VALIDATION.md
111
+ docs_cpu/ISA_MATRIX.md
111
112
  docs_cpu/MOTION_BOUNDARY.md
112
113
  docs_cpu/MOTION_BOUNDARY_EN.md
113
114
  docs_cpu/QUICKSTART.md
@@ -59,8 +59,11 @@
59
59
  #include <intrin.h>
60
60
  #include <windows.h>
61
61
  #elif defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))
62
+ #ifndef BNB_CPUID_H_INCLUDED
63
+ #define BNB_CPUID_H_INCLUDED
62
64
  #include <cpuid.h>
63
65
  #endif
66
+ #endif
64
67
 
65
68
  namespace bnb_cache_detail {
66
69
 
@@ -13,7 +13,13 @@
13
13
 
14
14
  #if (defined(__x86_64__) || defined(__i386__)) && !defined(_MSC_VER)
15
15
 
16
+ #ifndef BNB_CPUID_H_INCLUDED
17
+ #define BNB_CPUID_H_INCLUDED
18
+ // GCC 8's <cpuid.h> has no include guard, so include it exactly once across the build
19
+ // (cpu_features.h and cpu_cache.h both need it; without this sentinel old GCC errors
20
+ // with "redefinition of __get_cpuid_count").
16
21
  #include <cpuid.h>
22
+ #endif
17
23
  // clang's <cpuid.h> defines __cpuid/__cpuid_count as function-like macros. Windows
18
24
  // headers (intrin.h -> psdk_inc/intrin-impl.h, pulled in later via cpu_cache.h) declare
19
25
  // void __cpuid(int[4], int), which those macros then break ("too few arguments provided
@@ -4036,3 +4036,379 @@ long long fused_add_scale_rmsnorm_cpu(
4036
4036
  // 第二步:对 x 做 RMSNorm(x 现在就是 h),res 传 NULL
4037
4037
  return fused_rmsnorm_fwd_cpu(x, nullptr, weight, out, xs, M, D, eps);
4038
4038
  }
4039
+
4040
+ // ---------------------------------------------------------------------------
4041
+ // 2-/3-bit blockwise dequantization (research prototype, see q3bit_proto.py).
4042
+ // The scalar reference is cross-checked against the vector paths by selftest_cpu.
4043
+ // ---------------------------------------------------------------------------
4044
+ static inline float bnb_q3bit_value(int code, float scale, float amax) {
4045
+ return (float)code * scale - amax;
4046
+ }
4047
+
4048
+ void dequantize_3bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
4049
+ long long n_blocks, long long block) {
4050
+ for (long long b = 0; b < n_blocks; ++b) {
4051
+ const unsigned char* p = packed + b * (block / 8) * 3;
4052
+ const float amax = absmax[b];
4053
+ const float scale = amax / 3.5f;
4054
+ float* dst = out + b * block;
4055
+ for (long long g = 0; g < block; g += 8) {
4056
+ const unsigned char* q = p + (g / 8) * 3;
4057
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
4058
+ for (int j = 0; j < 8; ++j) {
4059
+ dst[g + j] = bnb_q3bit_value((int)((v >> (3 * j)) & 7), scale, amax);
4060
+ }
4061
+ }
4062
+ }
4063
+ }
4064
+
4065
+ void dequantize_2bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
4066
+ long long n_blocks, long long block) {
4067
+ for (long long b = 0; b < n_blocks; ++b) {
4068
+ const unsigned char* p = packed + b * (block / 4);
4069
+ const float amax = absmax[b];
4070
+ const float scale = amax / 1.5f;
4071
+ float* dst = out + b * block;
4072
+ for (long long g = 0; g < block; g += 8) {
4073
+ const unsigned char* q = p + (g / 4);
4074
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8);
4075
+ for (int j = 0; j < 8; ++j) {
4076
+ dst[g + j] = bnb_q3bit_value((int)((v >> (2 * j)) & 3), scale, amax);
4077
+ }
4078
+ }
4079
+ }
4080
+ }
4081
+
4082
+ #if BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
4083
+ static inline void bnb_dequant_3bit_avx2(const unsigned char* packed, const float* absmax,
4084
+ float* out, long long n_blocks, long long block) {
4085
+ for (long long b = 0; b < n_blocks; ++b) {
4086
+ const unsigned char* p = packed + b * (block / 8) * 3;
4087
+ const float amax = absmax[b];
4088
+ const float scale = amax / 3.5f;
4089
+ const __m256 vs = _mm256_set1_ps(scale);
4090
+ const __m256 va = _mm256_set1_ps(amax);
4091
+ float* dst = out + b * block;
4092
+ for (long long g = 0; g < block; g += 8) {
4093
+ const unsigned char* q = p + (g / 8) * 3;
4094
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
4095
+ __m256i codes = _mm256_setr_epi32(
4096
+ (int)(v & 7), (int)((v >> 3) & 7), (int)((v >> 6) & 7), (int)((v >> 9) & 7),
4097
+ (int)((v >> 12) & 7), (int)((v >> 15) & 7), (int)((v >> 18) & 7),
4098
+ (int)((v >> 21) & 7));
4099
+ __m256 f = _mm256_sub_ps(_mm256_mul_ps(_mm256_cvtepi32_ps(codes), vs), va);
4100
+ _mm256_storeu_ps(dst + g, f);
4101
+ }
4102
+ }
4103
+ }
4104
+
4105
+ static inline void bnb_dequant_2bit_avx2(const unsigned char* packed, const float* absmax,
4106
+ float* out, long long n_blocks, long long block) {
4107
+ for (long long b = 0; b < n_blocks; ++b) {
4108
+ const unsigned char* p = packed + b * (block / 4);
4109
+ const float amax = absmax[b];
4110
+ const float scale = amax / 1.5f;
4111
+ const __m256 vs = _mm256_set1_ps(scale);
4112
+ const __m256 va = _mm256_set1_ps(amax);
4113
+ float* dst = out + b * block;
4114
+ for (long long g = 0; g < block; g += 8) {
4115
+ const unsigned char* q = p + (g / 4);
4116
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8);
4117
+ __m256i codes = _mm256_setr_epi32(
4118
+ (int)(v & 3), (int)((v >> 2) & 3), (int)((v >> 4) & 3), (int)((v >> 6) & 3),
4119
+ (int)((v >> 8) & 3), (int)((v >> 10) & 3), (int)((v >> 12) & 3),
4120
+ (int)((v >> 14) & 3));
4121
+ __m256 f = _mm256_sub_ps(_mm256_mul_ps(_mm256_cvtepi32_ps(codes), vs), va);
4122
+ _mm256_storeu_ps(dst + g, f);
4123
+ }
4124
+ }
4125
+ }
4126
+ #endif
4127
+
4128
+ #if defined(_M_ARM64) || defined(__aarch64__)
4129
+ static inline void bnb_dequant_3bit_neon(const unsigned char* packed, const float* absmax,
4130
+ float* out, long long n_blocks, long long block) {
4131
+ for (long long b = 0; b < n_blocks; ++b) {
4132
+ const unsigned char* p = packed + b * (block / 8) * 3;
4133
+ const float amax = absmax[b];
4134
+ const float scale = amax / 3.5f;
4135
+ const float32x4_t vs = vdupq_n_f32(scale);
4136
+ const float32x4_t va = vdupq_n_f32(amax);
4137
+ float* dst = out + b * block;
4138
+ for (long long g = 0; g < block; g += 8) {
4139
+ const unsigned char* q = p + (g / 8) * 3;
4140
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
4141
+ uint32_t codes[8];
4142
+ for (int j = 0; j < 8; ++j) codes[j] = (v >> (3 * j)) & 7;
4143
+ float32x4_t c0 = vcvtq_f32_u32(vld1q_u32(codes));
4144
+ float32x4_t c1 = vcvtq_f32_u32(vld1q_u32(codes + 4));
4145
+ vst1q_f32(dst + g, vsubq_f32(vmulq_f32(c0, vs), va));
4146
+ vst1q_f32(dst + g + 4, vsubq_f32(vmulq_f32(c1, vs), va));
4147
+ }
4148
+ }
4149
+ }
4150
+
4151
+ static inline void bnb_dequant_2bit_neon(const unsigned char* packed, const float* absmax,
4152
+ float* out, long long n_blocks, long long block) {
4153
+ for (long long b = 0; b < n_blocks; ++b) {
4154
+ const unsigned char* p = packed + b * (block / 4);
4155
+ const float amax = absmax[b];
4156
+ const float scale = amax / 1.5f;
4157
+ const float32x4_t vs = vdupq_n_f32(scale);
4158
+ const float32x4_t va = vdupq_n_f32(amax);
4159
+ float* dst = out + b * block;
4160
+ for (long long g = 0; g < block; g += 8) {
4161
+ const unsigned char* q = p + (g / 4);
4162
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8);
4163
+ uint32_t codes[8];
4164
+ for (int j = 0; j < 8; ++j) codes[j] = (v >> (2 * j)) & 3;
4165
+ float32x4_t c0 = vcvtq_f32_u32(vld1q_u32(codes));
4166
+ float32x4_t c1 = vcvtq_f32_u32(vld1q_u32(codes + 4));
4167
+ vst1q_f32(dst + g, vsubq_f32(vmulq_f32(c0, vs), va));
4168
+ vst1q_f32(dst + g + 4, vsubq_f32(vmulq_f32(c1, vs), va));
4169
+ }
4170
+ }
4171
+ }
4172
+ #endif
4173
+
4174
+ void dequantize_3bit_cpu(const unsigned char* packed, const float* absmax, float* out,
4175
+ long long n_blocks, long long block) {
4176
+ #if defined(_M_ARM64) || defined(__aarch64__)
4177
+ bnb_dequant_3bit_neon(packed, absmax, out, n_blocks, block);
4178
+ #elif BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
4179
+ bnb_dequant_3bit_avx2(packed, absmax, out, n_blocks, block);
4180
+ #else
4181
+ dequantize_3bit_scalar_cpu(packed, absmax, out, n_blocks, block);
4182
+ #endif
4183
+ }
4184
+
4185
+ void dequantize_2bit_cpu(const unsigned char* packed, const float* absmax, float* out,
4186
+ long long n_blocks, long long block) {
4187
+ #if defined(_M_ARM64) || defined(__aarch64__)
4188
+ bnb_dequant_2bit_neon(packed, absmax, out, n_blocks, block);
4189
+ #elif BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
4190
+ bnb_dequant_2bit_avx2(packed, absmax, out, n_blocks, block);
4191
+ #else
4192
+ dequantize_2bit_scalar_cpu(packed, absmax, out, n_blocks, block);
4193
+ #endif
4194
+ }
4195
+
4196
+ // ---------------------------------------------------------------------------
4197
+ // Fused 3-bit dequant + GEMM (research prototype promoted to csrc).
4198
+ // C[M,N] = A[M,K] @ dequant(B)[N,K]^T. B is packed per row: nblocks blocks of
4199
+ // `block` elements (block % 8 == 0), each block block/8*3 bytes plus one fp32 absmax.
4200
+ // A B-row tile is dequantized into a small buffer (L2-resident) and the dot products run
4201
+ // from there, so the full fp32 table (e.g. 0.95 GB lm_head) is never materialized.
4202
+ // ---------------------------------------------------------------------------
4203
+ static inline void bnb_q3bit_dot(const float* a, const float* b, int K, float* out) {
4204
+ #if BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
4205
+ __m256 acc = _mm256_setzero_ps();
4206
+ int k = 0;
4207
+ for (; k + 8 <= K; k += 8) {
4208
+ acc = _mm256_fmadd_ps(_mm256_loadu_ps(a + k), _mm256_loadu_ps(b + k), acc);
4209
+ }
4210
+ __m128 lo = _mm256_castps256_ps128(acc);
4211
+ __m128 hi = _mm256_extractf128_ps(acc, 1);
4212
+ lo = _mm_add_ps(lo, hi);
4213
+ __m128 shuf = _mm_movehdup_ps(lo);
4214
+ __m128 sums = _mm_add_ps(lo, shuf);
4215
+ shuf = _mm_movehl_ps(shuf, sums);
4216
+ sums = _mm_add_ss(sums, shuf);
4217
+ float total = _mm_cvtss_f32(sums);
4218
+ for (; k < K; ++k) total += a[k] * b[k];
4219
+ *out = total;
4220
+ #else
4221
+ float total = 0.0f;
4222
+ for (int k = 0; k < K; ++k) total += a[k] * b[k];
4223
+ *out = total;
4224
+ #endif
4225
+ }
4226
+
4227
+ static void bnb_q3bit_gemm_worker(const float* A, int M, const unsigned char* packed,
4228
+ const float* absmax, float* C, int N, int K, int block,
4229
+ int ntile, int row_start, int row_end) {
4230
+ const int nblocks = K / block;
4231
+ const int row_stride = (block / 8) * 3;
4232
+ std::vector<float> buf((size_t)ntile * K);
4233
+ for (int n0 = row_start; n0 < row_end; n0 += ntile) {
4234
+ const int nt = std::min(ntile, row_end - n0);
4235
+ for (int t = 0; t < nt; ++t) {
4236
+ const unsigned char* prow = packed + (size_t)(n0 + t) * nblocks * row_stride;
4237
+ const float* amax_row = absmax + (size_t)(n0 + t) * nblocks;
4238
+ float* dst = buf.data() + (size_t)t * K;
4239
+ for (int b = 0; b < nblocks; ++b) {
4240
+ dequantize_3bit_cpu(prow + (size_t)b * row_stride, amax_row + b,
4241
+ dst + (size_t)b * block, 1, block);
4242
+ }
4243
+ }
4244
+ for (int m = 0; m < M; ++m) {
4245
+ const float* arow = A + (size_t)m * K;
4246
+ float* crow = C + (size_t)m * N + n0;
4247
+ for (int t = 0; t < nt; ++t) {
4248
+ bnb_q3bit_dot(arow, buf.data() + (size_t)t * K, K, crow + t);
4249
+ }
4250
+ }
4251
+ }
4252
+ }
4253
+
4254
+ void q3bit_gemm_cpu(const float* A, int M, const unsigned char* packed, const float* absmax,
4255
+ float* C, int N, int K, int block, int nthreads, int ntile) {
4256
+ if (!A || !packed || !absmax || !C || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
4257
+ (block % 8) != 0 || (K % block) != 0) {
4258
+ return;
4259
+ }
4260
+ if (ntile <= 0) ntile = 64;
4261
+ if (nthreads <= 0) {
4262
+ nthreads = (int)std::thread::hardware_concurrency();
4263
+ if (nthreads <= 0) nthreads = 1;
4264
+ }
4265
+ std::vector<std::thread> threads;
4266
+ threads.reserve((size_t)nthreads);
4267
+ for (int th = 0; th < nthreads; ++th) {
4268
+ const int row_start = (int)((long long)N * th / nthreads);
4269
+ const int row_end = (int)((long long)N * (th + 1) / nthreads);
4270
+ if (row_start >= row_end) continue;
4271
+ threads.emplace_back(bnb_q3bit_gemm_worker, A, M, packed, absmax, C, N, K, block,
4272
+ ntile, row_start, row_end);
4273
+ }
4274
+ for (auto& th : threads) th.join();
4275
+ }
4276
+
4277
+ // ---------------------------------------------------------------------------
4278
+ // Transposed variant for training backward: dX[M,K] = dY[M,N] @ dequant(B)[N,K].
4279
+ // Each thread accumulates a private dX (M*K floats, small) over its B-row range; the
4280
+ // partials are summed at the end, so there is no locking in the hot loop.
4281
+ // ---------------------------------------------------------------------------
4282
+ struct BnbQ3GemmTCtx {
4283
+ const float* dY;
4284
+ const unsigned char* packed;
4285
+ const float* absmax;
4286
+ int M, N, K, block, ntile, row_start, row_end, row_stride;
4287
+ };
4288
+
4289
+ static void bnb_q3bit_gemm_t_worker(const BnbQ3GemmTCtx* ctx, float* part) {
4290
+ const int nblocks = ctx->K / ctx->block;
4291
+ std::vector<float> buf((size_t)ctx->ntile * ctx->K);
4292
+ for (size_t i = 0; i < (size_t)ctx->M * ctx->K; ++i) part[i] = 0.0f;
4293
+ for (int n0 = ctx->row_start; n0 < ctx->row_end; n0 += ctx->ntile) {
4294
+ const int nt = std::min(ctx->ntile, ctx->row_end - n0);
4295
+ for (int t = 0; t < nt; ++t) {
4296
+ const unsigned char* prow = ctx->packed +
4297
+ (size_t)(n0 + t) * nblocks * ctx->row_stride;
4298
+ const float* amax_row = ctx->absmax + (size_t)(n0 + t) * nblocks;
4299
+ float* dst = buf.data() + (size_t)t * ctx->K;
4300
+ for (int b = 0; b < nblocks; ++b) {
4301
+ dequantize_3bit_cpu(prow + (size_t)b * ctx->row_stride, amax_row + b,
4302
+ dst + (size_t)b * ctx->block, 1, ctx->block);
4303
+ }
4304
+ }
4305
+ for (int m = 0; m < ctx->M; ++m) {
4306
+ float* dxrow = part + (size_t)m * ctx->K;
4307
+ const float* dyrow = ctx->dY + (size_t)m * ctx->N + n0;
4308
+ for (int t = 0; t < nt; ++t) {
4309
+ const float s = dyrow[t];
4310
+ const float* brow = buf.data() + (size_t)t * ctx->K;
4311
+ #if BNB_AVX2_CODE && (defined(__AVX2__) || (defined(__GNUC__) && (defined(__x86_64__) || defined(__i386__))))
4312
+ int k = 0;
4313
+ const __m256 vs = _mm256_set1_ps(s);
4314
+ for (; k + 8 <= ctx->K; k += 8) {
4315
+ __m256 acc = _mm256_loadu_ps(dxrow + k);
4316
+ acc = _mm256_fmadd_ps(vs, _mm256_loadu_ps(brow + k), acc);
4317
+ _mm256_storeu_ps(dxrow + k, acc);
4318
+ }
4319
+ for (; k < ctx->K; ++k) dxrow[k] += s * brow[k];
4320
+ #else
4321
+ for (int k = 0; k < ctx->K; ++k) dxrow[k] += s * brow[k];
4322
+ #endif
4323
+ }
4324
+ }
4325
+ }
4326
+ }
4327
+
4328
+ void q3bit_gemm_t_cpu(const float* dY, int M, const unsigned char* packed,
4329
+ const float* absmax, float* dX, int N, int K, int block,
4330
+ int nthreads, int ntile) {
4331
+ if (!dY || !packed || !absmax || !dX || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
4332
+ (block % 8) != 0 || (K % block) != 0) {
4333
+ return;
4334
+ }
4335
+ if (ntile <= 0) ntile = 64;
4336
+ if (nthreads <= 0) {
4337
+ nthreads = (int)std::thread::hardware_concurrency();
4338
+ if (nthreads <= 0) nthreads = 1;
4339
+ }
4340
+ const int row_stride = (block / 8) * 3;
4341
+ std::vector<std::vector<float>> partials((size_t)nthreads);
4342
+ std::vector<BnbQ3GemmTCtx> ctxs((size_t)nthreads);
4343
+ std::vector<std::thread> threads;
4344
+ threads.reserve((size_t)nthreads);
4345
+ for (int t = 0; t < nthreads; ++t) {
4346
+ const int row_start = (int)((long long)N * t / nthreads);
4347
+ const int row_end = (int)((long long)N * (t + 1) / nthreads);
4348
+ if (row_start >= row_end) continue;
4349
+ partials[t].assign((size_t)M * K, 0.0f);
4350
+ ctxs[t] = BnbQ3GemmTCtx{dY, packed, absmax, M, N, K, block, ntile, row_start,
4351
+ row_end, row_stride};
4352
+ threads.emplace_back(bnb_q3bit_gemm_t_worker, &ctxs[t], partials[t].data());
4353
+ }
4354
+ for (auto& th : threads) th.join();
4355
+ for (size_t i = 0; i < (size_t)M * K; ++i) dX[i] = 0.0f;
4356
+ for (auto& part : partials) {
4357
+ for (size_t i = 0; i < (size_t)M * K; ++i) dX[i] += part[i];
4358
+ }
4359
+ }
4360
+
4361
+ void q3bit_gemm_t_ref_cpu(const float* dY, int M, const unsigned char* packed,
4362
+ const float* absmax, float* dX, int N, int K, int block) {
4363
+ const int nblocks = K / block;
4364
+ const int row_stride = (block / 8) * 3;
4365
+ for (int i = 0; i < M * K; ++i) dX[i] = 0.0f;
4366
+ for (int n = 0; n < N; ++n) {
4367
+ const unsigned char* prow = packed + (size_t)n * nblocks * row_stride;
4368
+ const float* amax_row = absmax + (size_t)n * nblocks;
4369
+ for (int m = 0; m < M; ++m) {
4370
+ const float s = dY[(size_t)m * N + n];
4371
+ float* dxrow = dX + (size_t)m * K;
4372
+ for (int b = 0; b < nblocks; ++b) {
4373
+ const float amax = amax_row[b];
4374
+ const float scale = amax / 3.5f;
4375
+ const unsigned char* p = prow + (size_t)b * row_stride;
4376
+ for (int i = 0; i < block; i += 8) {
4377
+ const unsigned char* q = p + (i / 8) * 3;
4378
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
4379
+ for (int j = 0; j < 8; ++j) {
4380
+ int code = (int)((v >> (3 * j)) & 7);
4381
+ dxrow[b * block + i + j] += s * ((float)code * scale - amax);
4382
+ }
4383
+ }
4384
+ }
4385
+ }
4386
+ }
4387
+ }
4388
+
4389
+ void q3bit_gemm_ref_cpu(const float* A, int M, const unsigned char* packed,
4390
+ const float* absmax, float* C, int N, int K, int block) {
4391
+ const int nblocks = K / block;
4392
+ const int row_stride = (block / 8) * 3;
4393
+ for (int n = 0; n < N; ++n) {
4394
+ for (int m = 0; m < M; ++m) {
4395
+ float sum = 0.0f;
4396
+ for (int b = 0; b < nblocks; ++b) {
4397
+ const unsigned char* prow = packed + (size_t)n * nblocks * row_stride +
4398
+ (size_t)b * row_stride;
4399
+ const float amax = absmax[(size_t)n * nblocks + b];
4400
+ const float scale = amax / 3.5f;
4401
+ for (int i = 0; i < block; i += 8) {
4402
+ const unsigned char* q = prow + (i / 8) * 3;
4403
+ unsigned v = (unsigned)q[0] | ((unsigned)q[1] << 8) | ((unsigned)q[2] << 16);
4404
+ for (int j = 0; j < 8; ++j) {
4405
+ int code = (int)((v >> (3 * j)) & 7);
4406
+ float bv = (float)code * scale - amax;
4407
+ sum += A[(size_t)m * K + b * block + i + j] * bv;
4408
+ }
4409
+ }
4410
+ }
4411
+ C[(size_t)m * N + n] = sum;
4412
+ }
4413
+ }
4414
+ }
@@ -595,6 +595,40 @@ long long fused_add_scale_rmsnorm_cpu(
595
595
  float* x, const float* y, const float* scale,
596
596
  const float* weight, float* out, float* xs, long long M, long long D, float eps
597
597
  );
598
+
599
+ // ---------------------------------------------------------------------------
600
+ // 2-/3-bit blockwise dequantization (from the 2bit/3bit CPU training research,
601
+ // see q3bit_proto.py / report 7.139-7.141). Layout matches the prototype:
602
+ // blocks of `block` elements; 3-bit block = block/8*3 bytes, 2-bit = block/4;
603
+ // one float absmax per block; value(code) = code*(absmax/center) - absmax,
604
+ // center = (2^bits - 1) / 2.
605
+ // `dequantize_*_cpu` dispatch to the best available path (NEON on ARM, AVX2 on
606
+ // x86, scalar otherwise); the `*_scalar_cpu` variants are the reference used by
607
+ // the C selftest to cross-check the vector paths.
608
+ // ---------------------------------------------------------------------------
609
+ void dequantize_3bit_cpu(const unsigned char* packed, const float* absmax, float* out,
610
+ long long n_blocks, long long block);
611
+ void dequantize_2bit_cpu(const unsigned char* packed, const float* absmax, float* out,
612
+ long long n_blocks, long long block);
613
+ void dequantize_3bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
614
+ long long n_blocks, long long block);
615
+ void dequantize_2bit_scalar_cpu(const unsigned char* packed, const float* absmax, float* out,
616
+ long long n_blocks, long long block);
617
+
618
+ // Fused 3-bit dequant + GEMM: C[M,N] = A[M,K] @ dequant_3bit(B)[N,K]^T, computed directly
619
+ // from the packed table without materializing the fp32 weight matrix (lm_head / large-vocab
620
+ // use; benchmarked faster than fp32 matmul on this CPU, see report 7.143).
621
+ void q3bit_gemm_cpu(const float* A, int M, const unsigned char* packed, const float* absmax,
622
+ float* C, int N, int K, int block, int nthreads, int ntile);
623
+ // Scalar reference for tests.
624
+ void q3bit_gemm_ref_cpu(const float* A, int M, const unsigned char* packed,
625
+ const float* absmax, float* C, int N, int K, int block);
626
+ // Transposed variant for training backward: dX[M,K] = dY[M,N] @ dequant(B)[N,K].
627
+ void q3bit_gemm_t_cpu(const float* dY, int M, const unsigned char* packed,
628
+ const float* absmax, float* dX, int N, int K, int block,
629
+ int nthreads, int ntile);
630
+ void q3bit_gemm_t_ref_cpu(const float* dY, int M, const unsigned char* packed,
631
+ const float* absmax, float* dX, int N, int K, int block);
598
632
  #if defined(__cplusplus)
599
633
  }
600
634
  #endif
@@ -1013,4 +1013,53 @@ long long cfused_add_scale_rmsnorm_cpu(
1013
1013
  return fused_add_scale_rmsnorm_cpu(x, y, scale, weight, out, xs, M, D, eps);
1014
1014
  }
1015
1015
 
1016
+ // 2-/3-bit blockwise dequantization (research prototype; see cpu_ops.h).
1017
+ long long cbdequantize_3bit_cpu(
1018
+ const unsigned char* packed, const float* absmax, float* out,
1019
+ long long n_blocks, long long block
1020
+ ) {
1021
+ if (!packed || !absmax || !out || n_blocks <= 0 || block <= 0 || (block % 8) != 0) return -1;
1022
+ dequantize_3bit_cpu(packed, absmax, out, n_blocks, block);
1023
+ return 0;
1024
+ }
1025
+
1026
+ long long cbdequantize_2bit_cpu(
1027
+ const unsigned char* packed, const float* absmax, float* out,
1028
+ long long n_blocks, long long block
1029
+ ) {
1030
+ if (!packed || !absmax || !out || n_blocks <= 0 || block <= 0 || (block % 4) != 0) return -1;
1031
+ dequantize_2bit_cpu(packed, absmax, out, n_blocks, block);
1032
+ return 0;
1033
+ }
1034
+
1035
+ // Transposed fused 3-bit dequant+GEMM: dX[M,K] = dY[M,N] @ dequant(B)[N,K].
1036
+ long long cbq3bit_gemm_t_cpu(
1037
+ const float* dY, long long M, const unsigned char* packed, const float* absmax,
1038
+ float* dX, long long N, long long K, long long block, long long nthreads, long long ntile
1039
+ ) {
1040
+ if (!dY || !packed || !absmax || !dX || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
1041
+ (block % 8) != 0 || (K % block) != 0 || M > 2147483647LL || N > 2147483647LL ||
1042
+ K > 2147483647LL) {
1043
+ return -1;
1044
+ }
1045
+ q3bit_gemm_t_cpu(dY, (int)M, packed, absmax, dX, (int)N, (int)K, (int)block,
1046
+ (int)nthreads, (int)ntile);
1047
+ return 0;
1048
+ }
1049
+
1050
+ // Fused 3-bit dequant + GEMM (C[M,N] = A[M,K] @ dequant(B)[N,K]^T).
1051
+ long long cbq3bit_gemm_cpu(
1052
+ const float* A, long long M, const unsigned char* packed, const float* absmax,
1053
+ float* C, long long N, long long K, long long block, long long nthreads, long long ntile
1054
+ ) {
1055
+ if (!A || !packed || !absmax || !C || M <= 0 || N <= 0 || K <= 0 || block <= 0 ||
1056
+ (block % 8) != 0 || (K % block) != 0 || M > 2147483647LL || N > 2147483647LL ||
1057
+ K > 2147483647LL) {
1058
+ return -1;
1059
+ }
1060
+ q3bit_gemm_cpu(A, (int)M, packed, absmax, C, (int)N, (int)K, (int)block,
1061
+ (int)nthreads, (int)ntile);
1062
+ return 0;
1063
+ }
1064
+
1016
1065
  }
@@ -59,3 +59,11 @@ cd /tmp
59
59
  bash container_isa_report.sh 2>&1 | tee /tmp/cpu_forge_isa_report.txt
60
60
  bash container_isa_runtime_check.sh 2>&1 | tee /tmp/cpu_forge_runtime_report.txt
61
61
  ```
62
+
63
+ ## 2026-10-19 补充:manylinux_2_17 并行 wheel 已可构建
64
+
65
+ 之前记录"EPYC 上 pip 选到 manylinux_2_17 serial wheel"的缺口已解决:`build_oldgomp_linux.sh`
66
+ 用 CentOS7 devtoolset-8 产出带 OpenMP 的 manylinux_2_17 wheel(C 自检 9/9,优化器内核比 serial
67
+ 快 4×)。注意该 wheel 的 avx512 .so 仍是 zig serial;在 AVX-512 机器上比较 AVX2 OpenMP 与
68
+ AVX-512 serial 时可用 `BNB_CPU_DISABLE_AVX512=1` 切换。详见 `bench_results/oldglibc_openmp_wheel.md`。
69
+
@@ -0,0 +1,79 @@
1
+ # ISA 兼容矩阵(2026-10-19 回归)
2
+
3
+ 本文件由 `isa_matrix.sh` 的实测输出整理;原始日志见 `bench_results/isa_matrix_raw.log`,
4
+ 构建脚本 `research/isa_build.py`(zig 交叉编译)。复现(x86 开发机 + WSL):
5
+
6
+ ```bash
7
+ python D:\work\research\isa_build.py # 编译全部变体(Windows,zig)
8
+ wsl -d Ubuntu -- bash /mnt/d/work/bitsandbytes-CPU/isa_matrix.sh # 审计 + 运行时自检
9
+ ```
10
+
11
+ ## 构建矩阵
12
+
13
+ | 目标 | 工具链 | 产物 | glibc 需求 | 用途 |
14
+ |---|---|---|---|---|
15
+ | x86_64 AVX2 | zig 0.14 clang | `libbitsandbytes_cpu.so` | ≤2.16 | manylinux_2_17 wheel(serial,无 OpenMP) |
16
+ | x86_64 generic | zig 0.14 clang | `libbitsandbytes_cpu_generic.so` | ≤2.16 | 无 AVX2 x86 的 portable 回退 |
17
+ | x86_64 AVX-512 | zig 0.14 clang | `libbitsandbytes_cpu_avx512.so` | ≤2.16 | AVX-512/BF16 机器加速(EPYC 实测) |
18
+ | x86_64 原生 OpenMP | 本机 g++ 15 | manylinux_2_39 wheel | ≤2.39 | 新 glibc 的并行版(auditwheel 打包 libgomp) |
19
+ | aarch64 gnu | zig 0.14 clang | `build_arm64/libbitsandbytes_cpu.so` | ≤2.27 | 源码/自编 Linux ARM |
20
+ | aarch64 musl | zig 0.14 clang | `build_arm64/libbitsandbytes_cpu_musl.so` | 无 | Termux/静态 musl |
21
+ | aarch64 静态自检 | zig 0.14 clang | `build_arm64/selftest_arm64_musl` | 无 | qemu-aarch64 运行时验证 |
22
+
23
+ ## 静态指令审计
24
+
25
+ | 变体 | AVX 类指令 | zmm(AVX-512) | BF16 指令 | 导出 `has_avx512bf16_cpu` |
26
+ |---|---|---|---|---|
27
+ | x86_64 AVX2 | 8,058 | 0 | 0 | 否 |
28
+ | x86_64 generic | **0** | 0 | 0 | 否 |
29
+ | x86_64 AVX-512 | 9,890 | 2,947 | **182** | 是 |
30
+ | aarch64 gnu/musl | —(ARM) | — | — | 否(NEON 类指令 674 条(含新增的 2/3-bit dequant NEON 路径),AArch64 机器码) |
31
+
32
+ 结论:generic 库**零 AVX 指令**(可在任何 x86_64 上跑);AVX2 库无 AVX-512/zmm;
33
+ AVX-512 库的 182 条 BF16 指令与 `_cpu_caps` 的 `avx512_bf16` 门控一致(不会在无 BF16
34
+ 的机器上误载入)。
35
+
36
+ ## 运行时自检(C 内核独立自检,7 项:quantize 往返 / gemm_8bit / gemv_4bit /
37
+ AdamW8bit / 向量-标量叉验证 / 3-bit dequant / 2-bit dequant)
38
+
39
+ | 变体 | 运行方式 | 结果 |
40
+ |---|---|---|
41
+ | x86_64 AVX2 | 本机(R5-4500U,AVX2) | **7/7 PASS** |
42
+ | x86_64 generic | 本机 | **7/7 PASS** |
43
+ | x86_64 AVX-512 | 本机无 AVX-512 → 静态审计;EPYC 9Y24 容器真机 | 静态 182 BF16 指令;EPYC 上 **5/5 PASS**、GDN 5/5(见 `EPYC_VALIDATION.md`) |
44
+ | aarch64 musl | qemu-aarch64(x86 开发机) | **7/7 PASS** |
45
+
46
+ ## 边界
47
+
48
+ - aarch64 是**架构级**验证(qemu + 静态 musl),不是 Android/bionic 真机;Termux 真机
49
+ 历史结论仍有效。
50
+ - aarch64 gnu 库当前 zig 默认 glibc 目标为 2.27(若将来发 aarch64 manylinux wheel,
51
+ 需显式 `-target aarch64-linux-gnu.2.17`)。
52
+ - AVX-512 运行时验证依赖真实硬件(EPYC 容器),本机只能做静态/编译检查。
53
+
54
+ ## 2/3-bit dequant 内核(2026-10-19 加入 csrc)
55
+
56
+ 从 2bit/3bit 训练原型移植进正式 `csrc`:
57
+ - `cpu_ops.h` / `cpu_ops.cpp`:`dequantize_3bit_cpu` / `dequantize_2bit_cpu`(按架构分派:
58
+ ARM NEON / x86 AVX2 / 标量回退),另有 `*_scalar_cpu` 参考实现;
59
+ - `pythonInterface.cpp`:导出 `cbdequantize_3bit_cpu` / `cbdequantize_2bit_cpu`(供 ctypes/未来
60
+ Python API 使用);
61
+ - `selftest_cpu.c`:新增第 6/7 项——C 里量化后分别用标量参考与 AVX2/NEON 路径反量化,
62
+ 要求 maxdiff < 1e-6。x86 AVX2、generic、aarch64 qemu 全部 7/7 PASS。
63
+
64
+ aarch64 库的 NEON 类指令从 570 条增加到 **674 条**(新增 dequant NEON 路径的直接证据)。
65
+
66
+ ## 8. csrc fused 3-bit GEMM(第 8 项自检)
67
+
68
+ `q3bit_gemm_cpu` / `cbq3bit_gemm_cpu`(`cpu_ops.cpp` + `pythonInterface.cpp`):
69
+ `C[M,N] = A[M,K] @ dequant_3bit(B)[N,K]^T`,按 B 行 tile 现场解包到 L2 缓冲,
70
+ 不物化 fp32 权重矩阵。自检第 8 项要求它与标量参考的相对误差 < 1e-4;x86 AVX2/generic、
71
+ aarch64 qemu 全部 **8/8 PASS**。性能见 `bench_results/2bit3bit_proto.md` §7
72
+ (lm_head 形状:比 fp32 matmul 快 14–22%,内存 -89%)。
73
+
74
+ ## 9. 训练反向的转置融合 GEMM(第 9 项自检)
75
+
76
+ `q3bit_gemm_t_cpu` / `cbq3bit_gemm_t_cpu`:`dX[M,K] = dY[M,N] @ dequant(B)[N,K]`,线程私有
77
+ dX 部分和再归约。自检第 9 项(transposed fused GEMM vs 标量参考,rel < 1e-4):x86 AVX2 原生、
78
+ x86 generic、aarch64 qemu 全部 **9/9 PASS**。性能见 `bench_results/2bit3bit_proto.md` §8
79
+ (反向比 oneDNN fp32 慢 2.4×,暂不替换;前向融合仍严格更优)。