bitsandbytes-cpu-fork 0.50.2.dev6__py3-none-win_amd64.whl → 0.50.2.dev9__py3-none-win_amd64.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- bitsandbytes/__init__.py +4 -1
- bitsandbytes/efst.py +7 -2
- bitsandbytes/libbitsandbytes_cpu.dll +0 -0
- bitsandbytes/optim/__init__.py +4 -0
- bitsandbytes/optim/adamw4bit.py +239 -0
- bitsandbytes/q3bit.py +481 -0
- bitsandbytes_cpu_cli.py +67 -17
- bitsandbytes_cpu_fork-0.50.2.dev9.dist-info/METADATA +346 -0
- {bitsandbytes_cpu_fork-0.50.2.dev6.dist-info → bitsandbytes_cpu_fork-0.50.2.dev9.dist-info}/RECORD +13 -11
- bitsandbytes_cpu_fork-0.50.2.dev6.dist-info/METADATA +0 -452
- {bitsandbytes_cpu_fork-0.50.2.dev6.dist-info → bitsandbytes_cpu_fork-0.50.2.dev9.dist-info}/WHEEL +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev6.dist-info → bitsandbytes_cpu_fork-0.50.2.dev9.dist-info}/entry_points.txt +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev6.dist-info → bitsandbytes_cpu_fork-0.50.2.dev9.dist-info}/licenses/LICENSE +0 -0
- {bitsandbytes_cpu_fork-0.50.2.dev6.dist-info → bitsandbytes_cpu_fork-0.50.2.dev9.dist-info}/top_level.txt +0 -0
|
@@ -1,452 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: bitsandbytes-cpu-fork
|
|
3
|
-
Version: 0.50.2.dev6
|
|
4
|
-
Summary: bitsandbytes with a pure-CPU training backend (AVX2 / NEON): 8-bit optimizers, fused GDN, gemm_8bit, disk balancer. No GPU required.
|
|
5
|
-
License-Expression: MIT
|
|
6
|
-
Project-URL: repository, https://github.com/Ganyvnanoaa09-0906/bitsandbytes-CPU
|
|
7
|
-
Project-URL: Upstream bitsandbytes, https://github.com/bitsandbytes-foundation/bitsandbytes
|
|
8
|
-
Keywords: cpu,optimizers,optimization,8-bit,quantization,compression,avx2,neon
|
|
9
|
-
Classifier: Development Status :: 4 - Beta
|
|
10
|
-
Classifier: Intended Audience :: Developers
|
|
11
|
-
Classifier: Intended Audience :: Science/Research
|
|
12
|
-
Classifier: Operating System :: Microsoft :: Windows
|
|
13
|
-
Classifier: Operating System :: POSIX :: Linux
|
|
14
|
-
Classifier: Programming Language :: C++
|
|
15
|
-
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
-
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
-
Requires-Python: >=3.10
|
|
23
|
-
Description-Content-Type: text/markdown
|
|
24
|
-
License-File: LICENSE
|
|
25
|
-
Requires-Dist: numpy>=1.17
|
|
26
|
-
Requires-Dist: packaging>=20.9
|
|
27
|
-
Requires-Dist: psutil>=5.9
|
|
28
|
-
Provides-Extra: test
|
|
29
|
-
Requires-Dist: pytest~=8.3; extra == "test"
|
|
30
|
-
Dynamic: license-file
|
|
31
|
-
|
|
32
|
-
> **This is the CPU fork** — `bitsandbytes-cpu-fork`, not upstream `bitsandbytes`.
|
|
33
|
-
> It replaces the CUDA kernels with a pure-CPU backend (AVX2 on x86-64, NEON on ARM64):
|
|
34
|
-
> 8-bit optimizers, fused Gated DeltaNet, `gemm_8bit`, block-wise 4/8-bit quantisation.
|
|
35
|
-
> No GPU, no CUDA toolkit, no NVIDIA driver. Full guide:
|
|
36
|
-
> [`docs_cpu/QUICKSTART.md`](docs_cpu/QUICKSTART.md) (中文) ·
|
|
37
|
-
> [`docs_cpu/QUICKSTART_EN.md`](docs_cpu/QUICKSTART_EN.md).
|
|
38
|
-
|
|
39
|
-
## Install
|
|
40
|
-
|
|
41
|
-
**`torch` is deliberately not a dependency.** On PyPI the name `torch` resolves to the
|
|
42
|
-
CUDA build, so declaring it made this CPU-only library download roughly 2.5 GB of
|
|
43
|
-
`nvidia-*` wheels — cublas, cudnn, nccl, nvshmem and thirteen more — onto machines with
|
|
44
|
-
no NVIDIA device at all. Measured on a clean Ubuntu install. `install_requires` cannot
|
|
45
|
-
express "the CPU build", because that lives on a different index, so install it first:
|
|
46
|
-
|
|
47
|
-
```bash
|
|
48
|
-
pip install torch --index-url https://download.pytorch.org/whl/cpu
|
|
49
|
-
pip install bitsandbytes-cpu-fork
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
**Linux wheels.** This release ships two Linux x86_64 wheels: a native one
|
|
53
|
-
(`manylinux_2_39`) with the OpenMP-parallel kernels, and a portable one
|
|
54
|
-
(`manylinux_2_17`, single-threaded) for older distributions. pip picks the best one
|
|
55
|
-
for the host automatically. If neither matches your glibc, install from the sdist
|
|
56
|
-
(it compiles `csrc` with the local toolchain via `build_linux.sh`).
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
### Windows
|
|
60
|
-
|
|
61
|
-
A wheel is published with the CPU library already built in, so no compiler and no
|
|
62
|
-
Visual C++ Redistributable are needed:
|
|
63
|
-
|
|
64
|
-
```bat
|
|
65
|
-
pip install bitsandbytes-cpu-fork
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
### Linux
|
|
69
|
-
|
|
70
|
-
Only the source distribution is published. `pip` builds it, and `setup.py` runs
|
|
71
|
-
`build_linux.sh` for you, so the library is compiled against your own glibc rather than
|
|
72
|
-
shipped prebuilt. That needs a C++ compiler with OpenMP:
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
sudo apt install g++ libomp-dev # Debian / Ubuntu
|
|
76
|
-
sudo dnf install gcc-c++ libomp-devel # Fedora / RHEL
|
|
77
|
-
sudo pacman -S gcc openmp # Arch
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
If the toolchain is missing the build **stops and names that command**, rather than
|
|
81
|
-
producing a wheel that installs cleanly and then fails at `import bitsandbytes`.
|
|
82
|
-
|
|
83
|
-
To build by hand instead, or to check the kernels without torch:
|
|
84
|
-
|
|
85
|
-
```bash
|
|
86
|
-
bash build_linux.sh --selftest
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
### Verify the install
|
|
90
|
-
|
|
91
|
-
```bash
|
|
92
|
-
python -m bitsandbytes
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
### Then ask it what it can do
|
|
96
|
-
|
|
97
|
-
`pip` installs a `bitsandbytes-cpu` command (also available as `bnb-cpu`). It is the
|
|
98
|
-
reference for this fork: what the hardware is, whether the kernels load, and **how to
|
|
99
|
-
call every feature it adds** — each entry carries the exact call, and every Python
|
|
100
|
-
example in it is executed before release.
|
|
101
|
-
|
|
102
|
-
```bash
|
|
103
|
-
bitsandbytes-cpu detect # CPU model, cores, SIMD, RAM, recommended thread count
|
|
104
|
-
bitsandbytes-cpu selftest # runs a 4-bit layer, an 8-bit optimizer step, the GDN kernel
|
|
105
|
-
bitsandbytes-cpu help # the whole reference
|
|
106
|
-
bitsandbytes-cpu help 8bitopt # one section: the 8-bit optimizer
|
|
107
|
-
bitsandbytes-cpu help 4bit # one section: 4-bit layers on a CPU
|
|
108
|
-
bitsandbytes-cpu help kernels # one section: the fused kernels, called directly
|
|
109
|
-
bitsandbytes-cpu doctor # which .dll/.so will load, and which symbols it has
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
Section keys: `intro start 4bit qlora 8bitopt kernels toolkit threads memory gdn disks
|
|
113
|
-
layers optim functional notes`. The same tables are published as
|
|
114
|
-
[`docs_cpu/API_REFERENCE.md`](docs_cpu/API_REFERENCE.md)
|
|
115
|
-
([English](docs_cpu/API_REFERENCE_EN.md)).
|
|
116
|
-
|
|
117
|
-
---
|
|
118
|
-
|
|
119
|
-
<p align="center"><img src="https://avatars.githubusercontent.com/u/175231607?s=200&v=4" alt=""></p>
|
|
120
|
-
<h1 align="center">bitsandbytes</h1>
|
|
121
|
-
<p align="center">
|
|
122
|
-
<a href="https://github.com/bitsandbytes-foundation/bitsandbytes/main/LICENSE"><img alt="License" src="https://img.shields.io/github/license/bitsandbytes-foundation/bitsandbytes.svg?color=blue"></a>
|
|
123
|
-
<a href="https://pepy.tech/project/bitsandbytes"><img alt="Downloads" src="https://static.pepy.tech/badge/bitsandbytes/month"></a>
|
|
124
|
-
<a href="https://github.com/bitsandbytes-foundation/bitsandbytes/actions/workflows/tests-nightly.yml"><img alt="Nightly Unit Tests" src="https://img.shields.io/github/actions/workflow/status/bitsandbytes-foundation/bitsandbytes/tests-nightly.yml?logo=github&label=Nightly%20Tests"></a>
|
|
125
|
-
<a href="https://github.com/bitsandbytes-foundation/bitsandbytes/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/bitsandbytes-foundation/bitsandbytes"></a>
|
|
126
|
-
<a href="https://pypi.org/project/bitsandbytes/"><img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/bitsandbytes"></a>
|
|
127
|
-
</p>
|
|
128
|
-
|
|
129
|
-
`bitsandbytes` enables accessible large language models via k-bit quantization for PyTorch. We provide three main features for dramatically reducing memory consumption for inference and training:
|
|
130
|
-
|
|
131
|
-
* 8-bit optimizers uses block-wise quantization to maintain 32-bit performance at a small fraction of the memory cost.
|
|
132
|
-
* LLM.int8() or 8-bit quantization enables large language model inference with only half the required memory and without any performance degradation. This method is based on vector-wise quantization to quantize most features to 8-bits and separately treating outliers with 16-bit matrix multiplication.
|
|
133
|
-
* QLoRA or 4-bit quantization enables large language model training with several memory-saving techniques that don't compromise performance. This method quantizes a model to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allow training.
|
|
134
|
-
|
|
135
|
-
The library includes quantization primitives for 8-bit & 4-bit operations, through `bitsandbytes.nn.Linear8bitLt` and `bitsandbytes.nn.Linear4bit` and 8-bit optimizers through `bitsandbytes.optim` module.
|
|
136
|
-
|
|
137
|
-
## System Requirements
|
|
138
|
-
bitsandbytes has the following minimum requirements for all platforms:
|
|
139
|
-
|
|
140
|
-
* Python 3.10+
|
|
141
|
-
* [PyTorch](https://pytorch.org/get-started/locally/) 2.4+
|
|
142
|
-
* _Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience._
|
|
143
|
-
|
|
144
|
-
#### Accelerator support:
|
|
145
|
-
|
|
146
|
-
<small>Note: this table reflects the status of the current development branch. For the latest stable release, see the
|
|
147
|
-
[document in the 0.50.0 tag](https://github.com/bitsandbytes-foundation/bitsandbytes/blob/0.50.0/README.md#accelerator-support).
|
|
148
|
-
</small>
|
|
149
|
-
|
|
150
|
-
##### Legend:
|
|
151
|
-
🚧 = Planned |
|
|
152
|
-
〰️ = Partially Supported |
|
|
153
|
-
✅ = Supported |
|
|
154
|
-
❌ = Not Supported
|
|
155
|
-
|
|
156
|
-
<table>
|
|
157
|
-
<thead>
|
|
158
|
-
<tr>
|
|
159
|
-
<th>Platform</th>
|
|
160
|
-
<th>Accelerator</th>
|
|
161
|
-
<th>Hardware Requirements</th>
|
|
162
|
-
<th>LLM.int8()</th>
|
|
163
|
-
<th>QLoRA 4-bit</th>
|
|
164
|
-
<th>8-bit Optimizers</th>
|
|
165
|
-
</tr>
|
|
166
|
-
</thead>
|
|
167
|
-
<tbody>
|
|
168
|
-
<tr>
|
|
169
|
-
<td colspan="6">🐧 <strong>Linux, glibc >= 2.24</strong></td>
|
|
170
|
-
</tr>
|
|
171
|
-
<tr>
|
|
172
|
-
<td align="right">x86-64</td>
|
|
173
|
-
<td>◻️ CPU</td>
|
|
174
|
-
<td>Minimum: AVX2<br>Optimized: AVX512F, AVX512BF16</td>
|
|
175
|
-
<td>✅</td>
|
|
176
|
-
<td>✅</td>
|
|
177
|
-
<td>✅</td>
|
|
178
|
-
</tr>
|
|
179
|
-
<tr>
|
|
180
|
-
<td></td>
|
|
181
|
-
<td>🟩 NVIDIA GPU <br><code>cuda</code></td>
|
|
182
|
-
<td>SM60+ minimum<br>SM75+ recommended</td>
|
|
183
|
-
<td>✅</td>
|
|
184
|
-
<td>✅</td>
|
|
185
|
-
<td>✅</td>
|
|
186
|
-
</tr>
|
|
187
|
-
<tr>
|
|
188
|
-
<td></td>
|
|
189
|
-
<td>🟥 AMD GPU <br><code>cuda</code></td>
|
|
190
|
-
<td>
|
|
191
|
-
CDNA: gfx908, gfx90a, gfx942, gfx950, gfx1250<br>
|
|
192
|
-
RDNA: gfx101X, gfx103X, gfx110X, gfx115X, gfx120X
|
|
193
|
-
</td>
|
|
194
|
-
<td>✅</td>
|
|
195
|
-
<td>✅</td>
|
|
196
|
-
<td>✅</td>
|
|
197
|
-
</tr>
|
|
198
|
-
<tr>
|
|
199
|
-
<td></td>
|
|
200
|
-
<td>🟦 Intel GPU <br><code>xpu</code></td>
|
|
201
|
-
<td>
|
|
202
|
-
Data Center GPU Max Series<br>
|
|
203
|
-
Arc A-Series (Alchemist)<br>
|
|
204
|
-
Arc B-Series (Battlemage)
|
|
205
|
-
</td>
|
|
206
|
-
<td>✅</td>
|
|
207
|
-
<td>✅</td>
|
|
208
|
-
<td>✅</td>
|
|
209
|
-
</tr>
|
|
210
|
-
<tr>
|
|
211
|
-
<td></td>
|
|
212
|
-
<td>🟪 Intel Gaudi <br><code>hpu</code></td>
|
|
213
|
-
<td>Gaudi2, Gaudi3</td>
|
|
214
|
-
<td>✅</td>
|
|
215
|
-
<td>〰️</td>
|
|
216
|
-
<td>❌</td>
|
|
217
|
-
</tr>
|
|
218
|
-
<tr>
|
|
219
|
-
<td align="right">aarch64</td>
|
|
220
|
-
<td>◻️ CPU</td>
|
|
221
|
-
<td></td>
|
|
222
|
-
<td>✅ *</td>
|
|
223
|
-
<td>✅</td>
|
|
224
|
-
<td>✅</td>
|
|
225
|
-
</tr>
|
|
226
|
-
<tr>
|
|
227
|
-
<td></td>
|
|
228
|
-
<td>🟩 NVIDIA GPU <br><code>cuda</code></td>
|
|
229
|
-
<td>SM75+</td>
|
|
230
|
-
<td>✅</td>
|
|
231
|
-
<td>✅</td>
|
|
232
|
-
<td>✅</td>
|
|
233
|
-
</tr>
|
|
234
|
-
<tr>
|
|
235
|
-
<td colspan="6">🪟 <strong>Windows 11 / Windows Server 2022+</strong></td>
|
|
236
|
-
</tr>
|
|
237
|
-
<tr>
|
|
238
|
-
<td align="right">x86-64</td>
|
|
239
|
-
<td>◻️ CPU</td>
|
|
240
|
-
<td>AVX2</td>
|
|
241
|
-
<td>✅</td>
|
|
242
|
-
<td>✅</td>
|
|
243
|
-
<td>✅</td>
|
|
244
|
-
</tr>
|
|
245
|
-
<tr>
|
|
246
|
-
<td></td>
|
|
247
|
-
<td>🟩 NVIDIA GPU <br><code>cuda</code></td>
|
|
248
|
-
<td>SM60+ minimum<br>SM75+ recommended</td>
|
|
249
|
-
<td>✅</td>
|
|
250
|
-
<td>✅</td>
|
|
251
|
-
<td>✅</td>
|
|
252
|
-
</tr>
|
|
253
|
-
<tr>
|
|
254
|
-
<td></td>
|
|
255
|
-
<td>🟥 AMD GPU <br><code>cuda</code></td>
|
|
256
|
-
<td>
|
|
257
|
-
CDNA: gfx908, gfx90a<br>
|
|
258
|
-
RDNA: gfx101X, gfx103X, gfx110X, gfx115X, gfx120X
|
|
259
|
-
</td>
|
|
260
|
-
<td>✅</td>
|
|
261
|
-
<td>✅</td>
|
|
262
|
-
<td>✅</td>
|
|
263
|
-
</tr>
|
|
264
|
-
<tr>
|
|
265
|
-
<td></td>
|
|
266
|
-
<td>🟦 Intel GPU <br><code>xpu</code></td>
|
|
267
|
-
<td>
|
|
268
|
-
Arc A-Series (Alchemist) <br>
|
|
269
|
-
Arc B-Series (Battlemage)
|
|
270
|
-
</td>
|
|
271
|
-
<td>✅</td>
|
|
272
|
-
<td>✅</td>
|
|
273
|
-
<td>✅</td>
|
|
274
|
-
</tr>
|
|
275
|
-
<tr>
|
|
276
|
-
<td align="right">arm64</td>
|
|
277
|
-
<td>◻️ CPU</td>
|
|
278
|
-
<td></td>
|
|
279
|
-
<td>✅</td>
|
|
280
|
-
<td>✅</td>
|
|
281
|
-
<td>✅</td>
|
|
282
|
-
</tr>
|
|
283
|
-
<tr>
|
|
284
|
-
<td></td>
|
|
285
|
-
<td>🟩 NVIDIA GPU <br><code>cuda</code></td>
|
|
286
|
-
<td>SM121</td>
|
|
287
|
-
<td>✅</td>
|
|
288
|
-
<td>✅</td>
|
|
289
|
-
<td>✅</td>
|
|
290
|
-
</tr>
|
|
291
|
-
<tr>
|
|
292
|
-
<td colspan="6">🍎 <strong>macOS 14+</strong></td>
|
|
293
|
-
</tr>
|
|
294
|
-
<tr>
|
|
295
|
-
<td align="right">arm64</td>
|
|
296
|
-
<td>◻️ CPU</td>
|
|
297
|
-
<td>Apple M1+</td>
|
|
298
|
-
<td>✅ *</td>
|
|
299
|
-
<td>✅</td>
|
|
300
|
-
<td>✅</td>
|
|
301
|
-
</tr>
|
|
302
|
-
<tr>
|
|
303
|
-
<td></td>
|
|
304
|
-
<td>⬜ Metal <br><code>mps</code></td>
|
|
305
|
-
<td>Apple M1+</td>
|
|
306
|
-
<td>✅ *</td>
|
|
307
|
-
<td>✅</td>
|
|
308
|
-
<td>🚧</td>
|
|
309
|
-
</tbody>
|
|
310
|
-
</table>
|
|
311
|
-
<sup>* While supported, these marked features may lack in performance optimizations.</sup>
|
|
312
|
-
|
|
313
|
-
## :book: Documentation
|
|
314
|
-
* [Official Documentation](https://huggingface.co/docs/bitsandbytes/main)
|
|
315
|
-
* 🤗 [Transformers](https://huggingface.co/docs/transformers/quantization/bitsandbytes)
|
|
316
|
-
* 🤗 [Diffusers](https://huggingface.co/docs/diffusers/quantization/bitsandbytes)
|
|
317
|
-
* 🤗 [PEFT](https://huggingface.co/docs/peft/developer_guides/quantization#quantize-a-model)
|
|
318
|
-
|
|
319
|
-
## :computer: CPU Backend Extensions (this fork)
|
|
320
|
-
|
|
321
|
-
This fork adds **training-grade CPU kernels** to the bitsandbytes CPU backend, for
|
|
322
|
-
machines **without NVIDIA GPUs** (AVX2 only / ARM64 NEON, e.g. consumer laptops,
|
|
323
|
-
Mini-PCs). The upstream CPU backend is inference-oriented; these
|
|
324
|
-
extensions make pure-CPU LoRA / quantization / diffusion training practical.
|
|
325
|
-
|
|
326
|
-
### Install (Windows, no compiler needed)
|
|
327
|
-
|
|
328
|
-
```bash
|
|
329
|
-
pip install bitsandbytes-cpu-fork
|
|
330
|
-
```
|
|
331
|
-
|
|
332
|
-
The wheel bundles the prebuilt CPU kernel and its OpenMP runtime, so nothing has to
|
|
333
|
-
be built after installing; `pip` pulls `torch` and `numpy` itself. One wheel covers
|
|
334
|
-
every Python 3 version on 64-bit Windows. The import name is unchanged, so existing
|
|
335
|
-
code and the Transformers / PEFT / Diffusers integrations keep working as-is:
|
|
336
|
-
|
|
337
|
-
```python
|
|
338
|
-
import bitsandbytes as bnb # the fork, not upstream
|
|
339
|
-
print(bnb.__version__) # 0.50.2.dev6
|
|
340
|
-
```
|
|
341
|
-
|
|
342
|
-
Building from source instead (any platform) is described in
|
|
343
|
-
[`docs_cpu/QUICKSTART.md`](docs_cpu/QUICKSTART.md) §2 and §2A.
|
|
344
|
-
|
|
345
|
-
> 📚 **Docs** (`docs_cpu/`):
|
|
346
|
-
> - `TECHNICAL_GUIDE.md` — **技术文档**:改了什么、每个内核的架构与细节(面向技术人员)
|
|
347
|
-
> - `QUICKSTART.md` — **快速入门手册**:命令、用途与参数(面向首次使用者)
|
|
348
|
-
> - `TECH_REPORT.md` — **技术报告**:完整方法论、实验数据与结论(含负面结论)
|
|
349
|
-
> - 灾难恢复指南已并入 `QUICKSTART.md` §9。
|
|
350
|
-
> - 🌐 **English**: 英文为**独立文件** —— `QUICKSTART_EN.md`、`TECHNICAL_GUIDE_EN.md`、
|
|
351
|
-
> `TECH_REPORT_EN.md`(内容与中文版对应,以中文为准;灾难恢复同样在 `QUICKSTART_EN.md` §9)。
|
|
352
|
-
>
|
|
353
|
-
> 🌐 **English README**: see [`README_EN.md`](README_EN.md).
|
|
354
|
-
>
|
|
355
|
-
> > **关于硬盘均衡负载**:它能在内存紧张时帮助维持训练,但**不能替代 Windows 虚拟内存**。
|
|
356
|
-
> > 训练期间仍**建议保留至少 1GB 虚拟内存**以应对突发内存峰值。
|
|
357
|
-
|
|
358
|
-
### What's added
|
|
359
|
-
|
|
360
|
-
| Kernel | Entry point | Purpose |
|
|
361
|
-
|---|---|---|
|
|
362
|
-
| **Gated DeltaNet fwd/bwd** | `gdn_fwd_cpu` / `gdn_bwd_cpu` (csrc/cpu_gdn.cpp) | Fused Gated DeltaNet (Qwen3-Next/3.5 linear attention) — ~35× vs per-step Python loop; chunk-checkpointed backward |
|
|
363
|
-
| **Fused 8-bit blockwise dequant GEMM** | `cgemm_8bit_inference_cpu_fp32` (`bitsandbytes::gemm_8bit`, `bnb.functional.fused_dequant_linear_8bit`) | `out = A @ dequant8(B)` with weights kept uint8 (¼ DRAM traffic), no fp32 temporary |
|
|
364
|
-
| **Fused 8-bit optimizer** | `coptimizer_update_8bit_blockwise_cpu` (`bnb.optim.AdamW8bit` etc.) | Single-pass dequant→update→requant; optimizer state memory ≈ 1/3.8 of fp32 |
|
|
365
|
-
| **Blockwise 8/4-bit quant & dequant** | `cquantize_blockwise_cpu_*` / `cdequantize_blockwise_cpu_*` | AVX2 LUT quantize + dequantize; NF4/FP4 kernels with NEON/AVX2 paths |
|
|
366
|
-
| **4-bit inference GEMV** | `cgemv_4bit_inference_cpu_*` | Fused 4-bit dequant GEMV/GEMM for AVX2 machines (symbol alias fixed) |
|
|
367
|
-
| **GDN runtime patch** | `bitsandbytes/gdn_cpu.py` | `patch_transformers()` / `patch_fla()` — route transformers GDN slow paths to the fused kernel |
|
|
368
|
-
| **iGPU (DirectML) block-level executor** | `gpu_scheduler.py` (`IgpuExecutor`, `train.py --igpu`) | Experimental: frozen weights resident on the DirectML iGPU, forward/backward on-device, **one sync per step**; **off by default** — measured **+30~35%** for GEMM-dense seq≤256 training, **negative** at long sequences (driver limits) |
|
|
369
|
-
|
|
370
|
-
### Build
|
|
371
|
-
|
|
372
|
-
```bash
|
|
373
|
-
# Linux (x86_64 AVX2 or aarch64 NEON, g++/clang + OpenMP)
|
|
374
|
-
# deps: sudo apt install g++ libomp-dev (Debian/Ubuntu)
|
|
375
|
-
bash build_linux.sh # -> bitsandbytes/libbitsandbytes_cpu.so
|
|
376
|
-
bash build_linux.sh --selftest # also build+run torch-free C self-test
|
|
377
|
-
|
|
378
|
-
# Windows (MSVC, manual cl build, no CMake)
|
|
379
|
-
build_manual\build_manual.bat amd # (in VS x64 Native Tools prompt)
|
|
380
|
-
```
|
|
381
|
-
|
|
382
|
-
> `.gitattributes` 强制 C/C++ 源码与构建脚本用 LF 换行,clone 到 Windows/Linux 均可
|
|
383
|
-
> 直接编译(CRLF 会让 MSVC 吞行、Linux `g++` 报 `bad interpreter`)。
|
|
384
|
-
> `disk_balancer`(`--flash`)跨平台:Windows 用 ctypes;Linux 用 `/proc/mounts` + psutil + `/sys/block/*/queue/rotational`;**WSL 已支持**——缓存只落在 WSL 自己的 ext4(默认 `~/.cache/bnb_flash_cache`),9P 宿主盘 `/mnt/[盘符]`、网络盘、只读挂载、tmpfs 一律拒绝(显式 `--flash_paths` 指过去会直接报错)。零拷贝写盘队列有在飞字节上限(`max_inflight_bytes`),卸载不会把源张量内存偷偷留住。
|
|
385
|
-
|
|
386
|
-
### Prebuilt wheels (optional)
|
|
387
|
-
|
|
388
|
-
预编译的 wheel(`pip install bitsandbytes-<ver>-py3-none-<platform>.whl`)可从 **Release**
|
|
389
|
-
获取,无需自己编译(Windows `win_amd64` / Linux `linux_x86_64` / aarch64
|
|
390
|
-
`linux_aarch64`)。wheel 内含 `bitsandbytes/` 包 + 编译好的 native 库
|
|
391
|
-
(`.dll`/`.so`),安装即用。
|
|
392
|
-
|
|
393
|
-
### Self-test (no torch required)
|
|
394
|
-
|
|
395
|
-
```bash
|
|
396
|
-
# C-level kernel self-test: quantize roundtrip / gemm_8bit / 4bit GEMV / 8bit optimizer
|
|
397
|
-
bash build_linux.sh --selftest
|
|
398
|
-
```
|
|
399
|
-
|
|
400
|
-
### Notes
|
|
401
|
-
|
|
402
|
-
- This fork is based on bitsandbytes v0.45.1 (MIT, Facebook) and keeps the upstream
|
|
403
|
-
MIT license. See `NOTICE.md` for contributions.
|
|
404
|
-
- The CPU backend keeps the upstream NEON/AVX paths for plain quant/dequant;
|
|
405
|
-
the **fused training kernels** (GDN, gemm_8bit, 8-bit optimizer) are the
|
|
406
|
-
addition of this fork.
|
|
407
|
-
- Not recommended for use without the compiled native library — build first
|
|
408
|
-
(Windows: `build_manual\build_manual.bat amd`, Linux: `build_linux.sh`).
|
|
409
|
-
|
|
410
|
-
## :heart: Sponsors
|
|
411
|
-
The continued maintenance and development of `bitsandbytes` is made possible thanks to the generous support of our sponsors. Their contributions help ensure that we can keep improving the project and delivering valuable updates to the community.
|
|
412
|
-
|
|
413
|
-
<kbd><a href="https://hf.co" target="_blank"><img width="100" src="https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo.svg" alt="Hugging Face"></a></kbd>
|
|
414
|
-
|
|
415
|
-
## License
|
|
416
|
-
`bitsandbytes` is MIT licensed.
|
|
417
|
-
|
|
418
|
-
## How to cite us
|
|
419
|
-
If you found this library useful, please consider citing our work:
|
|
420
|
-
|
|
421
|
-
### QLoRA
|
|
422
|
-
|
|
423
|
-
```bibtex
|
|
424
|
-
@article{dettmers2023qlora,
|
|
425
|
-
title={Qlora: Efficient finetuning of quantized llms},
|
|
426
|
-
author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
|
|
427
|
-
journal={arXiv preprint arXiv:2305.14314},
|
|
428
|
-
year={2023}
|
|
429
|
-
}
|
|
430
|
-
```
|
|
431
|
-
|
|
432
|
-
### LLM.int8()
|
|
433
|
-
|
|
434
|
-
```bibtex
|
|
435
|
-
@article{dettmers2022llmint8,
|
|
436
|
-
title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale},
|
|
437
|
-
author={Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke},
|
|
438
|
-
journal={arXiv preprint arXiv:2208.07339},
|
|
439
|
-
year={2022}
|
|
440
|
-
}
|
|
441
|
-
```
|
|
442
|
-
|
|
443
|
-
### 8-bit Optimizers
|
|
444
|
-
|
|
445
|
-
```bibtex
|
|
446
|
-
@article{dettmers2022optimizers,
|
|
447
|
-
title={8-bit Optimizers via Block-wise Quantization},
|
|
448
|
-
author={Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},
|
|
449
|
-
journal={9th International Conference on Learning Representations, ICLR},
|
|
450
|
-
year={2022}
|
|
451
|
-
}
|
|
452
|
-
```
|
{bitsandbytes_cpu_fork-0.50.2.dev6.dist-info → bitsandbytes_cpu_fork-0.50.2.dev9.dist-info}/WHEEL
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|