gefen 1.0.0__tar.gz

gefen-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nadav Benedek
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gefen-1.0.0/MANIFEST.in

@@ -0,0 +1,3 @@
+# Include CUDA/C++ sources in source distributions so PyTorch JIT can build kernels after install.
+recursive-include kernels *.py *.cu *.cpp *.md
+prune kernels/.build

gefen-1.0.0/PKG-INFO

@@ -0,0 +1,117 @@
+Metadata-Version: 2.4
+Name: gefen
+Version: 1.0.0
+Summary: Gefen optimizer for memory-efficient PyTorch training
+Author: Nadav Benedek
+License: MIT License
+        
+        Copyright (c) 2026 Nadav Benedek
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Repository, https://github.com/ndvbd/Gefen
+Project-URL: Issues, https://github.com/ndvbd/Gefen/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: ninja
+Requires-Dist: numpy
+Requires-Dist: numba
+Requires-Dist: torch
+Dynamic: license-file
+
+# [Gefen: Optimized Stochastic Optimizer](https://arxiv.org/pdf/2606.13894)
+
+Gefen is a drop-in replacement for the AdamW optimizer for memory-efficient
+training. It keeps the familiar AdamW training recipe while dramatically
+reducing optimizer-state memory: an 8x reduction in AdamW memory footprint, or
+about 6.5 GiB saved per billion parameters, while maintaining AdamW-level
+performance. The reduced memory footprint lets you train larger models or use
+larger batch sizes and, as a result, achieve higher training throughput.
+All it takes is changing two lines of code: import Gefen and replace the AdamW
+optimizer constructor.
+
+## Installation
+
+Install from source:
+
+```bash
+git clone https://github.com/ndvbd/Gefen
+cd Gefen
+pip install -e .
+```
+
+On the first CUDA run, Gefen builds its fused CUDA kernels with PyTorch JIT and
+`nvcc`. This can take a few minutes. Later runs reuse the cached build for the
+same Python, PyTorch, CUDA version, and Gefen source checkout.
+
+This keeps the source install lightweight, but it requires a CUDA toolkit and
+host compiler compatible with your PyTorch installation. In the future, we plan
+to make this smoother with prebuilt wheels for common PyTorch/CUDA combinations.
+
+## Quick Start
+
+```python
+import torch
+from gefen import Gefen
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = torch.nn.Linear(128, 10).to(device)
+
+# optimizer = torch.optim.AdamW(
+optimizer = Gefen(  # Replace AdamW with Gefen:
+    model.parameters(),
+    lr=1e-3,
+    betas=(0.9, 0.999),
+    eps=1e-8,
+    weight_decay=0.0,
+)
+
+inputs = torch.randn(32, 128, device=device)
+targets = torch.randint(0, 10, (32,), device=device)
+
+logits = model(inputs)
+loss = torch.nn.functional.cross_entropy(logits, targets)
+loss.backward()
+
+optimizer.step()
+optimizer.zero_grad(set_to_none=True)
+
+print('Finished successfully.')
+```
+
+
+## Citation
+
+```bibtex
+@article{benedek2026gefen,
+  title={Gefen: Optimized Stochastic Optimizer},
+  author={Benedek, Nadav and Koren, Tomer and Fried, Ohad},
+  journal={arXiv preprint arXiv:2606.13894},
+  year={2026}
+}
+```

gefen-1.0.0/README.md

@@ -0,0 +1,71 @@
+# [Gefen: Optimized Stochastic Optimizer](https://arxiv.org/pdf/2606.13894)
+
+Gefen is a drop-in replacement for the AdamW optimizer for memory-efficient
+training. It keeps the familiar AdamW training recipe while dramatically
+reducing optimizer-state memory: an 8x reduction in AdamW memory footprint, or
+about 6.5 GiB saved per billion parameters, while maintaining AdamW-level
+performance. The reduced memory footprint lets you train larger models or use
+larger batch sizes and, as a result, achieve higher training throughput.
+All it takes is changing two lines of code: import Gefen and replace the AdamW
+optimizer constructor.
+
+## Installation
+
+Install from source:
+
+```bash
+git clone https://github.com/ndvbd/Gefen
+cd Gefen
+pip install -e .
+```
+
+On the first CUDA run, Gefen builds its fused CUDA kernels with PyTorch JIT and
+`nvcc`. This can take a few minutes. Later runs reuse the cached build for the
+same Python, PyTorch, CUDA version, and Gefen source checkout.
+
+This keeps the source install lightweight, but it requires a CUDA toolkit and
+host compiler compatible with your PyTorch installation. In the future, we plan
+to make this smoother with prebuilt wheels for common PyTorch/CUDA combinations.
+
+## Quick Start
+
+```python
+import torch
+from gefen import Gefen
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = torch.nn.Linear(128, 10).to(device)
+
+# optimizer = torch.optim.AdamW(
+optimizer = Gefen(  # Replace AdamW with Gefen:
+    model.parameters(),
+    lr=1e-3,
+    betas=(0.9, 0.999),
+    eps=1e-8,
+    weight_decay=0.0,
+)
+
+inputs = torch.randn(32, 128, device=device)
+targets = torch.randint(0, 10, (32,), device=device)
+
+logits = model(inputs)
+loss = torch.nn.functional.cross_entropy(logits, targets)
+loss.backward()
+
+optimizer.step()
+optimizer.zero_grad(set_to_none=True)
+
+print('Finished successfully.')
+```
+
+
+## Citation
+
+```bibtex
+@article{benedek2026gefen,
+  title={Gefen: Optimized Stochastic Optimizer},
+  author={Benedek, Nadav and Koren, Tomer and Fried, Ohad},
+  journal={arXiv preprint arXiv:2606.13894},
+  year={2026}
+}
+```

gefen-1.0.0/__init__.py

@@ -0,0 +1,9 @@
+__all__ = ["Gefen", "kernels"]
+
+
+def __getattr__(name):
+    if name == "Gefen":
+        from .gefen import Gefen
+
+        return Gefen
+    raise AttributeError("module {!r} has no attribute {!r}".format(__name__, name))

gefen-1.0.0/gefen.egg-info/PKG-INFO

@@ -0,0 +1,117 @@
+Metadata-Version: 2.4
+Name: gefen
+Version: 1.0.0
+Summary: Gefen optimizer for memory-efficient PyTorch training
+Author: Nadav Benedek
+License: MIT License
+        
+        Copyright (c) 2026 Nadav Benedek
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Repository, https://github.com/ndvbd/Gefen
+Project-URL: Issues, https://github.com/ndvbd/Gefen/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: ninja
+Requires-Dist: numpy
+Requires-Dist: numba
+Requires-Dist: torch
+Dynamic: license-file
+
+# [Gefen: Optimized Stochastic Optimizer](https://arxiv.org/pdf/2606.13894)
+
+Gefen is a drop-in replacement for the AdamW optimizer for memory-efficient
+training. It keeps the familiar AdamW training recipe while dramatically
+reducing optimizer-state memory: an 8x reduction in AdamW memory footprint, or
+about 6.5 GiB saved per billion parameters, while maintaining AdamW-level
+performance. The reduced memory footprint lets you train larger models or use
+larger batch sizes and, as a result, achieve higher training throughput.
+All it takes is changing two lines of code: import Gefen and replace the AdamW
+optimizer constructor.
+
+## Installation
+
+Install from source:
+
+```bash
+git clone https://github.com/ndvbd/Gefen
+cd Gefen
+pip install -e .
+```
+
+On the first CUDA run, Gefen builds its fused CUDA kernels with PyTorch JIT and
+`nvcc`. This can take a few minutes. Later runs reuse the cached build for the
+same Python, PyTorch, CUDA version, and Gefen source checkout.
+
+This keeps the source install lightweight, but it requires a CUDA toolkit and
+host compiler compatible with your PyTorch installation. In the future, we plan
+to make this smoother with prebuilt wheels for common PyTorch/CUDA combinations.
+
+## Quick Start
+
+```python
+import torch
+from gefen import Gefen
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = torch.nn.Linear(128, 10).to(device)
+
+# optimizer = torch.optim.AdamW(
+optimizer = Gefen(  # Replace AdamW with Gefen:
+    model.parameters(),
+    lr=1e-3,
+    betas=(0.9, 0.999),
+    eps=1e-8,
+    weight_decay=0.0,
+)
+
+inputs = torch.randn(32, 128, device=device)
+targets = torch.randint(0, 10, (32,), device=device)
+
+logits = model(inputs)
+loss = torch.nn.functional.cross_entropy(logits, targets)
+loss.backward()
+
+optimizer.step()
+optimizer.zero_grad(set_to_none=True)
+
+print('Finished successfully.')
+```
+
+
+## Citation
+
+```bibtex
+@article{benedek2026gefen,
+  title={Gefen: Optimized Stochastic Optimizer},
+  author={Benedek, Nadav and Koren, Tomer and Fried, Ohad},
+  journal={arXiv preprint arXiv:2606.13894},
+  year={2026}
+}
+```

gefen-1.0.0/gefen.egg-info/SOURCES.txt

@@ -0,0 +1,33 @@
+LICENSE
+MANIFEST.in
+README.md
+__init__.py
+gefen.py
+partitioning.py
+pyproject.toml
+quantization.py
+setup.py
+./__init__.py
+./gefen.py
+./partitioning.py
+./quantization.py
+./setup.py
+gefen.egg-info/PKG-INFO
+gefen.egg-info/SOURCES.txt
+gefen.egg-info/dependency_links.txt
+gefen.egg-info/requires.txt
+gefen.egg-info/top_level.txt
+kernels/__init__.py
+kernels/_build_notice.py
+kernels/automatic_gefen_fused.py
+kernels/automatic_gefen_fused_binding.cpp
+kernels/automatic_gefen_fused_kernel.cu
+kernels/automatic_vmean.py
+kernels/automatic_vmean_binding.cpp
+kernels/automatic_vmean_kernel.cu
+kernels/exact_histogram_fused.py
+kernels/exact_histogram_fused_binding.cpp
+kernels/exact_histogram_fused_kernel.cu
+kernels/period_variance.py
+kernels/period_variance_binding.cpp
+kernels/period_variance_kernel.cu

gefen-1.0.0/gefen.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

gefen-1.0.0/gefen.egg-info/requires.txt

@@ -0,0 +1,4 @@
+ninja
+numpy
+numba
+torch

gefen-1.0.0/gefen.egg-info/top_level.txt

@@ -0,0 +1 @@
+gefen