torchzero 0.3.10__py3-none-any.whl → 0.3.13__py3-none-any.whl

torchzero-0.3.10.dist-info/METADATA

@@ -1,379 +0,0 @@
-Metadata-Version: 2.4
-Name: torchzero
-Version: 0.3.10
-Summary: Modular optimization library for PyTorch.
-Author-email: Ivan Nikishev <nkshv2@gmail.com>
-License: MIT License
-        
-        Copyright (c) 2024 inikishev
-        
-        Permission is hereby granted, free of charge, to any person obtaining a copy
-        of this software and associated documentation files (the "Software"), to deal
-        in the Software without restriction, including without limitation the rights
-        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-        copies of the Software, and to permit persons to whom the Software is
-        furnished to do so, subject to the following conditions:
-        
-        The above copyright notice and this permission notice shall be included in all
-        copies or substantial portions of the Software.
-        
-        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-        SOFTWARE.
-        
-Project-URL: Homepage, https://github.com/inikishev/torchzero
-Project-URL: Repository, https://github.com/inikishev/torchzero
-Project-URL: Issues, https://github.com/inikishev/torchzero/isses
-Keywords: optimization,optimizers,torch,neural networks,zeroth order,second order
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: torch
-Requires-Dist: numpy
-Requires-Dist: typing_extensions
-Dynamic: license-file
-
-![tests](https://github.com/inikishev/torchzero/actions/workflows/tests.yml/badge.svg)
-
-# torchzero
-
-**Modular optimization library for PyTorch**
-
-`torchzero` is a PyTorch library providing a highly modular framework for creating and experimenting with a huge number of various optimization algorithms - various momentum techniques, gradient clipping, gradient approximations, line searches, quasi newton methods and more. All algorithms are implemented as modules that can be chained together freely.
-
-NOTE: torchzero is in active development, currently docs are in a state of flux.
-
-## Installation
-
-```bash
-pip install torchzero
-```
-
-pip version is always the latest one. Or install from this repo
-
-```bash
-pip install git+https://github.com/inikishev/torchzero
-```
-
-**Dependencies:**
-
-* Python >= 3.10
-* `torch`
-* `numpy`
-* `typing_extensions`
-
-## Quick Start / Usage Example
-
-Basic example:
-
-```python
-import torch
-from torch import nn
-import torchzero as tz
-
-# Define a simple model
-model = nn.Linear(10, 1)
-criterion = nn.MSELoss()
-inputs = torch.randn(5, 10)
-targets = torch.randn(5, 1)
-
-# Create an optimizer
-# The order of modules matters:
-# 1. ClipValue: clips gradients to (-10, 10) range.
-# 2. Adam: applies Adam update rule to clipped gradients.
-# 3. NormalizeByEMA: stabilizes the update by normalizing it to an exponential
-# moving average of past updates.
-# 4. WeightDecay - decoupled weight decay (can also move after LR to fully decouple)
-# 5. LR: Scales the computed update by the learning rate (supports LR schedulers).
-optimizer = tz.Modular(
-    model.parameters(),
-    tz.m.ClipValue(10),
-    tz.m.Adam(),
-    tz.m.NormalizeByEMA(max_ema_growth=1.1),
-    tz.m.WeightDecay(1e-4),
-    tz.m.LR(1e-1),
-)
-
-# Standard training loop
-for epoch in range(100):
-    optimizer.zero_grad()
-    output = model(inputs)
-    loss = criterion(output, targets)
-    loss.backward()
-    optimizer.step()
-    if (epoch+1) % 10 == 0: print(f"Epoch {epoch+1}, Loss: {loss.item()}")
-```
-
-## Overview of Available Modules
-
-`torchzero` provides a huge number of various modules:
-
-* **Optimizers**: Optimization algorithms.
-  * `Adam`.
-  * `Shampoo`.
-  * `SOAP` (my current recommendation).
-  * `Muon`.
-  * `SophiaH`.
-  * `Adagrad` and `FullMatrixAdagrad`.
-  * `Lion`.
-  * `RMSprop`.
-  * `OrthoGrad`.
-  * `Rprop`.
-
-  Additionally many other optimizers can be easily defined via modules:
-  * Grams: `[tz.m.Adam(), tz.m.GradSign()]`
-  * LaProp: `[tz.m.RMSprop(), tz.m.EMA(0.9)]`
-  * Signum: `[tz.m.HeavyBall(), tz.m.Sign()]`
-  * Full matrix version of any diagonal optimizer, like Adam: `tz.m.FullMatrixAdagrad(beta=0.999, inner=tz.m.EMA(0.9))`
-  * Cautious version of any optimizer, like SOAP: `[tz.m.SOAP(), tz.m.Cautious()]`
-
-* **Momentum**:
-  * `NAG`: Nesterov Accelerated Gradient.
-  * `HeavyBall`: Classic momentum (Polyak's momentum).
-  * `EMA`: Exponential moving average.
-  * `Averaging` (`Medianveraging`, `WeightedAveraging`): Simple, median, or weighted averaging of updates.
-  * `Cautious`, `ScaleByGradCosineSimilarity`: Momentum cautioning.
-  * `MatrixMomentum`, `AdaptiveMatrixMomentum`: Second order momentum.
-
-* **Stabilization**: Gradient stabilization techniques.
-  * `ClipNorm`: Clips gradient L2 norm.
-  * `ClipValue`: Clips gradient values element-wise.
-  * `Normalize`: Normalizes gradients to unit norm.
-  * `Centralize`: Centralizes gradients by subtracting the mean.
-  * `ClipNormByEMA`, `NormalizeByEMA`, `ClipValueByEMA`: Clipping/Normalization based on EMA of past values.
-  * `ClipNormGrowth`, `ClipValueGrowth`: Limits norm or value growth.
-
-* **Gradient approximations**: Methods for approximating gradients.
-  * `FDM`: Finite difference method.
-  * `RandomizedFDM` (`MeZO`, `SPSA`, `RDSA`, `Gaussian smoothing`): Randomized finite difference methods (also subspaces).
-  * `ForwardGradient`: Randomized gradient approximation via forward mode automatic differentiation.
-
-* **Second order**: Second order methods.
-  * `Newton`: Classic Newton's method.
-  * `NewtonCG`: Matrix-free newton's method with conjugate gradient solver.
-  * `NystromSketchAndSolve`: Nyström sketch-and-solve method.
-  * `NystromPCG`: NewtonCG with Nyström preconditioning (usually beats NewtonCG).
-  * `HigherOrderNewton`: Higher order Newton's method with trust region.
-
-* **Quasi-Newton**: Approximate second-order optimization methods.
-  * `LBFGS`: Limited-memory BFGS.
-  * `LSR1`: Limited-memory SR1.
-  * `OnlineLBFGS`: Online LBFGS.
-  * `BFGS`, `DFP`, `PSB`, `SR1`, `SSVM`, `BroydenBad`, `BroydenGood`, `ColumnUpdatingMethod`, `FletcherVMM`, `GradientCorrection`, `Greenstadt1`, `Greenstadt2`, `Horisho`, `McCormick`, `Pearson`, `ProjectedNewtonRaphson`, `ThomasOptimalMethod`: Classic full-matrix quasi-newton methods.
-  * `PolakRibiere`, `FletcherReeves`, `HestenesStiefel`, `DaiYuan`, `LiuStorey`, `ConjugateDescent`, `HagerZhang`, `HybridHS_DY`, `ProjectedGradientMethod`: Conjugate gradient methods.
-
-* **Line Search**:
-  * `Backtracking`, `AdaptiveBacktracking`: Backtracking line searches (adaptive is my own).
-  * `StrongWolfe`: Cubic interpolation line search satisfying strong Wolfe conditions.
-  * `ScipyMinimizeScalar`: Wrapper for SciPy's scalar minimization for line search.
-  * `TrustRegion`: First order trust region method.
-
-* **Learning Rate**:
-  * `LR`: Controls learning rate and adds support for LR schedulers.
-  * `PolyakStepSize`: Polyak's method.
-  * `Warmup`: Learning rate warmup.
-
-* **Projections**: This can implement things like GaLore but I haven't done that yet.
-  * `FFTProjection`, `DCTProjection`: Use any update rule in Fourier or DCT domain (doesn't seem to help though).
-  * `VectorProjection`, `TensorizeProjection`, `BlockPartition`, `TensorNormsProjection`: Structural projection methods (for block BFGS etc.).
-
-* **Smoothing**: Smoothing-based optimization methods.
-  * `LaplacianSmoothing`: Laplacian smoothing for gradients (implements Laplacian Smooth GD).
-  * `GaussianHomotopy`: Smoothing via randomized Gaussian homotopy.
-
-* **Weight Decay**:.
-  * `WeightDecay`: Standard L2 or L1 weight decay.
-
-* **Ops**: This has low level operations, also stuff like grafting and gradient accumulation.
-
-* **Wrappers**.
-  * `Wrap`: Wraps any PyTorch optimizer, allowing to use it as a module.
-
-* **Experimental**: various horrible atrocities
-
-## Advanced Usage
-
-### Closure
-
-Certain modules, particularly line searches and gradient approximations require a closure, similar to L-BFGS in PyTorch. Also some modules require closure to accept an additional `backward` argument, refer to example below:
-
-```python
-# basic training loop
-for inputs, targets in dataloader:
-
-    def closure(backward=True): # make sure it is True by default
-        preds = model(inputs)
-        loss = criterion(preds, targets)
-
-        if backward: # gradient approximations always call with backward=False.
-            optimizer.zero_grad()
-            loss.backward()
-
-        return loss
-
-    loss = optimizer.step(closure)
-```
-
-The code above will also work with any other optimizer because all PyTorch optimizers and most custom ones support closure, so there is no need to rewrite training loop.
-
-Non-batched example (rosenbrock):
-
-```py
-import torchzero as tz
-
-def rosen(x, y):
-    return (1 - x) ** 2 + 100 * (y - x ** 2) ** 2
-
-W = torch.tensor([-1.1, 2.5], requires_grad=True)
-
-def closure(backward=True):
-    loss = rosen(*W)
-    if backward:
-        W.grad = None # same as opt.zero_grad()
-        loss.backward()
-    return loss
-
-opt = tz.Modular([W], tz.m.NewtonCG(), tz.m.StrongWolfe())
-for step in range(20):
-    loss = opt.step(closure)
-    print(f'{step} - {loss}')
-```
-
-### Module combinations
-
-There are practically no rules to the ordering of the modules - anything will work. For example any method can be made zeroth order by putting it after some gradient approximation module such as GaussianSmoothing:
-
-```python
-opt = tz.Modular(
-    bench.parameters(),
-    tz.m.GaussianSmoothing(h=0.01, n_samples=10),
-    tz.m.NewtonCG(hvp_method='forward'),
-    tz.m.AdaptiveBacktracking(),
-)
-```
-
-GaussianSmoothing actually creates a new **closure** which approximates the gradient. To NewtonCG this closure is just like
-any other closure, so it works seamlessly.
-
-Any module can be projected (this is how it will work once I implement GaLore, but I haven't done that yet):
-
-```python
-tz.m.GaLore([tz.m.GraftModules(tz.m.Shampoo(), tz.m.RMSprop()), tz.m.LR(1e-2)])
-```
-
-### Low level modules
-
-torchzero provides a lot of low-level modules that can be used to recreate update rules, or combine existing update rules
-in new ways. Here are some equivalent ways to make Adam in order of their involvement:
-
-```python
-tz.m.Adam()
-```
-
-```python
-# Adam is debiased RMSprop applied to EMA
-tz.m.RMSprop(0.999, debiased=True, init='zeros', inner=tz.m.EMA(0.9))
-```
-
-```python
-tz.m.DivModules(
-    tz.m.EMA(0.9, debiased=True),
-    [tz.m.SqrtEMASquared(0.999, debiased=True), tz.m.Add(1e-8)]
-)
-```
-
-```python
-tz.m.DivModules(
-    [tz.m.EMA(0.9), tz.m.Debias(beta1=0.9, beta2=0.999)],
-    [tz.m.EMASquared(0.999), tz.m.Sqrt(), tz.m.Add(1e-8)]
-)
-```
-
-```python
-tz.m.DivModules(
-    [tz.m.EMA(0.9), tz.m.Debias(beta1=0.9)],
-    [
-        tz.m.Pow(2),
-        tz.m.EMA(0.999),
-        tz.m.AccumulateMaximum() if amsgrad else tz.m.Identity(),
-        tz.m.Sqrt(),
-        tz.m.Debias2(beta=0.999),
-        tz.m.Add(1e-8)]
-)
-```
-
-### Quick guide to implementing new modules
-
-Modules are quite similar to torch.optim.Optimizer, the main difference is that everything is stored in the Vars object,
-not in the module itself. Also both per-parameter settings and state are stored in per-parameter dictionaries. Feel free to modify the example below.
-
-```python
-import torch
-from torchzero.core import Module, Var
-
-class HeavyBall(Module):
-    def __init__(self, momentum: float = 0.9, dampening: float = 0):
-        defaults = dict(momentum=momentum, dampening=dampening)
-        super().__init__(defaults)
-
-    def step(self, var: Var):
-        # a module takes a Var object, modifies it or creates a new one, and returns it
-        # Var has a bunch of attributes, including parameters, gradients, update, closure, loss
-        # for now we are only interested in update, and we will apply the heavyball rule to it.
-
-        params = var.params
-        update = var.get_update() # list of tensors
-
-        exp_avg_list = []
-        for p, u in zip(params, update):
-            state = self.state[p]
-            settings = self.settings[p]
-            momentum = settings['momentum']
-            dampening = settings['dampening']
-
-            if 'momentum_buffer' not in state:
-                state['momentum_buffer'] = torch.zeros_like(p)
-
-            buf = state['momentum_buffer']
-            u *= 1 - dampening
-
-            buf.mul_(momentum).add_(u)
-
-            # clone because further modules might modify exp_avg in-place
-            # and it is part of self.state
-            exp_avg_list.append(buf.clone())
-
-        # set new update to var
-        var.update = exp_avg_list
-        return var
-```
-
-There are a some specialized base modules that make it much easier to implement some specific things.
-
-* `GradApproximator` for gradient approximations
-* `LineSearch` for line searches
-* `Projection` for projections like GaLore or into fourier domain.
-* `QuasiNewtonH` for full-matrix quasi-newton methods that update hessian inverse approximation (because they are all very similar)
-* `ConguateGradientBase` for conjugate gradient methods, basically the only difference is how beta is calculated.
-
-The documentation on how to actually use them is to write itself in the near future.
-
-## License
-
-This project is licensed under the MIT License
-
-## Project Links
-
-TODO (there are docs but from very old version)
-
-## Other stuff
-
-There are also wrappers providing `torch.optim.Optimizer` interface for for `scipy.optimize`, NLOpt and Nevergrad.
-
-They are in `torchzero.optim.wrappers.scipy.ScipyMinimize`, `torchzero.optim.wrappers.nlopt.NLOptOptimizer`, and `torchzero.optim.wrappers.nevergrad.NevergradOptimizer`. Make sure closure has `backward` argument as described in **Advanced Usage**.
-
-Apparently <https://github.com/avaneev/biteopt> is diabolical so I will add a wrapper for it too very soon.

torchzero-0.3.10.dist-info/RECORD

@@ -1,139 +0,0 @@
-docs/source/conf.py,sha256=jd80ZT2IdCx7nlQrpOTJL8UhGBNm6KYyXlpp0jmRiAw,1849
-tests/test_identical.py,sha256=NZ7A8Rm1U9Q16d-cG2G_wccpPtNALyoKYJt9qMownMc,11568
-tests/test_module.py,sha256=qX3rjdSJsbA8JO17bPTUIDspe7bg2dogqxMw__KV7SU,2039
-tests/test_opts.py,sha256=VSko5fUuACo_y6iab_akke0gMhCUEEUJ9ahpBqWBoM4,41715
-tests/test_tensorlist.py,sha256=SwzLKLrs2ppMtm_7UrfTDTlD-ObZd7JQ_FNHbp059tc,72460
-tests/test_utils_optimizer.py,sha256=bvC0Ehvs2L8fohpyIF5Vfr9OKTycpnODWLPflXilU1c,8414
-tests/test_vars.py,sha256=MqCJXrbj-C75APm1heykzcEWewinihlSjekkYDx-TFk,6726
-torchzero/__init__.py,sha256=L7IJ1qZ3o8E9oRwlJZBK2_2yII_eeGEk57Of6EfVbrk,112
-torchzero/core/__init__.py,sha256=Zib_4is13LFAabp_7VU8QXZpQEEZGzsH94vgRI0HxAg,150
-torchzero/core/module.py,sha256=Yfzn48dDbxYZJLpWnLYFIbqBb4sB3GekSZ7QGIplYAg,27525
-torchzero/core/transform.py,sha256=yK1wYgp03THzRN9y_f9-5q2nonEZMa0CfDFAdOxnqEU,11778
-torchzero/modules/__init__.py,sha256=8C73_dFzfWUWhii1UF86FUy8x75RPiAVLAm4sLTikBg,359
-torchzero/modules/functional.py,sha256=HXNzmPe7LsPadryEm7zrcEKqGej16QDwSgBkbEvggFM,6492
-torchzero/modules/clipping/__init__.py,sha256=ZaffMF7mIRK6hZSfuZadgjNTX6hF5ANiLBny2w3S7I8,250
-torchzero/modules/clipping/clipping.py,sha256=XKFKvzNgsvuYUmvHyulE6PkZv_aeLQjp0CgtFj0013s,12516
-torchzero/modules/clipping/ema_clipping.py,sha256=MGouZEN0BorliHAZhue0afhC3AhZJ6wrnwBRzDTHjX4,5978
-torchzero/modules/clipping/growth_clipping.py,sha256=50c1YOUPVL8eWzH6zJINnNP68oiZkDcq7rR6HnWjVFc,6674
-torchzero/modules/experimental/__init__.py,sha256=zxxNKPZHnkVnx1ZjKNX_nkV4Wc_EdODM6qJGn7Pgb3w,766
-torchzero/modules/experimental/absoap.py,sha256=-KwQXmI12hvHbMGPHM0APAxDQztlFhlSOG55KK6PvpI,9901
-torchzero/modules/experimental/adadam.py,sha256=o0KPLaF4J7L_Ty71RNgsysk6IEuC4DRE5nGQkGIP_dA,4078
-torchzero/modules/experimental/adamY.py,sha256=LZabWX_vccDaG6_UVZl9ALJ-3nCZu-NEygJQ_Bwzel8,4018
-torchzero/modules/experimental/adasoap.py,sha256=XtxEvBWYdcqfWnQqOFa_-SrOwd_nXHzLftiw-YXDACQ,7408
-torchzero/modules/experimental/curveball.py,sha256=JdgojuSYLNe9u3bmqcYrFm8brUD4kvKm9XYx78GzpKI,3257
-torchzero/modules/experimental/diagonal_higher_order_newton.py,sha256=u4-a5qJ_97XiZUDlClE2cASkBsx_NTJNPk6BWWybiqE,7158
-torchzero/modules/experimental/eigendescent.py,sha256=0cM1p4rYbrpwBNXgBEMblVyX0xBWTzojSC1EsUnXH6k,4707
-torchzero/modules/experimental/etf.py,sha256=FsLOCmQf24PPoRf5wsRUjVqk32uW9uTzaf1ERjFxRK8,5744
-torchzero/modules/experimental/gradmin.py,sha256=UixSLdca4ekYHOipEivdXfBAV-uEL9TZm5nCFXVaNco,3684
-torchzero/modules/experimental/newton_solver.py,sha256=3dZ7FG-2vGxJKkFF9P2LCs-LI_epcvZbyNtJOtw47pg,3055
-torchzero/modules/experimental/newtonnewton.py,sha256=QCGnY_CFo0i_NUB7D-6ezeNpG6wLkTD5lHBiakFIqbM,3033
-torchzero/modules/experimental/reduce_outward_lr.py,sha256=VFjcTpmLwpfhUR8u_5rbzPgHVR6K3fvti7jVy1DnsYU,1300
-torchzero/modules/experimental/soapy.py,sha256=7qsh9Y9U9oeQDwuDSVqnz71AD0nUYY3q0XN2XoMFWaw,6721
-torchzero/modules/experimental/spectral.py,sha256=SN7tToIpmna0IZ1NgObvqEbO48NnVbwqRwKi8ROsb7s,7374
-torchzero/modules/experimental/structured_newton.py,sha256=CWfVJ2LPZUuz1bMnlgOM6tlYPd2etjgLDIcyAfAG_y8,3464
-torchzero/modules/experimental/subspace_preconditioners.py,sha256=9Tl1PCN9crFUvVn6343GHoI3kv6CVnUWP1dfhwUvAFU,5130
-torchzero/modules/experimental/tada.py,sha256=84YcLhG34CbWq84L-AUj-A4uxpzdIVayaARHRm2f9b8,1564
-torchzero/modules/grad_approximation/__init__.py,sha256=DVFjf0cXuF70NA0nJ2WklpP01PQgrRZxUjUQjjQeSos,195
-torchzero/modules/grad_approximation/fdm.py,sha256=cUgy98Bz0Br4q6ViNxn6EVOZX2jE0nDXVZLUGhxpDcA,3589
-torchzero/modules/grad_approximation/forward_gradient.py,sha256=cNgx8kc8r0fWj8xdU2b85W3fenNDQZKuIsJLM3UzSig,3867
-torchzero/modules/grad_approximation/grad_approximator.py,sha256=TODFUwBgTmjfbnO6Sc833fnvYzYaqqYTEba_13s-qOI,2906
-torchzero/modules/grad_approximation/rfdm.py,sha256=VsRlf95JnG6HdlIsJANcfJjMk7c_B9a5-fH9dSTBA10,11328
-torchzero/modules/higher_order/__init__.py,sha256=W94CY8K1NFxs9TPi415UssKVKz5MV_bH9adax1uZsYM,50
-torchzero/modules/higher_order/higher_order_newton.py,sha256=BwiSlcGobam04SgWFcB1p_-TSuzu2rWgGVnmvP6Si9k,9751
-torchzero/modules/line_search/__init__.py,sha256=nkOUPLe88wE91ICEhprl2pJsvaKtbI3KzYOdT83AGsg,253
-torchzero/modules/line_search/backtracking.py,sha256=ZgeLAYqrw-6BeEGp8wmOgFoLtUKROF7w7LpAREe0xZU,7704
-torchzero/modules/line_search/line_search.py,sha256=CfOENZgAPSdyv1wvSbhw6gdpfbQnXGdOnLsq29wjvzU,7229
-torchzero/modules/line_search/scipy.py,sha256=SvDCZ1DPOLZcSeOFvf3tXAf1ty-9qRVfGFMWVF5q708,2293
-torchzero/modules/line_search/strong_wolfe.py,sha256=xOU4XFekh4TIepm9ztJTYpcGucEMPwAeb_cDK4Rp0ho,7620
-torchzero/modules/line_search/trust_region.py,sha256=xUZApOTW4uXFBk_Uq_YBktiXcoSAKdDc6O5vjTwquGw,3101
-torchzero/modules/lr/__init__.py,sha256=kh2k_tma-oTOALR6AlD5XHdTPSMgU4A04Oa0hAqrEpI,89
-torchzero/modules/lr/adaptive.py,sha256=6s06Gvu1UmoT89hrMkXvJWHkEOMNcy5mMiyxy3V9lQs,3904
-torchzero/modules/lr/lr.py,sha256=1gU2QzMA5PV2KkzOkxxrZZKGcz-Kbjyp7WNurOM36ys,2655
-torchzero/modules/momentum/__init__.py,sha256=pSD7vxu8PySrYOSHQMi3C9heYdcQr8y6WC_rwMybZm0,544
-torchzero/modules/momentum/averaging.py,sha256=NmRodxsSekEDGIuFGDYOvJL-WkdMN3YF-naBdtfjxx8,3247
-torchzero/modules/momentum/cautious.py,sha256=JuaFYfyf9S3nTcqeZz5ylXKepqi0eqglOAQ0uNG0eT8,7373
-torchzero/modules/momentum/ema.py,sha256=qJV__nIbcD9e8qvwbvsATnYkQrdnmMiA91ju52IqSxw,10699
-torchzero/modules/momentum/experimental.py,sha256=eYnP6NmBDegwX9XC_dYMJP3vquBpM1LyQc03v3vW6-8,6900
-torchzero/modules/momentum/matrix_momentum.py,sha256=LR12UugXM8ocwTB8zBYpt03oZeZU0cb0UoFR6qO34V8,6818
-torchzero/modules/momentum/momentum.py,sha256=4Pgk-3HM7Av_ILT6oXtvnM1CB1yit8AkFnYWLvnUAqs,2655
-torchzero/modules/ops/__init__.py,sha256=hxMZFSXX7xvitXkuBiYykVGX3p03Xprm_QA2CMg4eW8,1601
-torchzero/modules/ops/accumulate.py,sha256=yKNgw8ZsaVRPjuzPzLJOvALkjik0aWx30Eu91FefRoA,3741
-torchzero/modules/ops/binary.py,sha256=98jyjkJ8BPuSH-mb4g2BnFi6UzvRZRf__Pt-jnD3pNU,9690
-torchzero/modules/ops/debug.py,sha256=zueWyNVvaJmxRD8QG8m_ys9jc7zRfSr8GAuxqz5dDTI,851
-torchzero/modules/ops/misc.py,sha256=GmnKDjMXaTUjPcC5e7Jftk6k2NQ0Ivv4ceUApxMckIQ,15978
-torchzero/modules/ops/multi.py,sha256=T1aVaRr6bLWvjoj1cyxaDncROypT6rmmmji8mvBHczo,5713
-torchzero/modules/ops/reduce.py,sha256=reGvusJyCzM8VdHbWyJRYFePPBXfVP0jZeXIEKGIJGc,5668
-torchzero/modules/ops/split.py,sha256=eM4Qsz6pMNF22bk3NF2rtvyxSOt9U-EyYxMAyjvTrMQ,2265
-torchzero/modules/ops/switch.py,sha256=ddsxq4bsH86iWW6mMdcQw3c0mU1s2FA-PRZpVOia7PY,2506
-torchzero/modules/ops/unary.py,sha256=3ysDHXs6snsQNBj3c288BT8G6T30Nvo0QM3PcdfQ2ww,4888
-torchzero/modules/ops/utility.py,sha256=8XFjQO4ghCmGD2H-lYTgaBzik_9pB3Uxt7xCxQrv5Ig,3080
-torchzero/modules/optimizers/__init__.py,sha256=BbT2nhIt4p74t1cO8ziQgzqZHaLvyuleXQbccugd06M,554
-torchzero/modules/optimizers/adagrad.py,sha256=NHpWcnIRM2LyPnNtDVTdluX4n1qqqWs9IHpFD8uYeLo,5500
-torchzero/modules/optimizers/adam.py,sha256=u6ieXHn_lHZozmGiKhSA73pApI83eeTNIyOrxBTFL1o,4009
-torchzero/modules/optimizers/lion.py,sha256=4yy6d0SLpGXndu8NCuYhdsNshMEYhONu_FPYXdupA_s,1119
-torchzero/modules/optimizers/muon.py,sha256=exbp7wVpIryiOxmbf9RAfZ9a6XXuOWTUqdjn-i57Fq4,9628
-torchzero/modules/optimizers/orthograd.py,sha256=cN5g7OusfeUlh0jn0kjkvpcVjqR01eGoi9WK1sSPnug,2021
-torchzero/modules/optimizers/rmsprop.py,sha256=jM5ohfABYUljy2RrtG_bY9PMHNzSkROYjqFPxnlXE6o,4309
-torchzero/modules/optimizers/rprop.py,sha256=d0R8UR-f9Pb64VrsJegrCyteLYa5TAmgObjgirqLaBo,11030
-torchzero/modules/optimizers/shampoo.py,sha256=hmfgPghzmjmba3PH1vLzaz0lOvLiIX9rCKrT71YZb40,8420
-torchzero/modules/optimizers/soap.py,sha256=7adybqncrkt31rNveQwXp8eeZKWf0LDhC5wt7GbmDcM,11052
-torchzero/modules/optimizers/sophia_h.py,sha256=He9YrHeaQhiz4CJm-3H_d_M07MGTsP663v8wx4BlaZI,4273
-torchzero/modules/projections/__init__.py,sha256=OCxlh_-Tx-xpl31X03CeFJH9XveH563oEsWc8rUvX0A,196
-torchzero/modules/projections/dct.py,sha256=0tswjgta3mE5D5Yjw9mJWqPBDga0OIe3lKlwd1AXASc,2369
-torchzero/modules/projections/fft.py,sha256=wNDZP5-3b2-bND3qH2yvX3SqFaljbLkPTQ1gUnlH5fU,2955
-torchzero/modules/projections/galore.py,sha256=etaG2gxazxuDEu-e2r7lKIKMTPEGGS5Vi7HXccmD3kY,241
-torchzero/modules/projections/projection.py,sha256=QUV_Gi6QlPiWEmcc7rwucr2yuYwYFGvSRUAT4uucqMY,10049
-torchzero/modules/projections/structural.py,sha256=f8-72zViXJ6S2gxDagkrrul9IaOPsYXZmX8sFLYkxCc,5683
-torchzero/modules/quasi_newton/__init__.py,sha256=Yc-NV__cJCiYLr2BZG4VsYa3VVq4gCxBMcirQEXSNIo,630
-torchzero/modules/quasi_newton/cg.py,sha256=lvmwJNTR7AEcpDIvpcLnMrZrOLwNld8GFAC19CcTKoY,11661
-torchzero/modules/quasi_newton/lbfgs.py,sha256=BDiv3f7qN8-Nhs8LqtWwk7Wwv88NtXXYle5WwKeekm4,9198
-torchzero/modules/quasi_newton/lsr1.py,sha256=A0Pstikb6JrQbwM5RZjLw9WJEHiMRy3PsPF1_iLkrK4,6053
-torchzero/modules/quasi_newton/olbfgs.py,sha256=Tz2eubiN7OXGN1mbXT4VKPd9kynpXzcLas7mrvBax-k,8333
-torchzero/modules/quasi_newton/quasi_newton.py,sha256=4hRII9GFE5MzNtXkHH_T1hEJ1T8T4-Q4A4MXlhf64mc,25142
-torchzero/modules/quasi_newton/experimental/__init__.py,sha256=3qpZGgdsx6wpoafWaNWx-eamRl1FuxVCWQZq8Y7Cl98,39
-torchzero/modules/quasi_newton/experimental/modular_lbfgs.py,sha256=oLbJ96sl-2XBwLbJrrTZiLJIhKhTPOD6-wny7hbSno4,10767
-torchzero/modules/second_order/__init__.py,sha256=jolCGaIVkID9hpxgx0Tc22wgjVlwuWekWjKTMe5jKXw,114
-torchzero/modules/second_order/newton.py,sha256=ZYIcLpifcOHL_KRC6YoNs-MJQKM39urXUQzReWnWeXE,6583
-torchzero/modules/second_order/newton_cg.py,sha256=YAEAD_8YU_H8Y-o6JI0Ywgk-kpAQOFBQm2Bjzaz9Bjs,2865
-torchzero/modules/second_order/nystrom.py,sha256=aM6dlDv7znGYNXZgKN6B6AhZ1Tpp01JMs83B1hcXE3w,6061
-torchzero/modules/smoothing/__init__.py,sha256=tUTGN0A-EQC7xuLV2AuHFWk-t7D6jIJlpV_3qyfRqLk,80
-torchzero/modules/smoothing/gaussian.py,sha256=KbCgRXGntdPbt4-ojalrHkniYgYXk2294b-2C4MIFi8,6109
-torchzero/modules/smoothing/laplacian.py,sha256=Vp2EnCQhyfGc3CbyOLc6_ZiVx_jvnOISf9vlHkIH4Jo,4998
-torchzero/modules/weight_decay/__init__.py,sha256=j2Vq3DDxLYIPJmXWgAJ6dL-rXzcDEZxxvhJqRT3H0-U,95
-torchzero/modules/weight_decay/weight_decay.py,sha256=UFL9W5w5nzTZGWvCwyGLe9UWBKN8FTClme1Klt7XZPw,3034
-torchzero/modules/wrappers/__init__.py,sha256=6b5Ac-8u18IVp_Jnw1T1xQExwpQhpQ0JwNV9GyC_Yj8,31
-torchzero/modules/wrappers/optim_wrapper.py,sha256=-wNI-fN8eaMSkvPIcPa34yxH0St5aLn7jaaLeh2DUsM,3569
-torchzero/optim/__init__.py,sha256=aXf7EkywqYiR50I4QeeVXro9aBhKiqfbY_BCia59sgU,46
-torchzero/optim/utility/__init__.py,sha256=pUacok4XmebfxofE-QWZLgViajsU-3JkXcWi9OS-Jrw,24
-torchzero/optim/utility/split.py,sha256=ZbazNuMTYunm75V_5ard0A_LletGaYAg-Pm2rANJKrE,1610
-torchzero/optim/wrappers/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-torchzero/optim/wrappers/directsearch.py,sha256=Y2-7Sy4mYRPXPh0FTlsY_XOk5pCGjZsnbrlWCPZNp6A,10141
-torchzero/optim/wrappers/fcmaes.py,sha256=TQvIktXV8ldy6smBX-S7ZcQEbSmSZyj567TuYShbvJg,3513
-torchzero/optim/wrappers/mads.py,sha256=lC7edtrFS37PgmX7z9-eoqw6prl0k5BDB4NVBVQXJWE,2945
-torchzero/optim/wrappers/nevergrad.py,sha256=qslMb-4_kfjU3Dd0UbbzE2SdLViil3Qjo2v0FtPE3Fg,4000
-torchzero/optim/wrappers/nlopt.py,sha256=AaVEKfjbrt5DFION44_-g-jQAoVi4lCvBBPU5UDGO9Q,8151
-torchzero/optim/wrappers/optuna.py,sha256=YN1I3rzsi20A9963pWNWd7W75FkxalVb5z5fCRQeWA0,2280
-torchzero/optim/wrappers/scipy.py,sha256=pR26v8v0a-o2u0sbsKXpZ9JUKqXMaaI8gGLI8xYx3-s,19239
-torchzero/utils/__init__.py,sha256=7beAjXvnmBQoy5hwYHY_PBUtrrbYb9Z7-KrYgfcFkPE,844
-torchzero/utils/compile.py,sha256=N8AWLv_7oBUHYornmvvx_L4uynjiD-x5Hj1tBwei3-w,5127
-torchzero/utils/derivatives.py,sha256=sAVd0Q1xmIPpo_AxRuoow66Hy_3goX_9o3lQK_1TyW0,16909
-torchzero/utils/numberlist.py,sha256=cbG0UsSb9WCRxVhw8sd7Yf0bDy_gSqtghiJtkUxIO6U,6139
-torchzero/utils/ops.py,sha256=n4Su1sbgTzlHczuPEHkuWenTtNBCa_MvlQ_hCZkIPnQ,314
-torchzero/utils/optimizer.py,sha256=r52qu6pEcRH4lCXVlLxW5IweA6L-VrQj6RCMfdhzRpw,12466
-torchzero/utils/optuna_tools.py,sha256=F-1Xg0n_29MVEb6lqgUFFNIl9BNJ6MOdIJPduoNH4JU,1325
-torchzero/utils/params.py,sha256=nQo270aOURU7rJ_D102y2pSXbzhJPK0Z_ehx4mZBMes,5784
-torchzero/utils/python_tools.py,sha256=T5W7MpR7pNXiWSVw7gj-UuE9Ch0p9LRWuUZfg9Vtb-I,2794
-torchzero/utils/tensorlist.py,sha256=qSbiliVo1euFAksdHHHRbPUdYYxfkw1dvhpXj71wGy0,53162
-torchzero/utils/torch_tools.py,sha256=ohqnnZRlqdfp5PAfMSbQDIEKygW0_ARjxSEBp3Zo9nU,4756
-torchzero/utils/linalg/__init__.py,sha256=Dzbho3_z7JDdKzYD-QdLArg0ZEoC2BVGdlE3JoAnXHQ,272
-torchzero/utils/linalg/benchmark.py,sha256=wiIMn-GY2xxWbHVf8CPbJddUPeUPq9OUDkvbp1iILYI,479
-torchzero/utils/linalg/matrix_funcs.py,sha256=-LecWrPWbJvfeCgIzUhfWARa2aSZvJ12lHX7Jno38O4,3099
-torchzero/utils/linalg/orthogonalize.py,sha256=mDCkET7qgDZqf_y6oPYAK3d2L5HrB8gzOFPl0YoONaY,399
-torchzero/utils/linalg/qr.py,sha256=L-RXuYV-SIHI-Llq4y1rQ_Tz-yamds0_QNZeHapbjNE,2507
-torchzero/utils/linalg/solve.py,sha256=P0PMi0zro3G3Rd0X-JeoLk7tqYDB0js0aB4bpQ0OABU,5235
-torchzero/utils/linalg/svd.py,sha256=wBxl-JSciINV-N6zvM4SGdveqMr6idq51h68LyQQRYg,660
-torchzero-0.3.10.dist-info/licenses/LICENSE,sha256=r9ZciAoZoqKC_FNADE0ORukj1p1XhLXEbegdsAyqhJs,1087
-torchzero-0.3.10.dist-info/METADATA,sha256=_J7AbrIa-nD6UWbuydCwxAnSpKcC9O1Vp_rM896ZkYQ,14081
-torchzero-0.3.10.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-torchzero-0.3.10.dist-info/top_level.txt,sha256=YDdpIOb7HyKV9THOtOYsFFMTbxvCO0kiol4-83tDj-A,21
-torchzero-0.3.10.dist-info/RECORD,,

torchzero-0.3.10.dist-info/licenses/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2024 inikishev
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.