lycoris-lora 3.0.0.dev14__tar.gz → 3.0.0.dev16__tar.gz

lycoris_lora-3.0.0.dev16/PKG-INFO

@@ -0,0 +1,25 @@
+Metadata-Version: 2.1
+Name: lycoris_lora
+Version: 3.0.0.dev16
+Summary: Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion
+Home-page: https://github.com/KohakuBlueleaf/LyCORIS
+Author: Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao
+Author-email: apolloyeh0123@gmail.com
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: GPU :: NVIDIA CUDA :: 11.8
+Classifier: Environment :: GPU :: NVIDIA CUDA :: 12
+Classifier: Environment :: GPU :: NVIDIA CUDA :: 12 :: 12.1
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+License-File: LICENSE.md
+Requires-Dist: torch
+Requires-Dist: einops
+Requires-Dist: toml
+Requires-Dist: tqdm

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/README.md

@@ -1,6 +1,10 @@
+
+![pypi](https://img.shields.io/pypi/v/lycoris-lora.svg)
+![versions](https://img.shields.io/pypi/pyversions/lycoris-lora.svg)
+
 # LyCORIS - Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion. (ICLR'24)
 
-![banner image](docs/images/banner.png)
+![banner image](docs/images/banner2.png)
 
 A project that implements different parameter-efficient fine-tuning algorithms for Stable Diffusion.
 
@@ -20,21 +24,21 @@ Please check [List of Implemented Algorithms](docs/Algo-List.md) and [Guidelines
 
 A simple comparison of some of these methods are provided below (to be taken with a grain of salt)
 
-|                       | Full | LoRA | LoHa | LoKr low factor | LoKr high factor $^+$ |
-| --------------------- | ---- | ---- | ---- | --------------- | ---------------- |
-| Fidelity              | ★   | ●   | ▲   | ◉              | ▲               |
-| Flexibility $^*$     | ★   | ●   | ◉   | ▲              | ● $^†$        |
-| Diversity             | ▲   | ◉   | ★   | ●              | ★               |
-| Size                  | ▲   | ●   | ●   | ●              | ★               |
-| Training Speed Linear | ★   | ●   | ●   | ★              | ★               |
-| Training Speed Conv   | ●   | ★   | ▲   | ●              | ●               |
+|                       | Full | LoRA | LoHa | LoKr low factor | LoKr high factor$^+$ |
+| --------------------- | ---- | ---- | ---- | --------------- | ---------------------- |
+| Fidelity              | ★   | ●   | ▲   | ◉              | ▲                     |
+| Flexibility$^*$     | ★   | ●   | ◉   | ▲              | ●$^†$              |
+| Diversity             | ▲   | ◉   | ★   | ●              | ★                     |
+| Size                  | ▲   | ●   | ●   | ●              | ★                     |
+| Training Speed Linear | ★   | ●   | ●   | ★              | ★                     |
+| Training Speed Conv   | ●   | ★   | ▲   | ●              | ●                     |
 
 ★ > ◉ > ● > ▲
 [> means better and smaller size is better]
 
-$^+$ Usually we take `factor <= 0.5 * sqrt(dim)` as low factor and `factor >= sqrt(dim` as high factor. For example, factor<=8 for SD1.x/SD2.x/SDXL can be seen as low factor, and, factor>=16 can be seen as high factor. <br>
-$^*$ Flexibility means anything related to generating images not similar to those in the training set, and combination of multiple concepts, whether they are trained together or not <br>
-$^†$ It may become more difficult to switch base model or combine multiple concepts in this situation <br>
+$^+$ Usually we take `factor <= 0.5 * sqrt(dim)` as low factor and `factor >= sqrt(dim` as high factor. For example, factor<=8 for SD1.x/SD2.x/SDXL can be seen as low factor, and, factor>=16 can be seen as high factor. `<br>`
+$^*$ Flexibility means anything related to generating images not similar to those in the training set, and combination of multiple concepts, whether they are trained together or not `<br>`
+$^†$ It may become more difficult to switch base model or combine multiple concepts in this situation `<br>`
 
 **The actual performance may vary depending on the datasets, tasks, and hyperparameters used. It is recommended to experiment with different settings to achieve optimal results.**
 
@@ -66,7 +70,7 @@ However, newer model types may not always be supported. If you encounter this is
 There are three different ways to train LyCORIS models.
 
 - With [kohya-ss/sd-scripts](https://github.com/kohya-ss/sd-scripts) (see a list of compatible graphical interfaces and colabs at the end of the section)
-- With [HCP-Diffusion](https://github.com/IrisRainbowNeko/HCP-Diffusion)
+- With [Naifu-Diffusion](https://github.com/Mikubill/naifu)
 - With your own script by using LyCORIS as standalone wrappers for **ANY** pytorch modules.
 
 In any case, please install this package in the corresponding virtual environment. You can either install it
@@ -159,9 +163,11 @@ forward_with_lyco = your_model(x)
 You can check my [HakuPhi](https://github.com/KohakuBlueleaf/HakuPhi) project to see how I utilize LyCORIS to finetune the Phi-1.5 models.
 
 #### Other method
+
 After LyCORIS3.0.0, Parametrize API and Functional API have been added, which provide more different ways on utilizing LyCORIS library.
 
-Check [xxx] for more informations.
+Check API reference for more informations.
+You can also take the [test suites](test/) as a kind of examples.
 
 #### Bitsandbytes support
 
@@ -250,44 +256,61 @@ See [docs/Conversion-scripts.md](docs/Conversion-scripts.md) for more informatio
 
 For full log, please see [Change.md](Change.md)
 
-## 2024/03/15 update to 2.2.0 - QLyCORIS and DoRA
+### 2024/06/xx update to 3.0.0 - Brand New Functional API, Parametrize API and Module API
+
+#### The reasons of 3.0.0
+
+We reconstruct the whole library with new Class definition and brand new Functional API system.
 
-#### New Algo
+We also removed lot of redundant/unused modules.
 
-* DoRA
-  * Ref: [DoRA: Weight-Decomposed Low-Rank Adaptation](https://github.com/KohakuBlueleaf/LyCORIS)
-* Weight decompose for LoHa and LoKr. (A.K.A DoHa/DoKr)
-  * DoRA/DoHa/DoKr will require smaller Learning rate!
+Since the whole library are changed significantly. We decide to call it 3.0.0 as a new major version.
 
-#### New Features
+#### Major Changes
 
-* Support "bypass" (a.k.a. adapter) mode for LoHa/LoKr/OFT/BOFT
-  * LoHa will require 2xFLOPs since we rebuild full diff weight and then do one more forward.
-  * LoKr, OFT, BOFT should be more efficient than LoHa in bypass mode.
-* Support [bnb 8bit/4bit Linear layer](https://github.com/TimDettmers/bitsandbytes) (a.k.a. QLyCORIS) with LoHa/LoKr/OFT/BOFT.
-  * This will force module to enable bypass mode.
+* New Module API
+* Add Parametrize API
+* Add Functional API
+  * LoCon/LoHa/LoKr/Diag-OFT/BOFT only.
+* Remove optional deps from install_requires
+* Remove lot of redundant/deprecated modules
+* Better testing
+* HunYuan DiT Support ([PR](https://github.com/kohya-ss/sd-scripts/pull/1378) in kohya-ss/sd-scripts)
 
-#### Fixes, slight changes
+#### Full change log
 
-* Refine some details about code quality. Based on the report from GitRoll. (Thx you gitroll!)
-* Remove redundant calculation in BOFT
-* rank_dropout has been removed from OFT/BOFT temporarily untill we ensure how to apply it.
-* Fix bugs in lokr when `lokr_w1_a` not exist.
-* Fix bugs in conversion scritps.
+##### New Features
+
+* LyCORIS now have consistent API for different algorithm like `bypass_forward_diff` or `get_diff_weight` method. Developers of other project can utilize these API to do more tricks or integrate LyCORIS into their framework more easily.
+* LyCORIS now have parametrize API which utilize `torch.nn.utils.parametrize.register_parametrization` to directly patch individual parameters. Which can be useful for MHA layer or other tricky modules.
+  * Currently only support 2~5D tensors. And LyCORIS will pretend these weights are weight of Linear/Conv1,2,3D then send it into LyCORIS modules
+  * More native implementation or more detailed control will be added in the future.
+* LyCORIS now have functional API. Developers who prefer functional more than Module things can utilize this feature.
+  * Functional API also allow developers who don't want to introduce new dependencies. Just copy-paste the source code and utilizing it. (with Apache-2 License, directly copy-paste is totally allowed)
+* Add support for Conv1d and Conv3d module on LoCon/LoHa/LoKr/Full/OFT/BOFT/GLoRA (not All algo in LyCORIS support them, you may receive error when apply unsopported algo), support inherited module (for example: `LoRACompatibleConv` or `LoRACompatibleLinear` from [`huggingface/diffusers`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/lora.py))
+* HunYuan DiT support.
+
+##### Improvements, Fixes, Slight Changes
+
+* Drop dependencies related to kohya-ss/sd-scripts:
+  * We now take kohya-ss/sd-scripts as optional dependency
+  * Which means `transformers`, `diffusers` and anything related to kohya are all optional deps now.
+* The definition of dropout and rank_dropout in each algorithm are changed. Since some concept of original rank_dropout in the lora of kohya-ss/sd-script is hard to applied to other algorithm. We can only design the dropout for each module seperatedly.
+* `apply_max_norm` issue are all fixed.
+* DyLoRA, (IA)^3, GLoRA are all rewritten and support Linear/Conv1,2,3d.
+* (IA)^3, GLoRA, Diag-OFT, BOFT are supported in `create_lycoris_from_weights`
+  * `lycoris.kohya.create_network_from_weights` also support them as well.
+* Fix wrong implementation of BOFT.
+* `create_lycoris_from_weights` and `create_network_from_weights` now have correct logging infos.
+* `get_module` and `make_module` are moved into modules' API.
 
 ## Todo list
 
-- [X] Module and Document for using LyCORIS in any other model, Not only SD.
-- [X] Proposition3 in [FedPara](https://arxiv.org/abs/2108.06098)
-  * also need custom backward to save the vram
-- [ ] Low rank + sparse representation
-  - [X] For extraction
-  - [ ] For training
-- [ ] Support more operation, not only linear and conv2d.
-- [X] Configure varying ranks or dimensions for specific modules as needed.
 - [ ] Automatically selecting an algorithm based on the specific rank requirement.
-- [ ] Explore other low-rank representations or parameter-efficient methods to fine-tune either the entire model or specific parts of it.
 - [ ] More experiments for different task, not only diffusion models.
+  - [X] LoKr and LoHa have been proven to be useful for Large Language Model.
+- [ ] Explore other low-rank representations or parameter-efficient methods to fine-tune either the entire model or specific parts of it.
+- [ ] Documentation for whole library.
 
 ## Citation
 

lycoris_lora-3.0.0.dev16/lycoris/functional/__init__.py

@@ -0,0 +1,9 @@
+from .general import (
+    rebuild_tucker,
+    factorization,
+    power2factorization,
+    FUNC_LIST,
+    tucker_weight,
+    tucker_weight_from_conv,
+    apply_dora_scale,
+)

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/lycoris/functional/general.py

@@ -77,3 +77,30 @@ def power2factorization(dimension: int, factor: int = -1) -> tuple[int, int]:
             n = dimension // m
 
     return dimension // n, n
+
+
+def tucker_weight_from_conv(up, down, mid):
+    up = up.reshape(up.size(0), up.size(1))
+    down = down.reshape(down.size(0), down.size(1))
+    return torch.einsum("m n ..., i m, n j -> i j ...", mid, up, down)
+
+
+def tucker_weight(wa, wb, t):
+    temp = torch.einsum("i j ..., j r -> i r ...", t, wb)
+    return torch.einsum("i j ..., i r -> r j ...", temp, wa)
+
+
+def apply_dora_scale(org_weight, rebuild, dora_scale, scale):
+    dora_norm_dims = org_weight.dim() - 1
+    weight = org_weight + rebuild
+    weight = weight.to(dora_scale.dtype)
+    weight_norm = (
+        weight.transpose(0, 1)
+        .reshape(weight.shape[1], -1)
+        .norm(dim=1, keepdim=True)
+        .reshape(weight.shape[1], *[1] * dora_norm_dims)
+        .transpose(0, 1)
+    )
+    merged_scale1 = weight / weight_norm * dora_scale
+    diff_weight = merged_scale1 - org_weight
+    return org_weight + diff_weight * scale

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/lycoris/modules/base.py

@@ -269,8 +269,9 @@ class LycorisBaseModule(ModuleCustomSD):
         weight, bias = self.get_merged_weight(
             multiplier, self.org_weight.shape, self.org_weight.device
         )
-        self.org_weight = weight
+        self.org_weight = weight.to(self.org_weight)
         if bias is not None:
+            bias = bias.to(self.org_weight)
             if self.org_module[0].bias is not None:
                 self.org_module[0].bias.data.copy_(bias)
             else:

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/lycoris/modules/boft.py

@@ -197,7 +197,7 @@ class ButterflyOFTModule(LycorisBaseModule):
         r = self.get_r()
         inp = org = self.org_forward(x)
         if self.op in {F.conv2d, F.conv1d, F.conv3d}:
-            org_out = org_out.transpose(1, -1)
+            inp = inp.transpose(1, -1)
 
         for i in range(m):
             bi = r[i]  # b_num, b_size, b_size

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/lycoris/modules/diag_oft.py

@@ -188,7 +188,7 @@ class DiagOFTModule(LycorisBaseModule):
         if diff:
             out = out - org_out
         out = oft_out.view(*shape, -1)
-        if self.rescale is not None:
+        if self.rescaled:
             out = self.rescale.transpose(-1, 0) * out
             out = out + (self.rescale.transpose(-1, 0) - 1) * org_out
         if self.op in {F.conv2d, F.conv1d, F.conv3d}:

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/lycoris/modules/glora.py

@@ -5,7 +5,7 @@ import torch.nn as nn
 import torch.nn.functional as F
 
 from .base import LycorisBaseModule
-from ..utils import tucker_weight_from_conv
+from ..functional import tucker_weight_from_conv
 from ..utils.bnb import LinearNF4
 
 

{lycoris_lora-3.0.0.dev14 → lycoris_lora-3.0.0.dev16}/lycoris/utils/__init__.py

@@ -417,138 +417,10 @@ def convert_diffusers_name_to_compvis(key):
     return key
 
 
-def tucker_weight_from_conv(up, down, mid):
-    up = up.reshape(up.size(0), up.size(1))
-    down = down.reshape(down.size(0), down.size(1))
-    return torch.einsum("m n ..., i m, n j -> i j ...", mid, up, down)
-
-
-def tucker_weight(wa, wb, t):
-    temp = torch.einsum("i j ..., j r -> i r ...", t, wb)
-    return torch.einsum("i j ..., i r -> r j ...", temp, wa)
-
-
-def apply_dora_scale(org_weight, rebuild, dora_scale, scale):
-    dora_norm_dims = org_weight.dim() - 1
-    weight = org_weight + rebuild
-    weight = weight.to(dora_scale.dtype)
-    weight_norm = (
-        weight.transpose(0, 1)
-        .reshape(weight.shape[1], -1)
-        .norm(dim=1, keepdim=True)
-        .reshape(weight.shape[1], *[1] * dora_norm_dims)
-        .transpose(0, 1)
-    )
-    merged_scale1 = weight / weight_norm * dora_scale
-    diff_weight = merged_scale1 - org_weight
-    return org_weight + diff_weight * scale
-
-
-@torch.no_grad()
-def rebuild_weight(module_type, params, orig_weight, orig_bias, scale=1):
-    if orig_weight is None:
-        return None, None
-    merged = orig_weight
-    merged_bias = orig_bias
-    if module_type == "locon":
-        up, down, mid, alpha, dora_scale = params
-        if alpha is not None:
-            scale *= alpha / up.size(1)
-        if mid is not None:
-            rebuild = tucker_weight_from_conv(up, down, mid)
-        else:
-            rebuild = up.reshape(up.size(0), -1) @ down.reshape(down.size(0), -1)
-        rebuild = rebuild.reshape(orig_weight.shape)
-        if dora_scale is None:
-            merged = orig_weight + rebuild * scale
-        else:
-            merged = apply_dora_scale(orig_weight, rebuild, dora_scale, scale)
-        del up, down, mid, alpha, params, rebuild
-    elif module_type == "hada":
-        w1a, w1b, w2a, w2b, t1, t2, alpha, dora_scale = params
-        if alpha is not None:
-            scale *= alpha / w1b.size(0)
-        if t1 is not None:
-            rebuild1 = tucker_weight(w1a, w1b, t1)
-        else:
-            rebuild1 = w1a @ w1b
-        if t2 is not None:
-            rebuild2 = tucker_weight(w2a, w2b, t2)
-        else:
-            rebuild2 = w2a @ w2b
-        rebuild = (rebuild1 * rebuild2).reshape(orig_weight.shape)
-        if dora_scale is None:
-            merged = orig_weight + rebuild * scale
-        else:
-            merged = apply_dora_scale(orig_weight, rebuild, dora_scale, scale)
-        del w1a, w1b, w2a, w2b, t1, t2, alpha, params, rebuild, rebuild1, rebuild2
-    elif module_type == "ia3":
-        weight, on_input = params
-        if not on_input:
-            weight = weight.reshape(-1, 1)
-        merged = orig_weight + weight * orig_weight * scale
-        del weight, on_input, params
-    elif module_type == "kron":
-        w1, w1a, w1b, w2, w2a, w2b, t1, t2, alpha, dora_scale = params
-        if alpha is not None and (w1b is not None or w2b is not None):
-            scale *= alpha / (w1b.size(0) if w1b else w2b.size(0))
-        if w1a is not None and w1b is not None:
-            if t1 is not None:
-                w1 = tucker_weight(w1a, w1b, t1)
-            else:
-                w1 = w1a @ w1b
-        if w2a is not None and w2b is not None:
-            if t2 is not None:
-                w2 = tucker_weight(w2a, w2b, t2)
-            else:
-                w2 = w2a @ w2b
-        if len(w2.shape) == 4:
-            w1 = w1.unsqueeze(2).unsqueeze(2)
-        w2 = w2.contiguous()
-        rebuild = torch.kron(w1, w2).reshape(orig_weight.shape)
-        if dora_scale is None:
-            merged = orig_weight + rebuild * scale
-        else:
-            merged = apply_dora_scale(orig_weight, rebuild, dora_scale, scale)
-        del w1, w1a, w1b, w2, w2a, w2b, t1, t2, alpha, params, rebuild
-    elif module_type == "full":
-        rebuild = params[0].reshape(orig_weight.shape)
-        rebuild_b = (
-            params[1].reshape(orig_bias.shape) if orig_bias is not None else None
-        )
-        merged = orig_weight + rebuild * scale
-        if rebuild_b is not None:
-            merged_bias = orig_bias + rebuild_b * scale
-        del params, rebuild, rebuild_b
-    elif module_type == "norm":
-        rebuild = params[0].reshape(orig_weight.shape)
-        rebuild_b = params[1].reshape(orig_bias.shape)
-        merged = orig_weight + rebuild * scale
-        merged_bias = orig_bias + rebuild_b * scale
-    else:
-        return None, None
-
-    return merged, merged_bias
-
-
 @torch.no_grad()
 def merge(tes, unet, lyco_state_dict, scale: float = 1.0, device="cpu"):
-    UNET_TARGET_REPLACE_MODULE = [
-        "Linear",
-        "Conv2d",
-        "LayerNorm",
-        "GroupNorm",
-        "GroupNorm32",
-    ]
-    TEXT_ENCODER_TARGET_REPLACE_MODULE = [
-        "Embedding",
-        "Linear",
-        "Conv2d",
-        "LayerNorm",
-        "GroupNorm",
-        "GroupNorm32",
-        "Embedding",
-    ]
+    from ..modules import make_module, get_module
+
     LORA_PREFIX_UNET = "lora_unet"
     LORA_PREFIX_TEXT_ENCODER = "lora_te"
     merged = 0
@@ -557,29 +429,25 @@ def merge(tes, unet, lyco_state_dict, scale: float = 1.0, device="cpu"):
         prefix,
         root_module: torch.nn.Module,
         lyco_state_dict: Dict[str, torch.Tensor],
-        target_replace_modules,
     ):
         nonlocal merged
         for child_name, child_module in tqdm(
             list(root_module.named_modules()), desc=f"Merging {prefix}"
         ):
-            if child_module.__class__.__name__ in target_replace_modules:
-                lora_name = prefix + "." + child_name
-                lora_name = lora_name.replace(".", "_")
-
-                result, result_b = rebuild_weight(
-                    *get_module(lyco_state_dict, lora_name),
-                    getattr(child_module, "weight"),
-                    getattr(child_module, "bias", None),
-                    scale,
-                )
-                if result is not None:
-                    key_dict.pop(lora_name)
-                    merged += 1
-                    child_module.requires_grad_(False)
-                    child_module.weight.copy_(result)
-                if result_b is not None:
-                    child_module.bias.copy_(result_b)
+            lora_name = prefix + "." + child_name
+            lora_name = lora_name.replace(".", "_")
+
+            lyco_type, params = get_module(lyco_state_dict, lora_name)
+            if lyco_type is None:
+                continue
+            module = make_module(lyco_type, params, lora_name, child_module)
+            if module is None:
+                continue
+            module.to(device)
+            module.merge_to(scale)
+            key_dict.pop(convert_diffusers_name_to_compvis(lora_name), None)
+            key_dict.pop(lora_name, None)
+            merged += 1
 
     key_dict = {}
     for k, v in tqdm(list(lyco_state_dict.items()), desc="Converting Dtype and Device"):
@@ -592,12 +460,7 @@ def merge(tes, unet, lyco_state_dict, scale: float = 1.0, device="cpu"):
             k = f"{convert_key}.{weight_key}"
         else:
             key_dict[module] = key_dict.get(module, []) + [k]
-        if device == "cpu":
-            lyco_state_dict[k] = v.float().cpu()
-        else:
-            lyco_state_dict[k] = v.to(
-                device, dtype=tes[0].parameters().__next__().dtype
-            )
+        lyco_state_dict[k] = v.float().cpu()
 
     for idx, te in enumerate(tes):
         if len(tes) > 1:
@@ -606,15 +469,15 @@ def merge(tes, unet, lyco_state_dict, scale: float = 1.0, device="cpu"):
             prefix = LORA_PREFIX_TEXT_ENCODER
         merge_state_dict(
             prefix,
-            te.to(device),
+            te,
             lyco_state_dict,
-            TEXT_ENCODER_TARGET_REPLACE_MODULE,
         )
+        torch.cuda.empty_cache()
     merge_state_dict(
         LORA_PREFIX_UNET,
-        unet.to(device),
+        unet,
         lyco_state_dict,
-        UNET_TARGET_REPLACE_MODULE,
     )
+    torch.cuda.empty_cache()
     print(f"Unused state dict key: {key_dict}")
     print(f"{merged} Modules been merged")

lycoris_lora-3.0.0.dev16/lycoris_lora.egg-info/PKG-INFO

@@ -0,0 +1,25 @@
+Metadata-Version: 2.1
+Name: lycoris_lora
+Version: 3.0.0.dev16
+Summary: Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion
+Home-page: https://github.com/KohakuBlueleaf/LyCORIS
+Author: Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao
+Author-email: apolloyeh0123@gmail.com
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: GPU :: NVIDIA CUDA :: 11.8
+Classifier: Environment :: GPU :: NVIDIA CUDA :: 12
+Classifier: Environment :: GPU :: NVIDIA CUDA :: 12 :: 12.1
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+License-File: LICENSE.md
+Requires-Dist: torch
+Requires-Dist: einops
+Requires-Dist: toml
+Requires-Dist: tqdm

lycoris_lora-3.0.0.dev16/setup.py

@@ -0,0 +1,29 @@
+from setuptools import setup, find_packages
+
+
+setup(
+    name="lycoris_lora",
+    packages=find_packages(),
+    version="3.0.0.dev16",
+    url="https://github.com/KohakuBlueleaf/LyCORIS",
+    description="Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion",
+    author="Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao",
+    author_email="apolloyeh0123@gmail.com",
+    zip_safe=False,
+    install_requires=["torch", "einops", "toml", "tqdm"],
+    python_requires=">=3.10",
+    classifiers=[
+        'Development Status :: 4 - Beta',
+        'Environment :: GPU :: NVIDIA CUDA :: 11.8',
+        'Environment :: GPU :: NVIDIA CUDA :: 12',
+        'Environment :: GPU :: NVIDIA CUDA :: 12 :: 12.1',
+        'Intended Audience :: Developers',
+        'Intended Audience :: End Users/Desktop',
+        'License :: OSI Approved :: Apache Software License',
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 3.10',
+        'Programming Language :: Python :: 3.11',
+        'Programming Language :: Python :: 3.12',
+        'Operating System :: OS Independent',
+    ],
+)

lycoris_lora-3.0.0.dev14/PKG-INFO

@@ -1,13 +0,0 @@
-Metadata-Version: 2.1
-Name: lycoris_lora
-Version: 3.0.0.dev14
-Summary: Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion
-Home-page: https://github.com/KohakuBlueleaf/LyCORIS
-Author: Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao
-Author-email: apolloyeh0123@gmail.com
-Requires-Python: >=3.10
-License-File: LICENSE.md
-Requires-Dist: torch
-Requires-Dist: einops
-Requires-Dist: toml
-Requires-Dist: tqdm

lycoris_lora-3.0.0.dev14/lycoris/functional/__init__.py

@@ -1 +0,0 @@
-from .general import rebuild_tucker, factorization, power2factorization, FUNC_LIST

lycoris_lora-3.0.0.dev14/lycoris_lora.egg-info/PKG-INFO

@@ -1,13 +0,0 @@
-Metadata-Version: 2.1
-Name: lycoris_lora
-Version: 3.0.0.dev14
-Summary: Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion
-Home-page: https://github.com/KohakuBlueleaf/LyCORIS
-Author: Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao
-Author-email: apolloyeh0123@gmail.com
-Requires-Python: >=3.10
-License-File: LICENSE.md
-Requires-Dist: torch
-Requires-Dist: einops
-Requires-Dist: toml
-Requires-Dist: tqdm

lycoris_lora-3.0.0.dev14/setup.py

@@ -1,15 +0,0 @@
-from setuptools import setup, find_packages
-
-
-setup(
-    name="lycoris_lora",
-    packages=find_packages(),
-    version="3.0.0.dev14",
-    url="https://github.com/KohakuBlueleaf/LyCORIS",
-    description="Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion",
-    author="Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao",
-    author_email="apolloyeh0123@gmail.com",
-    zip_safe=False,
-    install_requires=["torch", "einops", "toml", "tqdm"],
-    python_requires=">=3.10",
-)