lycoris-lora 0.1.5.dev2__tar.gz → 0.1.5.dev4__tar.gz

{lycoris_lora-0.1.5.dev2 → lycoris_lora-0.1.5.dev4}/Algo.md

@@ -104,7 +104,7 @@ $W' = W_1 \otimes W_2$ and an ac x bd matrix.
 
 In meaning of matrix, $W_2$ becomes weight and $W_1$ becomes weight scale of $W_2$
 
-## About rank
+### About rank
 
 And we can decompose $W_2$ using LoRA with rank, r.
 

{lycoris_lora-0.1.5.dev2 → lycoris_lora-0.1.5.dev4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lycoris_lora
-Version: 0.1.5.dev2
+Version: 0.1.5.dev4
 Summary: Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion
 Home-page: https://github.com/KohakuBlueleaf/LyCORIS
 Author: Shih-Ying Yeh(KohakuBlueLeaf), Yu-Guan Hsieh, Zhidong Gao

{lycoris_lora-0.1.5.dev2 → lycoris_lora-0.1.5.dev4}/README.md

@@ -39,6 +39,13 @@ See [Algo.md](https://github.com/KohakuBlueleaf/LyCORIS/blob/main/Algo.md) or [D
 * This algo produce very tiny file(about 200~300KB)
 * **Experimental**
 
+### LoKR
+* Basically same idea of LoHA, but use kronecker product
+* This algo is quite sensitive and you may need to tune the lr
+* This algo can learn both character and style, but since it is small (auto factor, full rank, 2.5MB), it is also hard to transfer.
+* This algo produce relatively small file(auto factor: 900~2500KB)
+* Use smaller factor will produce bigger file, you can tune it if you think 2.5MB full rank is not good enough.
+
 ---
 
 ## usage
@@ -78,22 +85,23 @@ python3 sd-scripts/train_network.py \
   --network_module lycoris.kohya \
   --network_dim "DIM_FOR_LINEAR" --network_alpha "ALPHA_FOR_LINEAR"\
   --network_args "conv_dim=DIM_FOR_CONV" "conv_alpha=ALPHA_FOR_CONV" \
-  "dropout=DROPOUT_RATE" "algo=lora" \
+  "dropout=DROPOUT_RATE" "algo=locon" \
 ```
 to train lycoris module for SD model
 
 * algo list:
-  * lora: Conventional Methods
+  * locon: Conventional Methods
   * loha: Hadamard product representation introduced by FedPara
+  * lokr: Kronecker product representation
+  * ia3 : (IA)^3
 
 * Tips:
   * Use network_dim=0 or conv_dim=0 to disable linear/conv layer
-  * LoHa doesn't support dropout yet.
+  * LoHa/LoKr/(IA)^3 doesn't support dropout yet.
 
 
 ### For a1111's sd-webui
-download [Extension](https://github.com/KohakuBlueleaf/a1111-sd-webui-locon) into sd-webui, and then use built-in lora system to run your model.
-**LoHa Model supported**
+download [Extension](https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris) into sd-webui, and then use LyCORIS model in the extra netowrks tabs.
 
 **Not For Kohya-ss' Additional Network**
 
@@ -120,8 +128,9 @@ see [Demo.md](https://github.com/KohakuBlueleaf/LyCORIS/blob/main/Demo.md) and [
 ## Change Log
 For full log, please see [Change.md](https://github.com/KohakuBlueleaf/LyCORIS/blob/main/Change.md)
 
-### 2023/04/05 Update to 0.1.5.dev1
+### 2023/04/08 Update for 0.1.5
 * Add (IA)^3 algorithm
+* Add lokr algorithm
 
 ## Todo list
 - [ ] Module and Document for using LyCORIS in any other model, Not only SD.

{lycoris_lora-0.1.5.dev2 → lycoris_lora-0.1.5.dev4}/lycoris/kohya.py

@@ -28,6 +28,7 @@ def create_network(multiplier, network_dim, network_alpha, vae, text_encoder, un
               or kwargs.get('use_conv_cp', False))
     network_module = {
         'lora': LoConModule,
+        'locon': LoConModule,
         'loha': LohaModule,
         'ia3':  IA3Module,
         'lokr': LokrModule,