diffsynth 2.1.8__tar.gz → 2.1.9__tar.gz

{diffsynth-2.1.8 → diffsynth-2.1.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: diffsynth
-Version: 2.1.8
+Version: 2.1.9
 Summary: Enjoy the magic of Diffusion models!
 Author: ModelScope Team
 License-Expression: Apache-2.0
@@ -28,6 +28,7 @@ Requires-Dist: librosa; extra == "audio"
 Provides-Extra: quant
 Requires-Dist: bitsandbytes; extra == "quant"
 Requires-Dist: comfy-kitchen; extra == "quant"
+Requires-Dist: entropack[cuda13]; extra == "quant"
 Requires-Dist: torchao>=0.16; extra == "quant"
 Provides-Extra: training
 Requires-Dist: deepspeed; extra == "training"
@@ -62,6 +63,7 @@ Requires-Dist: torchcodec; extra == "all"
 Requires-Dist: librosa; extra == "all"
 Requires-Dist: bitsandbytes; extra == "all"
 Requires-Dist: comfy-kitchen; extra == "all"
+Requires-Dist: entropack[cuda13]; extra == "all"
 Requires-Dist: torchao>=0.16; extra == "all"
 Requires-Dist: deepspeed; extra == "all"
 Requires-Dist: tensorboard; extra == "all"
@@ -112,6 +114,13 @@ See also:
 
 > Currently, the development personnel of this project are limited, with most of the work handled by [Artiprocher](https://github.com/Artiprocher) and [mi804](https://github.com/mi804). Therefore, the progress of new feature development will be relatively slow, and the speed of responding to and resolving issues is limited. We apologize for this and ask developers to understand.
 
+- **September 29, 2026** We are excited to open-source EntroPack, a general-purpose tensor compression library for PyTorch that provides lossless compression with exact recovery and rate-controlled lossy compression, with GPU encoding and decoding. It is integrated into DiffSynth-Studio as a quantization backend, compressing model weights at arbitrary target bitrates in both inference and LoRA training. For more information, please refer to:
+
+  * Project Page: https://github.com/modelscope/entropack
+  * Technical Report: https://arxiv.org/abs/2609.34185
+  * Pre-quantized Models: https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants
+  * Example Code: [Z-Image-Turbo](/examples/z_image/model_inference_low_vram/Z-Image-Turbo-EntroPack.py), [Qwen-Image-2.1](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-EntroPack.py), [MiniMax-H3](/examples/minimax_h3/model_inference_low_vram/MiniMax-H3-EntroPack.py), [LoRA Training Example](/examples/z_image/model_training/special/quant_training/Z-Image-Turbo-entropack_lossy_quant.sh)
+
 - **September 20, 2026** Qwen-Image-2.1 was open-sourced. The model supports text-to-image generation, image editing, and RGBA output with a transparency channel. DiffSynth-Studio now provides full support for it. For details, please refer to the [documentation](/docs/zh/Model_Details/Qwen-Image-2.1.md) and [example code](/examples/qwen_image_21/).
 
 - **September 14, 2026** We trained and open-sourced DiffSynth-Music, a controllable music generation model based on ACE-Step, which supports five control modes: Beats, Vocals, Accompany, Prosody, and Reference. For more information, please refer to:
@@ -388,6 +397,7 @@ Quick start: experience popular and the latest models:
 | Architecture | Model ID | Inference | Low VRAM Inference | Full Training | Validation After Full Training | LoRA Training | Validation After LoRA Training |
 |-|-|-|-|-|-|-|-|
 | Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1.py) |
+| Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1-Turbo](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1-Turbo) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1-Turbo.py) |
 | MiniMax-H3 | [MiniMax/MiniMax-H3: FL2VA](https://www.modelscope.cn/models/MiniMax/MiniMax-H3) | [code](/examples/minimax_h3/model_inference/MiniMax-H3-FL2VA.py) | [code](/examples/minimax_h3/model_inference_low_vram/MiniMax-H3-FL2VA.py) | [code](/examples/minimax_h3/model_training/full/MiniMax-H3-FL2VA.sh) | [code](/examples/minimax_h3/model_training/validate_full/MiniMax-H3-FL2VA.py) | [code](/examples/minimax_h3/model_training/lora/MiniMax-H3-FL2VA.sh) | [code](/examples/minimax_h3/model_training/validate_lora/MiniMax-H3-FL2VA.py) |
 | MiniMax-H3 | [DiffSynth-Studio/MiniMax-H3-NF4: FL2VA pruned](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-NF4) | [code](/examples/minimax_h3/model_inference/MiniMax-H3-NF4-Pruned-FL2VA.py) | [code](/examples/minimax_h3/model_inference_low_vram/MiniMax-H3-NF4-Pruned-FL2VA.py) | - | - | [code](/examples/minimax_h3/model_training/lora/MiniMax-H3-NF4-Pruned-FL2VA.sh) | [code](/examples/minimax_h3/model_training/validate_lora/MiniMax-H3-NF4-Pruned-FL2VA.py) |
 | ACE-Step | [ACE-Step/acestep-v15-xl-sft](https://www.modelscope.cn/models/ACE-Step/acestep-v15-xl-sft) | [code](/examples/ace_step/model_inference/acestep-v15-xl-sft.py) | [code](/examples/ace_step/model_inference_low_vram/acestep-v15-xl-sft.py) | [code](/examples/ace_step/model_training/full/acestep-v15-xl-sft.sh) | [code](/examples/ace_step/model_training/validate_full/acestep-v15-xl-sft.py) | [code](/examples/ace_step/model_training/lora/acestep-v15-xl-sft.sh) | [code](/examples/ace_step/model_training/validate_lora/acestep-v15-xl-sft.py) |
@@ -670,6 +680,8 @@ https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-44
 | Architecture | Model ID | Inference | Low VRAM Inference | Full Training | Validation After Full Training | LoRA Training | Validation After LoRA Training |
 |-|-|-|-|-|-|-|-|
 | Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1.py) |
+| Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1-Turbo](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1-Turbo) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1-Turbo.py) |
+| Qwen-Image-2.1 | [PAI/Qwen-Image-2.1-Fun-Controlnet-Union](https://www.modelscope.cn/models/PAI/Qwen-Image-2.1-Fun-Controlnet-Union) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1-Fun-Controlnet-Union.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-Fun-Controlnet-Union.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1-Fun-Controlnet-Union.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1-Fun-Controlnet-Union.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1-Fun-Controlnet-Union.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1-Fun-Controlnet-Union.py) |
 | SenseNova-U1 | [SenseNova/SenseNova-U1.5-8B-MoT: T2I](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT) | [code](/examples/sensenova_u1/model_inference/SenseNova-U1.5-8B-MoT.py) | [code](/examples/sensenova_u1/model_inference_low_vram/SenseNova-U1.5-8B-MoT.py) | [code](/examples/sensenova_u1/model_training/full/SenseNova-U1.5-8B-MoT.sh) | [code](/examples/sensenova_u1/model_training/validate_full/SenseNova-U1.5-8B-MoT.py) | [code](/examples/sensenova_u1/model_training/lora/SenseNova-U1.5-8B-MoT.sh) | [code](/examples/sensenova_u1/model_training/validate_lora/SenseNova-U1.5-8B-MoT.py) |
 | SenseNova-U1 | [SenseNova/SenseNova-U1.5-8B-MoT: Edit](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT) | [code](/examples/sensenova_u1/model_inference/SenseNova-U1.5-8B-MoT-Edit.py) | [code](/examples/sensenova_u1/model_inference_low_vram/SenseNova-U1.5-8B-MoT-Edit.py) | [code](/examples/sensenova_u1/model_training/full/SenseNova-U1.5-8B-MoT-Edit.sh) | [code](/examples/sensenova_u1/model_training/validate_full/SenseNova-U1.5-8B-MoT-Edit.py) | [code](/examples/sensenova_u1/model_training/lora/SenseNova-U1.5-8B-MoT-Edit.sh) | [code](/examples/sensenova_u1/model_training/validate_lora/SenseNova-U1.5-8B-MoT-Edit.py) |
 | SenseNova-U1 | [SenseNova/SenseNova-U1.5-8B-MoT-LoRAs: 8-step](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT-LoRAs) | [code](/examples/sensenova_u1/model_inference/SenseNova-U1.5-8B-MoT-LoRA-8step.py) | [code](/examples/sensenova_u1/model_inference_low_vram/SenseNova-U1.5-8B-MoT-LoRA-8step.py) | - | - | - | - |

{diffsynth-2.1.8 → diffsynth-2.1.9}/README.md

@@ -41,6 +41,13 @@ See also:
 
 > Currently, the development personnel of this project are limited, with most of the work handled by [Artiprocher](https://github.com/Artiprocher) and [mi804](https://github.com/mi804). Therefore, the progress of new feature development will be relatively slow, and the speed of responding to and resolving issues is limited. We apologize for this and ask developers to understand.
 
+- **September 29, 2026** We are excited to open-source EntroPack, a general-purpose tensor compression library for PyTorch that provides lossless compression with exact recovery and rate-controlled lossy compression, with GPU encoding and decoding. It is integrated into DiffSynth-Studio as a quantization backend, compressing model weights at arbitrary target bitrates in both inference and LoRA training. For more information, please refer to:
+
+  * Project Page: https://github.com/modelscope/entropack
+  * Technical Report: https://arxiv.org/abs/2609.34185
+  * Pre-quantized Models: https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants
+  * Example Code: [Z-Image-Turbo](/examples/z_image/model_inference_low_vram/Z-Image-Turbo-EntroPack.py), [Qwen-Image-2.1](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-EntroPack.py), [MiniMax-H3](/examples/minimax_h3/model_inference_low_vram/MiniMax-H3-EntroPack.py), [LoRA Training Example](/examples/z_image/model_training/special/quant_training/Z-Image-Turbo-entropack_lossy_quant.sh)
+
 - **September 20, 2026** Qwen-Image-2.1 was open-sourced. The model supports text-to-image generation, image editing, and RGBA output with a transparency channel. DiffSynth-Studio now provides full support for it. For details, please refer to the [documentation](/docs/zh/Model_Details/Qwen-Image-2.1.md) and [example code](/examples/qwen_image_21/).
 
 - **September 14, 2026** We trained and open-sourced DiffSynth-Music, a controllable music generation model based on ACE-Step, which supports five control modes: Beats, Vocals, Accompany, Prosody, and Reference. For more information, please refer to:
@@ -317,6 +324,7 @@ Quick start: experience popular and the latest models:
 | Architecture | Model ID | Inference | Low VRAM Inference | Full Training | Validation After Full Training | LoRA Training | Validation After LoRA Training |
 |-|-|-|-|-|-|-|-|
 | Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1.py) |
+| Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1-Turbo](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1-Turbo) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1-Turbo.py) |
 | MiniMax-H3 | [MiniMax/MiniMax-H3: FL2VA](https://www.modelscope.cn/models/MiniMax/MiniMax-H3) | [code](/examples/minimax_h3/model_inference/MiniMax-H3-FL2VA.py) | [code](/examples/minimax_h3/model_inference_low_vram/MiniMax-H3-FL2VA.py) | [code](/examples/minimax_h3/model_training/full/MiniMax-H3-FL2VA.sh) | [code](/examples/minimax_h3/model_training/validate_full/MiniMax-H3-FL2VA.py) | [code](/examples/minimax_h3/model_training/lora/MiniMax-H3-FL2VA.sh) | [code](/examples/minimax_h3/model_training/validate_lora/MiniMax-H3-FL2VA.py) |
 | MiniMax-H3 | [DiffSynth-Studio/MiniMax-H3-NF4: FL2VA pruned](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-NF4) | [code](/examples/minimax_h3/model_inference/MiniMax-H3-NF4-Pruned-FL2VA.py) | [code](/examples/minimax_h3/model_inference_low_vram/MiniMax-H3-NF4-Pruned-FL2VA.py) | - | - | [code](/examples/minimax_h3/model_training/lora/MiniMax-H3-NF4-Pruned-FL2VA.sh) | [code](/examples/minimax_h3/model_training/validate_lora/MiniMax-H3-NF4-Pruned-FL2VA.py) |
 | ACE-Step | [ACE-Step/acestep-v15-xl-sft](https://www.modelscope.cn/models/ACE-Step/acestep-v15-xl-sft) | [code](/examples/ace_step/model_inference/acestep-v15-xl-sft.py) | [code](/examples/ace_step/model_inference_low_vram/acestep-v15-xl-sft.py) | [code](/examples/ace_step/model_training/full/acestep-v15-xl-sft.sh) | [code](/examples/ace_step/model_training/validate_full/acestep-v15-xl-sft.py) | [code](/examples/ace_step/model_training/lora/acestep-v15-xl-sft.sh) | [code](/examples/ace_step/model_training/validate_lora/acestep-v15-xl-sft.py) |
@@ -599,6 +607,8 @@ https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-44
 | Architecture | Model ID | Inference | Low VRAM Inference | Full Training | Validation After Full Training | LoRA Training | Validation After LoRA Training |
 |-|-|-|-|-|-|-|-|
 | Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1.py) |
+| Qwen-Image-2.1 | [Qwen/Qwen-Image-2.1-Turbo](https://www.modelscope.cn/models/Qwen/Qwen-Image-2.1-Turbo) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1-Turbo.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1-Turbo.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1-Turbo.py) |
+| Qwen-Image-2.1 | [PAI/Qwen-Image-2.1-Fun-Controlnet-Union](https://www.modelscope.cn/models/PAI/Qwen-Image-2.1-Fun-Controlnet-Union) | [code](/examples/qwen_image_21/model_inference/Qwen-Image-2.1-Fun-Controlnet-Union.py) | [code](/examples/qwen_image_21/model_inference_low_vram/Qwen-Image-2.1-Fun-Controlnet-Union.py) | [code](/examples/qwen_image_21/model_training/full/Qwen-Image-2.1-Fun-Controlnet-Union.sh) | [code](/examples/qwen_image_21/model_training/validate_full/Qwen-Image-2.1-Fun-Controlnet-Union.py) | [code](/examples/qwen_image_21/model_training/lora/Qwen-Image-2.1-Fun-Controlnet-Union.sh) | [code](/examples/qwen_image_21/model_training/validate_lora/Qwen-Image-2.1-Fun-Controlnet-Union.py) |
 | SenseNova-U1 | [SenseNova/SenseNova-U1.5-8B-MoT: T2I](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT) | [code](/examples/sensenova_u1/model_inference/SenseNova-U1.5-8B-MoT.py) | [code](/examples/sensenova_u1/model_inference_low_vram/SenseNova-U1.5-8B-MoT.py) | [code](/examples/sensenova_u1/model_training/full/SenseNova-U1.5-8B-MoT.sh) | [code](/examples/sensenova_u1/model_training/validate_full/SenseNova-U1.5-8B-MoT.py) | [code](/examples/sensenova_u1/model_training/lora/SenseNova-U1.5-8B-MoT.sh) | [code](/examples/sensenova_u1/model_training/validate_lora/SenseNova-U1.5-8B-MoT.py) |
 | SenseNova-U1 | [SenseNova/SenseNova-U1.5-8B-MoT: Edit](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT) | [code](/examples/sensenova_u1/model_inference/SenseNova-U1.5-8B-MoT-Edit.py) | [code](/examples/sensenova_u1/model_inference_low_vram/SenseNova-U1.5-8B-MoT-Edit.py) | [code](/examples/sensenova_u1/model_training/full/SenseNova-U1.5-8B-MoT-Edit.sh) | [code](/examples/sensenova_u1/model_training/validate_full/SenseNova-U1.5-8B-MoT-Edit.py) | [code](/examples/sensenova_u1/model_training/lora/SenseNova-U1.5-8B-MoT-Edit.sh) | [code](/examples/sensenova_u1/model_training/validate_lora/SenseNova-U1.5-8B-MoT-Edit.py) |
 | SenseNova-U1 | [SenseNova/SenseNova-U1.5-8B-MoT-LoRAs: 8-step](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT-LoRAs) | [code](/examples/sensenova_u1/model_inference/SenseNova-U1.5-8B-MoT-LoRA-8step.py) | [code](/examples/sensenova_u1/model_inference_low_vram/SenseNova-U1.5-8B-MoT-LoRA-8step.py) | - | - | - | - |

{diffsynth-2.1.8 → diffsynth-2.1.9}/diffsynth/configs/model_configs.py

@@ -1735,6 +1735,12 @@ qwen_image_21_series = [
         "model_class": "diffsynth.models.qwen_image_21_text_encoder.QwenImage21TextEncoder",
         "state_dict_converter": "diffsynth.utils.state_dict_converters.qwen_image_21_text_encoder.QwenImage21TextEncoderStateDictConverter",
     },
+    {
+        # Example: ModelConfig(model_id="PAI/Qwen-Image-2.1-Fun-Controlnet-Union", origin_file_pattern="Qwen-Image-2.1-Fun-Controlnet-Union.safetensors")
+        "model_hash": "21fca5e7dc68c836804b46f735677546",
+        "model_name": "qwen_image_21_controlnet",
+        "model_class": "diffsynth.models.qwen_image_21_controlnet.QwenImage21ControlNet",
+    },
 ]
 
 MODEL_CONFIGS = (

{diffsynth-2.1.8 → diffsynth-2.1.9}/diffsynth/configs/vram_management_module_maps.py

@@ -30,7 +30,6 @@ VRAM_MANAGEMENT_MODULE_MAPS = {
         "torch.nn.Conv3d": "diffsynth.core.vram.layers.AutoWrappedModule",
         "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLTextRMSNorm": "diffsynth.core.vram.layers.AutoWrappedModule",
         "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLTextRotaryEmbedding": "diffsynth.core.vram.layers.AutoWrappedModule",
-        "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLVisionModel": "diffsynth.core.vram.layers.AutoWrappedModule",
         "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLVisionPatchEmbed": "diffsynth.core.vram.layers.AutoWrappedModule",
         "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLVisionRotaryEmbedding": "diffsynth.core.vram.layers.AutoWrappedModule",
     },
@@ -552,6 +551,10 @@ VRAM_MANAGEMENT_MODULE_MAPS = {
         "diffsynth.models.qwen_image_21_dit.QwenImage21RMSNorm": "diffsynth.core.vram.layers.AutoWrappedModule",
         "diffsynth.models.qwen_image_21_dit.QwenImage21ZeroCenterRMSNorm": "diffsynth.core.vram.layers.AutoWrappedModule",
     },
+    "diffsynth.models.qwen_image_21_controlnet.QwenImage21ControlNet": {
+        "torch.nn.Linear": "diffsynth.core.vram.layers.AutoWrappedLinear",
+        "diffsynth.models.qwen_image_21_dit.QwenImage21RMSNorm": "diffsynth.core.vram.layers.AutoWrappedModule",
+    },
     "diffsynth.models.qwen_image_21_vae.QwenImage21VAE": {
         "torch.nn.Conv2d": "diffsynth.core.vram.layers.AutoWrappedModule",
         "diffsynth.models.qwen_image_21_vae.QwenImage21RMS_norm": "diffsynth.core.vram.layers.AutoWrappedModule",
@@ -561,9 +564,10 @@ VRAM_MANAGEMENT_MODULE_MAPS = {
         "torch.nn.Embedding": "diffsynth.core.vram.layers.AutoWrappedModule",
         "torch.nn.LayerNorm": "diffsynth.core.vram.layers.AutoWrappedModule",
         "torch.nn.Conv3d": "diffsynth.core.vram.layers.AutoWrappedModule",
-        "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLVisionModel": "diffsynth.core.vram.layers.AutoWrappedModule",
         "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLTextRMSNorm": "diffsynth.core.vram.layers.AutoWrappedModule",
         "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLTextRotaryEmbedding": "diffsynth.core.vram.layers.AutoWrappedModule",
+        "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLVisionPatchEmbed": "diffsynth.core.vram.layers.AutoWrappedModule",
+        "transformers.models.qwen3_vl.modeling_qwen3_vl.Qwen3VLVisionRotaryEmbedding": "diffsynth.core.vram.layers.AutoWrappedModule",
     },
 }
 

{diffsynth-2.1.8 → diffsynth-2.1.9}/diffsynth/core/quant/backends/__init__.py

@@ -4,6 +4,7 @@ _LAZY_BACKENDS = {
     "bitsandbytes": ".bitsandbytes",
     "torchao": ".torchao",
     "comfy_kitchen": ".comfy_kitchen",
+    "entropack": ".entropack",
 }
 _loaded = set()
 

diffsynth-2.1.9/diffsynth/core/quant/backends/entropack.py

@@ -0,0 +1,143 @@
+from dataclasses import dataclass
+
+import torch
+
+from ..base import BackendConfig, QuantBackend, register_quant_backend
+from ..config import register_quant_method
+
+try:
+    import entropack as ep
+
+    _REQUIRED_ENTROPACK_API = (
+        ep.CompressedLinear, ep.CompressedFP8Linear, ep.CompressedINT8Linear,
+        ep.DFloat11Config, ep.TileANSConfig, ep.LatticeRANSConfig, ep.RawConfig,
+    )
+    ENTROPACK_AVAILABLE = True
+    _ENTROPACK_IMPORT_ERROR = None
+except (ImportError, AttributeError) as error:
+    ep = None
+    ENTROPACK_AVAILABLE = False
+    _ENTROPACK_IMPORT_ERROR = error
+
+
+if ENTROPACK_AVAILABLE:
+
+    @dataclass
+    class EntroPackDFloat11Config(BackendConfig, ep.DFloat11Config):
+        linear_cls = ep.CompressedLinear
+
+    @dataclass
+    class EntroPackTileANSConfig(BackendConfig, ep.TileANSConfig):
+        linear_cls = ep.CompressedLinear
+
+    @dataclass
+    class EntroPackLossyQuantConfig(BackendConfig, ep.LatticeRANSConfig):
+        linear_cls = ep.CompressedLinear
+
+    @dataclass
+    class EntroPackFP8CodedConfig(BackendConfig, ep.LatticeRANSConfig):
+        linear_cls = ep.CompressedFP8Linear
+
+    @dataclass
+    class EntroPackFP8RawConfig(BackendConfig, ep.RawConfig):
+        linear_cls = ep.CompressedFP8Linear
+
+    @dataclass
+    class EntroPackINT8CodedConfig(BackendConfig, ep.LatticeRANSConfig):
+        linear_cls = ep.CompressedINT8Linear
+
+    @dataclass
+    class EntroPackINT8RawConfig(BackendConfig, ep.RawConfig):
+        linear_cls = ep.CompressedINT8Linear
+
+
+@register_quant_backend("entropack")
+class EntroPackQuantBackend(QuantBackend):
+    """Adapter over entropack's `CompressedLinear` family; a method's config is the config its layers get."""
+
+    project_url = "https://github.com/modelscope/entropack"
+
+    def validate_environment(self):
+        if not ENTROPACK_AVAILABLE:
+            raise ImportError(
+                "entropack with the CompressedLinear / CompressedFP8Linear / CompressedINT8Linear "
+                "classes is required for this quantization method. Install it via "
+                '`pip install "entropack[cuda13]"` (use `cuda12` for a CUDA 12 environment) '
+                'or `pip install "diffsynth[quant]"`.'
+            ) from _ENTROPACK_IMPORT_ERROR
+
+    def capabilities(self):
+        return {
+            "is_serializable": True,
+            "is_differentiable": True,
+            "is_compileable": False,
+            "requires_calibration": False,
+        }
+
+    def quantized_linear_classes(self):
+        return (ep.CompressedLinear, ep.CompressedFP8Linear, ep.CompressedINT8Linear)
+
+    def unflatten_state_dict(self, state_dict, metadata):
+        return state_dict
+
+    def create_quantized_linear(self, linear, compute_device=None, model_device=None):
+        linear.requires_grad_(False)
+        if compute_device is not None:
+            linear = linear.to(device=compute_device)
+        quantized = self.config.linear_cls.from_linear(linear, config=self.config)
+        return quantized if model_device is None else quantized.to(device=model_device)
+
+    def create_quantized_linear_shell(self, linear, compute_dtype):
+        return self.config.linear_cls(
+            linear.in_features, linear.out_features, bias=linear.bias is not None,
+            dtype=compute_dtype, config=self.config,
+        )
+
+    def dequantize_to_linear(self, module, compute_dtype, compute_device=None, model_device=None):
+        if compute_device is not None:
+            module = module.to(device=compute_device)
+        weight = module.dequantize(compute_device).to(compute_dtype)
+        linear = torch.nn.Linear(module.in_features, module.out_features, bias=module.bias is not None, device="meta")
+        linear.weight = torch.nn.Parameter(weight, requires_grad=False)
+        if module.bias is not None:
+            linear.bias = torch.nn.Parameter(
+                module.bias.data.to(dtype=compute_dtype, device=weight.device), requires_grad=False,
+            )
+        return linear if model_device is None else linear.to(device=model_device)
+
+
+def _df11_config(kwargs):
+    return EntroPackDFloat11Config.from_kwargs(kwargs)
+
+
+def _tile_ans_config(kwargs):
+    return EntroPackTileANSConfig.from_kwargs(kwargs)
+
+
+def _lossy_config(kwargs):
+    return EntroPackLossyQuantConfig.from_kwargs(kwargs)
+
+
+def _fp8_coded_config(kwargs):
+    return EntroPackFP8CodedConfig.from_kwargs(kwargs)
+
+
+def _fp8_raw_config(kwargs):
+    return EntroPackFP8RawConfig.from_kwargs(kwargs)
+
+
+def _int8_coded_config(kwargs):
+    return EntroPackINT8CodedConfig.from_kwargs(kwargs)
+
+
+def _int8_raw_config(kwargs):
+    return EntroPackINT8RawConfig.from_kwargs(kwargs)
+
+
+register_quant_method("entropack_lossless_df11", "entropack", _df11_config, label="lossless, dfloat11")
+register_quant_method("entropack_lossless_tile_ans", "entropack", _tile_ans_config, label="lossless, tile_ans")
+register_quant_method("entropack_lossy_quant", "entropack", _lossy_config, label="lossy, lattice rate")
+register_quant_method("entropack_lossy_quant_fp8", "entropack", _fp8_coded_config, label="W8A8, fp8, lattice-coded")
+register_quant_method("entropack_lossy_quant_fp8_raw", "entropack", _fp8_raw_config, label="W8A8, fp8, codes verbatim")
+register_quant_method("entropack_lossy_quant_int8", "entropack", _int8_coded_config, label="W8A8, int8, lattice-coded")
+register_quant_method("entropack_lossy_quant_int8_raw", "entropack", _int8_raw_config, label="W8A8, int8, codes verbatim")

{diffsynth-2.1.8 → diffsynth-2.1.9}/diffsynth/diffusion/flow_match.py

@@ -60,26 +60,29 @@ class FlowMatchScheduler():
         return mu
     
     @staticmethod
-    def set_timesteps_qwen_image(num_inference_steps=100, denoising_strength=1.0, exponential_shift_mu=None, dynamic_shift_len=None):
+    def set_timesteps_qwen_image(num_inference_steps=100, denoising_strength=1.0, exponential_shift_mu=None, dynamic_shift_len=None, sigmas=None):
         sigma_min = 0.0
         sigma_max = 1.0
         num_train_timesteps = 1000
         shift_terminal = 0.02
         # Sigmas
-        sigma_start = sigma_min + (sigma_max - sigma_min) * denoising_strength
-        sigmas = torch.linspace(sigma_start, sigma_min, num_inference_steps + 1)[:-1]
-        # Mu
-        if exponential_shift_mu is not None:
-            mu = exponential_shift_mu
-        elif dynamic_shift_len is not None:
-            mu = FlowMatchScheduler._calculate_shift_qwen_image(dynamic_shift_len)
+        if sigmas is None:
+            sigma_start = sigma_min + (sigma_max - sigma_min) * denoising_strength
+            sigmas = torch.linspace(sigma_start, sigma_min, num_inference_steps + 1)[:-1]
+            # Mu
+            if exponential_shift_mu is not None:
+                mu = exponential_shift_mu
+            elif dynamic_shift_len is not None:
+                mu = FlowMatchScheduler._calculate_shift_qwen_image(dynamic_shift_len)
+            else:
+                mu = 0.8
+            sigmas = math.exp(mu) / (math.exp(mu) + (1 / sigmas - 1))
+            # Shift terminal
+            one_minus_z = 1 - sigmas
+            scale_factor = one_minus_z[-1] / (1 - shift_terminal)
+            sigmas = 1 - (one_minus_z / scale_factor)
         else:
-            mu = 0.8
-        sigmas = math.exp(mu) / (math.exp(mu) + (1 / sigmas - 1))
-        # Shift terminal
-        one_minus_z = 1 - sigmas
-        scale_factor = one_minus_z[-1] / (1 - shift_terminal)
-        sigmas = 1 - (one_minus_z / scale_factor)
+            sigmas = torch.tensor(sigmas)
         # Timesteps
         timesteps = sigmas * num_train_timesteps
         return sigmas, timesteps

{diffsynth-2.1.8 → diffsynth-2.1.9}/diffsynth/diffusion/training_module.py

@@ -280,11 +280,53 @@ class DiffusionTrainingModule(torch.nn.Module):
         return lora_target_modules
     
 
+    @staticmethod
+    def format_lora_target_modules(lora_target_modules):
+        def compress(paths):
+            if len(paths) == 1:
+                return ".".join(paths[0])
+            # Factor out the common prefix.
+            prefix_len = 0
+            while all(len(path) > prefix_len for path in paths) and len({path[prefix_len] for path in paths}) == 1:
+                prefix_len += 1
+            prefix = paths[0][:prefix_len]
+            paths = [path[prefix_len:] for path in paths]
+            # Factor out the common suffix.
+            suffix_len = 0
+            while all(len(path) > suffix_len for path in paths) and len({path[len(path) - 1 - suffix_len] for path in paths}) == 1:
+                suffix_len += 1
+            suffix = paths[0][len(paths[0]) - suffix_len:] if suffix_len > 0 else []
+            paths = [path[:len(path) - suffix_len] for path in paths]
+            # Group by the first token, then merge the groups sharing an identical tail.
+            groups = {}
+            for path in paths:
+                groups.setdefault(path[0], []).append(path[1:])
+            keys = list(groups.keys())
+            sorted_tails = [sorted(groups[key]) for key in keys]
+            if len(keys) > 1 and all(tail == sorted_tails[0] for tail in sorted_tails):
+                common_tail = groups[keys[0]]
+                tail_str = compress(common_tail) if len(common_tail) > 0 else ""
+                middle = "{" + ",".join(keys) + "}" + (("." + tail_str) if tail_str != "" else "")
+            else:
+                rendered = []
+                for key in keys:
+                    tail = groups[key]
+                    if len(tail) == 1 and len(tail[0]) == 0:
+                        rendered.append(key)
+                    elif len(tail) == 1:
+                        rendered.append(key + "." + ".".join(tail[0]))
+                    else:
+                        rendered.append(key + "." + compress(tail))
+                middle = "{" + ",".join(rendered) + "}" if len(rendered) > 1 else rendered[0]
+            return ".".join(prefix + ([middle] if middle != "" else []) + suffix)
+
+        return compress([name.split(".") for name in lora_target_modules])
+
+
     def parse_lora_target_modules(self, model, lora_target_modules):
         if lora_target_modules == "":
-            print("No LoRA target modules specified. The framework will automatically search for them.")
             lora_target_modules = self.auto_detect_lora_target_modules(model)
-            print(f"LoRA will be patched at {lora_target_modules}.")
+            print(f"No LoRA target modules specified. LoRA will be patched at {self.format_lora_target_modules(lora_target_modules)}.")
         else:
             lora_target_modules = lora_target_modules.split(",")
         return lora_target_modules

diffsynth-2.1.9/diffsynth/models/qwen_image_21_controlnet.py

@@ -0,0 +1,101 @@
+from .qwen_image_21_dit import QwenImage21TransformerBlock
+from ..core.gradient import gradient_checkpoint_forward
+import torch
+import torch.nn as nn
+
+
+class QwenImage21ControlNetBlock(QwenImage21TransformerBlock):
+    def __init__(
+        self,
+        dim: int,
+        num_attention_heads: int,
+        attention_head_dim: int,
+        mlp_ratio: int = 3,
+        eps: float = 1e-6,
+        block_id: int = 0,
+    ):
+        super().__init__(dim, num_attention_heads, attention_head_dim, mlp_ratio, eps)
+        self.block_id = block_id
+        if block_id == 0:
+            self.before_proj = nn.Linear(dim, dim)
+        self.after_proj = nn.Linear(dim, dim)
+
+    def forward(self, c, x, **kwargs):
+        if self.block_id == 0:
+            c = self.before_proj(c) + x
+            all_c = []
+        else:
+            all_c = list(torch.unbind(c))
+            c = all_c.pop(-1)
+
+        c = super().forward(c, **kwargs)
+        c_skip = self.after_proj(c)
+        all_c += [c_skip, c]
+        c = torch.stack(all_c)
+        return c
+
+
+class QwenImage21ControlNet(nn.Module):
+    _repeated_blocks = ["QwenImage21ControlNetBlock"]
+
+    def __init__(
+        self,
+        control_layers: tuple = (0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30),
+        control_in_dim: int = 129,
+        dim: int = 4096,
+        num_attention_heads: int = 32,
+        attention_head_dim: int = 128,
+        mlp_ratio: int = 3,
+        eps: float = 1e-6,
+    ):
+        super().__init__()
+        self.control_layers = tuple(sorted({int(i) for i in control_layers}))
+        self.control_in_dim = control_in_dim
+        self.control_layers_mapping = {i: n for n, i in enumerate(self.control_layers)}
+        self.control_img_in = nn.Linear(control_in_dim, dim)
+        self.control_blocks = nn.ModuleList(
+            [
+                QwenImage21ControlNetBlock(dim, num_attention_heads, attention_head_dim, mlp_ratio, eps, block_id=i)
+                for i in self.control_layers
+            ]
+        )
+
+    def forward(
+        self,
+        joint_hidden_states: torch.Tensor,
+        control_context: torch.Tensor,
+        image_positions: torch.Tensor,
+        modulation: torch.Tensor,
+        rotary_emb: torch.Tensor,
+        attention_mask,
+        target_token_mask: torch.Tensor | None,
+        segments,
+        key_valid: torch.Tensor | None,
+        control_scale: float = 1.0,
+        use_gradient_checkpointing: bool = False,
+        use_gradient_checkpointing_offload: bool = False,
+    ) -> dict[int, torch.Tensor]:
+        control_features = self.control_img_in(control_context)
+        c = torch.zeros_like(joint_hidden_states)
+        c[:, image_positions] = control_features.to(joint_hidden_states.dtype)
+        kwargs = dict(
+            modulation=modulation,
+            rotary_emb=rotary_emb,
+            attention_mask=attention_mask,
+            target_token_mask=target_token_mask,
+            kv_cache=None,
+            cache_write_slice=None,
+            segments=segments,
+            key_valid=key_valid,
+        )
+        for block in self.control_blocks:
+            c = gradient_checkpoint_forward(
+                block,
+                use_gradient_checkpointing,
+                use_gradient_checkpointing_offload,
+                c,
+                joint_hidden_states,
+                **kwargs,
+            )
+        hints = torch.unbind(c)[:-1]
+        return {block_id: hints[index] * control_scale for block_id, index in self.control_layers_mapping.items()}