diffsynth 2.1.4__tar.gz → 2.1.5__tar.gz

{diffsynth-2.1.4 → diffsynth-2.1.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: diffsynth
-Version: 2.1.4
+Version: 2.1.5
 Summary: Enjoy the magic of Diffusion models!
 Author: ModelScope Team
 License-Expression: Apache-2.0
@@ -804,6 +804,9 @@ https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-44
 | FLUX.1 | [black-forest-labs/FLUX.1-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-dev) | [code](/examples/flux/model_inference/FLUX.1-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev.py) |
 | FLUX.1 | [black-forest-labs/FLUX.1-Krea-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Krea-dev) | [code](/examples/flux/model_inference/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Krea-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Krea-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Krea-dev.py) |
 | FLUX.1 | [black-forest-labs/FLUX.1-Kontext-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Kontext-dev) | [code](/examples/flux/model_inference/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Kontext-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Kontext-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Kontext-dev.py) |
+| FLUX.1 | [black-forest-labs/FLUX.1-Fill-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Fill-dev) | [code](/examples/flux/model_inference/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Fill-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Fill-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Fill-dev.py) |
+| FLUX.1 | [black-forest-labs/FLUX.1-Redux-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Redux-dev) | [code](/examples/flux/model_inference/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Redux-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Redux-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Redux-dev.py) |
+| FLUX.1 | [HuanJue/Insert-Anything](https://www.modelscope.cn/models/HuanJue/Insert-Anything) | [code](/examples/flux/model_inference/Insert-Anything.py) | [code](/examples/flux/model_inference_low_vram/Insert-Anything.py) | - | - | [code](/examples/flux/model_training/lora/Insert-Anything.sh) | [code](/examples/flux/model_training/validate_lora/Insert-Anything.py) |
 | FLUX.1 | [alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta](https://www.modelscope.cn/models/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Inpainting-Beta.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Inpainting-Beta.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Inpainting-Beta.py) |
 | FLUX.1 | [InstantX/FLUX.1-dev-Controlnet-Union-alpha](https://www.modelscope.cn/models/InstantX/FLUX.1-dev-Controlnet-Union-alpha) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Union-alpha.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Union-alpha.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Union-alpha.py) |
 | FLUX.1 | [jasperai/Flux.1-dev-Controlnet-Upscaler](https://www.modelscope.cn/models/jasperai/Flux.1-dev-Controlnet-Upscaler) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Upscaler.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Upscaler.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Upscaler.py) |

{diffsynth-2.1.4 → diffsynth-2.1.5}/README.md

@@ -737,6 +737,9 @@ https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-44
 | FLUX.1 | [black-forest-labs/FLUX.1-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-dev) | [code](/examples/flux/model_inference/FLUX.1-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev.py) |
 | FLUX.1 | [black-forest-labs/FLUX.1-Krea-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Krea-dev) | [code](/examples/flux/model_inference/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Krea-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Krea-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Krea-dev.py) |
 | FLUX.1 | [black-forest-labs/FLUX.1-Kontext-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Kontext-dev) | [code](/examples/flux/model_inference/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Kontext-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Kontext-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Kontext-dev.py) |
+| FLUX.1 | [black-forest-labs/FLUX.1-Fill-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Fill-dev) | [code](/examples/flux/model_inference/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Fill-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Fill-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Fill-dev.py) |
+| FLUX.1 | [black-forest-labs/FLUX.1-Redux-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Redux-dev) | [code](/examples/flux/model_inference/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Redux-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Redux-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Redux-dev.py) |
+| FLUX.1 | [HuanJue/Insert-Anything](https://www.modelscope.cn/models/HuanJue/Insert-Anything) | [code](/examples/flux/model_inference/Insert-Anything.py) | [code](/examples/flux/model_inference_low_vram/Insert-Anything.py) | - | - | [code](/examples/flux/model_training/lora/Insert-Anything.sh) | [code](/examples/flux/model_training/validate_lora/Insert-Anything.py) |
 | FLUX.1 | [alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta](https://www.modelscope.cn/models/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Inpainting-Beta.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Inpainting-Beta.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Inpainting-Beta.py) |
 | FLUX.1 | [InstantX/FLUX.1-dev-Controlnet-Union-alpha](https://www.modelscope.cn/models/InstantX/FLUX.1-dev-Controlnet-Union-alpha) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Union-alpha.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Union-alpha.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Union-alpha.py) |
 | FLUX.1 | [jasperai/Flux.1-dev-Controlnet-Upscaler](https://www.modelscope.cn/models/jasperai/Flux.1-dev-Controlnet-Upscaler) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Upscaler.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Upscaler.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Upscaler.py) |

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/core/loader/config.py

@@ -75,11 +75,11 @@ class ModelConfig:
     def parse_download_source(self):
         if self.download_source is None:
             if os.environ.get('DIFFSYNTH_DOWNLOAD_SOURCE') is not None:
-                return os.environ.get('DIFFSYNTH_DOWNLOAD_SOURCE')
+                return os.environ.get('DIFFSYNTH_DOWNLOAD_SOURCE').lower()
             else:
                 return "modelscope"
         else:
-            return self.download_source
+            return self.download_source.lower()
         
     def parse_skip_download(self):
         if self.skip_download is None:

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/core/loader/model.py

@@ -119,6 +119,9 @@ def load_model(model_class, path, config=None, torch_dtype=torch.bfloat16, devic
         # Because some models override the behavior of `to()`,
         # especially those from libraries like Transformers.
         model = model.to(dtype=torch_dtype, device=device)
+    if quantize is not None:
+        # Downstream steps (e.g. LoRA hot-loading) need the config to handle the quantized layers.
+        model.quantize_config = quantize
     if hasattr(model, "eval"):
         model = model.eval()
     return model

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/core/quant/backends/bitsandbytes.py

@@ -11,26 +11,27 @@ except ImportError:
     BITSANDBYTES_AVAILABLE = False
 
 
-def _assign_params_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs):
-    """`_load_from_state_dict` replacement for shells: assigns tensors as-is, since packed weights do not pass the standard shape check."""
-    local_names = set()
-    for name, current in list(self._parameters.items()) + list(self._buffers.items()):
-        if current is None:
-            continue
-        local_names.add(name)
-        key = prefix + name
-        if key in state_dict:
-            value = state_dict[key]
-            if name in self._parameters:
-                self._parameters[name] = value if isinstance(value, torch.nn.Parameter) else torch.nn.Parameter(value, requires_grad=False)
-            else:
-                self._buffers[name] = value
-        elif strict:
-            missing_keys.append(key)
-    if strict:
-        for key in state_dict:
-            if key.startswith(prefix) and key[len(prefix):].split(".", 1)[0] not in local_names:
-                unexpected_keys.append(key)
+if BITSANDBYTES_AVAILABLE:
+    class BitsAndBytesLinear4bit(bnb.nn.Linear4bit):
+        def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs):
+            local_names = set()
+            for name, current in list(self._parameters.items()) + list(self._buffers.items()):
+                if current is None:
+                    continue
+                local_names.add(name)
+                key = prefix + name
+                if key in state_dict:
+                    value = state_dict[key]
+                    if name in self._parameters:
+                        self._parameters[name] = value if isinstance(value, torch.nn.Parameter) else torch.nn.Parameter(value, requires_grad=False)
+                    else:
+                        self._buffers[name] = value
+                elif strict:
+                    missing_keys.append(key)
+            if strict:
+                for key in state_dict:
+                    if key.startswith(prefix) and key[len(prefix):].split(".", 1)[0] not in local_names:
+                        unexpected_keys.append(key)
 
 
 @register_quant_backend("bitsandbytes")
@@ -55,7 +56,7 @@ class BitsAndBytesQuantBackend(QuantBackend):
         }
 
     def quantized_linear_classes(self):
-        return (bnb.nn.Linear4bit,)
+        return (BitsAndBytesLinear4bit,)
 
     def create_quantized_linear(self, linear, compute_device=None, model_device=None):
         """The `meta` shell avoids allocating an fp weight; bnb quantizes while `Params4bit` moves to `compute_device`."""
@@ -65,7 +66,7 @@ class BitsAndBytesQuantBackend(QuantBackend):
             compute_device = linear.weight.device
 
         with torch.device("meta"):
-            quant_linear = bnb.nn.Linear4bit(
+            quant_linear = BitsAndBytesLinear4bit(
                 linear.in_features,
                 linear.out_features,
                 bias=linear.bias is not None,
@@ -89,7 +90,7 @@ class BitsAndBytesQuantBackend(QuantBackend):
 
     def create_quantized_linear_shell(self, linear, compute_dtype):
         with torch.device("meta"):
-            shell = bnb.nn.Linear4bit(
+            shell = BitsAndBytesLinear4bit(
                 linear.in_features,
                 linear.out_features,
                 bias=linear.bias is not None,
@@ -98,7 +99,6 @@ class BitsAndBytesQuantBackend(QuantBackend):
                 quant_type=self.config.quant_type,
                 quant_storage=self.config.quant_storage,
             )
-        shell._load_from_state_dict = _assign_params_from_state_dict.__get__(shell)
         return shell
 
     def unflatten_state_dict(self, state_dict, metadata):

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/core/vram/layers.py

@@ -442,7 +442,7 @@ class AutoWrappedLinear(torch.nn.Linear, AutoTorchModule, LoRAHotLoadMixin):
         return out
 
 
-class AutoWrappedQuantizedModule(AutoTorchModule, LoRAHotLoadMixin):
+class AutoWrappedQuantizedModule(torch.nn.Linear, AutoTorchModule, LoRAHotLoadMixin):
     def __init__(
         self,
         module: torch.nn.Module,
@@ -467,7 +467,8 @@ class AutoWrappedQuantizedModule(AutoTorchModule, LoRAHotLoadMixin):
                 "Disk offload for quantized layers requires both `disk_map` and `quantize`, "
                 "so each layer can rebuild its packed weight and quant state lazily."
             )
-        super().__init__(
+        AutoTorchModule.__init__(
+            self,
             offload_dtype,
             offload_device,
             onload_dtype,
@@ -478,6 +479,8 @@ class AutoWrappedQuantizedModule(AutoTorchModule, LoRAHotLoadMixin):
             computation_device,
             vram_limit,
         )
+        self.in_features = module.in_features
+        self.out_features = module.out_features
         self.module = module
         self.name = name
         self.disk_offload = disk_offload

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/diffusion/base_pipeline.py

@@ -240,6 +240,13 @@ class BasePipeline(torch.nn.Module):
                 module.computation_device = device
                 
     
+    def check_quant_hot_load(self, module: torch.nn.Module):
+        # A quantized weight cannot absorb a fused LoRA, so hot-loading is the only option for it.
+        if getattr(module, "quantize_config", None) is None:
+            return module
+        return self.enable_lora_hot_loading(module)
+
+
     def load_lora(
         self,
         module: torch.nn.Module,
@@ -249,6 +256,7 @@ class BasePipeline(torch.nn.Module):
         state_dict=None,
         verbose=1,
     ):
+        module = self.check_quant_hot_load(module)
         if state_dict is None:
             if isinstance(lora_config, str):
                 lora = load_state_dict(lora_config, torch_dtype=self.torch_dtype, device=self.device)
@@ -370,7 +378,7 @@ class BasePipeline(torch.nn.Module):
             "computation_dtype": self.torch_dtype,
             "computation_device": self.device,
         }
-        model = enable_vram_management(model, module_map, vram_config=vram_config)
+        model = enable_vram_management(model, module_map, vram_config=vram_config, quantize=getattr(model, "quantize_config", None))
         return model
 
     def compile_pipeline(self, mode: str = "default", dynamic: bool = True, fullgraph: bool = False, compile_models: list = None, **kwargs):

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/diffusion/loss.py

@@ -63,7 +63,7 @@ def FlowMatchSFTAudioVideoLoss(pipe: BasePipeline, **inputs):
     return loss
 
 
-def FlowMatchSFTMiniMaxH3AudioVideoLoss(pipe: BasePipeline, **inputs):
+def FlowMatchSFTMiniMaxH3AudioVideoLoss(pipe: BasePipeline, training_cfg_scale: float = 1.0, inputs_nega: dict | None = None, **inputs):
     max_timestep_boundary = int(inputs.get("max_timestep_boundary", 1) * len(pipe.scheduler.timesteps))
     min_timestep_boundary = int(inputs.get("min_timestep_boundary", 0) * len(pipe.scheduler.timesteps))
 
@@ -81,11 +81,34 @@ def FlowMatchSFTMiniMaxH3AudioVideoLoss(pipe: BasePipeline, **inputs):
         training_target_audio = pipe.scheduler_audio.training_target(inputs["audio_input_latents"], audio_noise, timestep_audio)
 
     models = {name: getattr(pipe, name) for name in pipe.in_iteration_models}
+    if training_cfg_scale > 1.0:
+        if not inputs_nega:
+            raise ValueError(
+                "MiniMax-H3 CFG-aware training requires unconditional inputs. "
+                "When using split training, rebuild the data cache with the same "
+                "--training_cfg_scale value."
+            )
+        inputs_uncond = {**inputs, **inputs_nega}
+        inputs_uncond["use_gradient_checkpointing"] = False
+        inputs_uncond["use_gradient_checkpointing_offload"] = False
+        with torch.no_grad():
+            noise_pred_uncond, noise_pred_audio_uncond = pipe.model_fn(
+                **models, **inputs_uncond,
+                timestep_video=timestep_video, timestep_audio=timestep_audio,
+            )
+
     noise_pred, noise_pred_audio = pipe.model_fn(
         **models, **inputs,
         timestep_video=timestep_video, timestep_audio=timestep_audio,
     )
 
+    if training_cfg_scale > 1.0:
+        # The checkpoint's conditional prediction has CFG distilled into it.
+        # Rearrange the CFG equation to recover the raw velocity fitted to the
+        # standard flow-matching target, using the current model as the teacher.
+        noise_pred = (noise_pred + (training_cfg_scale - 1.0) * noise_pred_uncond) / training_cfg_scale
+        noise_pred_audio = (noise_pred_audio + (training_cfg_scale - 1.0) * noise_pred_audio_uncond) / training_cfg_scale
+
     loss = torch.nn.functional.mse_loss(noise_pred.float(), training_target.float())
     loss = loss * pipe.scheduler.training_weight(timestep_video)
     if "audio_input_latents" in inputs:

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/diffusion/training_module.py

@@ -359,7 +359,7 @@ class DiffusionTrainingModule(torch.nn.Module):
         # TODO: set `remove_unnecessary_params` to `True` by default
         remove_unnecessary_params=False,
         # TODO: move `loss_required_params` to `loss.py`
-        loss_required_params=("input_latents", "max_timestep_boundary", "min_timestep_boundary", "first_frame_latents", "video_latents", "audio_input_latents", "num_inference_steps"),
+        loss_required_params=("input_latents", "max_timestep_boundary", "min_timestep_boundary", "first_frame_latents", "video_latents", "audio_input_latents", "num_inference_steps", "cfg_scale"),
         force_remove_params_shared=tuple(),
         force_remove_params_posi=tuple(),
         force_remove_params_nega=tuple(),

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/pipelines/ace_step.py

@@ -70,7 +70,8 @@ class AceStepPipeline(BasePipeline):
         pipe.conditioner = model_pool.fetch_model("ace_step_conditioner")
         pipe.dit = model_pool.fetch_model("ace_step_dit")
         pipe.vae = model_pool.fetch_model("ace_step_vae")
-        pipe.vae.remove_weight_norm()
+        if pipe.vae is not None:
+            pipe.vae.remove_weight_norm()
         pipe.tokenizer_model = model_pool.fetch_model("ace_step_tokenizer")
 
         if text_tokenizer_config is not None:

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/pipelines/minimax_h3_audio_video.py

@@ -58,7 +58,21 @@ class MiniMaxH3Pipeline(BasePipeline):
         model_configs: list[ModelConfig] = [],
         processor_config: ModelConfig = ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/processor/"),
         vram_limit: float = None,
+        redirect_common_files: bool = True,
     ):
+        if redirect_common_files:
+            redirect_dict = {
+                "MiniMax/MiniMax-H3": "MiniMaxAI/MiniMax-H3",
+            }
+            for model_config in model_configs:
+                if model_config.require_downloading() and model_config.parse_download_source() == "huggingface":
+                    if model_config.model_id is not None and model_config.model_id in redirect_dict:
+                        print(f"The model is detected to be downloading from HuggingFace. {model_config.model_id} is redirected to {redirect_dict[model_config.model_id]}. You can use `redirect_common_files=False` to disable file redirection.")
+                        model_config.model_id = redirect_dict[model_config.model_id]
+            if processor_config is not None and processor_config.require_downloading() and processor_config.parse_download_source() == "huggingface":
+                if processor_config.model_id is not None and processor_config.model_id in redirect_dict:
+                    print(f"The model is detected to be downloading from HuggingFace. {processor_config.model_id} is redirected to {redirect_dict[processor_config.model_id]}. You can use `redirect_common_files=False` to disable file redirection.")
+                    processor_config.model_id = redirect_dict[processor_config.model_id]
         pipe = MiniMaxH3Pipeline(device=device, torch_dtype=torch_dtype)
         model_pool = pipe.download_and_load_models(model_configs, vram_limit)
         pipe.text_encoder = model_pool.fetch_model("minimax_h3_text_encoder")

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/pipelines/minimax_music3.py

@@ -45,7 +45,21 @@ class MiniMaxMusic3Pipeline(BasePipeline):
         model_configs: list[ModelConfig] = [],
         tokenizer_config: ModelConfig = None,
         vram_limit: float = None,
+        redirect_common_files: bool = True,
     ):
+        if redirect_common_files:
+            redirect_dict = {
+                "MiniMax/MiniMax-Music3": "MiniMaxAI/MiniMax-Music3",
+            }
+            for model_config in model_configs:
+                if model_config.require_downloading() and model_config.parse_download_source() == "huggingface":
+                    if model_config.model_id is not None and model_config.model_id in redirect_dict:
+                        print(f"The model is detected to be downloading from HuggingFace. {model_config.model_id} is redirected to {redirect_dict[model_config.model_id]}. You can use `redirect_common_files=False` to disable file redirection.")
+                        model_config.model_id = redirect_dict[model_config.model_id]
+            if tokenizer_config is not None and tokenizer_config.require_downloading() and tokenizer_config.parse_download_source() == "huggingface":
+                if tokenizer_config.model_id is not None and tokenizer_config.model_id in redirect_dict:
+                    print(f"The model is detected to be downloading from HuggingFace. {tokenizer_config.model_id} is redirected to {redirect_dict[tokenizer_config.model_id]}. You can use `redirect_common_files=False` to disable file redirection.")
+                    tokenizer_config.model_id = redirect_dict[tokenizer_config.model_id]
         pipe = MiniMaxMusic3Pipeline(device=device, torch_dtype=torch_dtype)
         model_pool = pipe.download_and_load_models(model_configs, vram_limit)
         pipe.text_encoder = model_pool.fetch_model("minimax_music3_text_encoder")
@@ -348,4 +362,4 @@ class MiniMaxMusic3Unit_Vocoder(PipelineUnit):
             waveform_chunks.append(waveform[...,
                                             left:waveform.shape[-1] - right])
         song = torch.cat(waveform_chunks, dim=-1)[0]
-        return {"audio": song.float().clamp(-1.0, 1.0)}
+        return {"audio": song.float().clamp(-1.0, 1.0).cpu()}

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth/version.py

@@ -1,5 +1,5 @@
 # Make sure to modify __release_datetime__ to release time when making official release.
-__version__ = '2.1.4'
+__version__ = '2.1.5'
 # default release datetime for branches under active development is set
 # to be a time far-far-away-into-the-future
 __release_datetime__ = '2099-10-13 08:56:12'

{diffsynth-2.1.4 → diffsynth-2.1.5}/diffsynth.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: diffsynth
-Version: 2.1.4
+Version: 2.1.5
 Summary: Enjoy the magic of Diffusion models!
 Author: ModelScope Team
 License-Expression: Apache-2.0
@@ -804,6 +804,9 @@ https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-44
 | FLUX.1 | [black-forest-labs/FLUX.1-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-dev) | [code](/examples/flux/model_inference/FLUX.1-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev.py) |
 | FLUX.1 | [black-forest-labs/FLUX.1-Krea-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Krea-dev) | [code](/examples/flux/model_inference/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Krea-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Krea-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Krea-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Krea-dev.py) |
 | FLUX.1 | [black-forest-labs/FLUX.1-Kontext-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Kontext-dev) | [code](/examples/flux/model_inference/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Kontext-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Kontext-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Kontext-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Kontext-dev.py) |
+| FLUX.1 | [black-forest-labs/FLUX.1-Fill-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Fill-dev) | [code](/examples/flux/model_inference/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Fill-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Fill-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Fill-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Fill-dev.py) |
+| FLUX.1 | [black-forest-labs/FLUX.1-Redux-dev](https://www.modelscope.cn/models/black-forest-labs/FLUX.1-Redux-dev) | [code](/examples/flux/model_inference/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_training/full/FLUX.1-Redux-dev.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-Redux-dev.py) | [code](/examples/flux/model_training/lora/FLUX.1-Redux-dev.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-Redux-dev.py) |
+| FLUX.1 | [HuanJue/Insert-Anything](https://www.modelscope.cn/models/HuanJue/Insert-Anything) | [code](/examples/flux/model_inference/Insert-Anything.py) | [code](/examples/flux/model_inference_low_vram/Insert-Anything.py) | - | - | [code](/examples/flux/model_training/lora/Insert-Anything.sh) | [code](/examples/flux/model_training/validate_lora/Insert-Anything.py) |
 | FLUX.1 | [alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta](https://www.modelscope.cn/models/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Inpainting-Beta.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Inpainting-Beta.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Inpainting-Beta.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Inpainting-Beta.py) |
 | FLUX.1 | [InstantX/FLUX.1-dev-Controlnet-Union-alpha](https://www.modelscope.cn/models/InstantX/FLUX.1-dev-Controlnet-Union-alpha) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Union-alpha.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Union-alpha.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Union-alpha.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Union-alpha.py) |
 | FLUX.1 | [jasperai/Flux.1-dev-Controlnet-Upscaler](https://www.modelscope.cn/models/jasperai/Flux.1-dev-Controlnet-Upscaler) | [code](/examples/flux/model_inference/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_inference_low_vram/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_training/full/FLUX.1-dev-Controlnet-Upscaler.sh) | [code](/examples/flux/model_training/validate_full/FLUX.1-dev-Controlnet-Upscaler.py) | [code](/examples/flux/model_training/lora/FLUX.1-dev-Controlnet-Upscaler.sh) | [code](/examples/flux/model_training/validate_lora/FLUX.1-dev-Controlnet-Upscaler.py) |