Defuser 0.0.22__tar.gz → 0.0.24__tar.gz

{defuser-0.0.22 → defuser-0.0.24}/Defuser.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: Defuser
-Version: 0.0.22
+Version: 0.0.24
 Summary: Model defuser helper for HF Transformers.
 Author-email: ModelCloud <qubitium@modelcloud.ai>
 License-Expression: Apache-2.0
@@ -92,7 +92,7 @@ Defuser currently supports the following `transformers>=5.3.0` `model_type` valu
 
 | Pattern | Supported model types                                                                                                                                                                                                               | Defused op performed ⚙️ |
 | --- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- |
-| Standard routed expert tensors 🧱 | `deepseek_v2`, `dots1`, `ernie4_5_moe`, `ernie4_5_vl_moe`, `exaone_moe`, `flex_olmo`, `glm4_moe_lite`, `glm4v_moe`, `hunyuan_v1_moe`, `jamba`, `laguna`, `lfm2_moe`, `minimax`, `minimax_m2`, `olmoe`, `qwen3_vl_moe`, `solar_open` | Splits fused expert tensors or registered expert buffers into numbered expert `nn.Linear` modules with per-expert `gate_proj`, `up_proj`, and `down_proj`. |
+| Standard routed expert tensors 🧱 | `deepseek_v2`, `dots1`, `ernie4_5_moe`, `ernie4_5_vl_moe`, `exaone_moe`, `flex_olmo`, `glm4_moe_lite`, `glm4v_moe`, `hunyuan_v1_moe`, `jamba`, `laguna`, `lfm2_moe`, `minimax`, `minimax_m2`, `olmoe`, `qwen3_vl_moe`, `solar_open`, `solar_open2` | Splits fused expert tensors or registered expert buffers into numbered expert `nn.Linear` modules with per-expert `gate_proj`, `up_proj`, and `down_proj`. |
 | Mixed sparse and shared experts | `deepseek_v3`, deepseek_v4`, `glm_moe_dsa`, `qwen3_5_moe`, `qwen3_5_moe_text`                                                                                                                                                       | Runtime expert tensor defusion for routed experts while preserving the model's shared-expert path. |
 | Transposed or packed expert tensors | `gpt_oss`, `phimoe`                                                                                                                                                                                                                 | Splits transposed fused expert `gate_up_proj` tensors into per-expert `gate_proj` + `up_proj`, preserves expert bias when present, and converts expert tensors into numbered expert `nn.Linear` modules. |
 | Flattened expert layout | `dbrx`                                                                                                                                                                                                                              | Rebuilds the flattened DBRX expert FFN weights into numbered expert `gate_proj`, `up_proj`, and `down_proj` `nn.Linear` modules. |

{defuser-0.0.22 → defuser-0.0.24}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: Defuser
-Version: 0.0.22
+Version: 0.0.24
 Summary: Model defuser helper for HF Transformers.
 Author-email: ModelCloud <qubitium@modelcloud.ai>
 License-Expression: Apache-2.0
@@ -92,7 +92,7 @@ Defuser currently supports the following `transformers>=5.3.0` `model_type` valu
 
 | Pattern | Supported model types                                                                                                                                                                                                               | Defused op performed ⚙️ |
 | --- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- |
-| Standard routed expert tensors 🧱 | `deepseek_v2`, `dots1`, `ernie4_5_moe`, `ernie4_5_vl_moe`, `exaone_moe`, `flex_olmo`, `glm4_moe_lite`, `glm4v_moe`, `hunyuan_v1_moe`, `jamba`, `laguna`, `lfm2_moe`, `minimax`, `minimax_m2`, `olmoe`, `qwen3_vl_moe`, `solar_open` | Splits fused expert tensors or registered expert buffers into numbered expert `nn.Linear` modules with per-expert `gate_proj`, `up_proj`, and `down_proj`. |
+| Standard routed expert tensors 🧱 | `deepseek_v2`, `dots1`, `ernie4_5_moe`, `ernie4_5_vl_moe`, `exaone_moe`, `flex_olmo`, `glm4_moe_lite`, `glm4v_moe`, `hunyuan_v1_moe`, `jamba`, `laguna`, `lfm2_moe`, `minimax`, `minimax_m2`, `olmoe`, `qwen3_vl_moe`, `solar_open`, `solar_open2` | Splits fused expert tensors or registered expert buffers into numbered expert `nn.Linear` modules with per-expert `gate_proj`, `up_proj`, and `down_proj`. |
 | Mixed sparse and shared experts | `deepseek_v3`, deepseek_v4`, `glm_moe_dsa`, `qwen3_5_moe`, `qwen3_5_moe_text`                                                                                                                                                       | Runtime expert tensor defusion for routed experts while preserving the model's shared-expert path. |
 | Transposed or packed expert tensors | `gpt_oss`, `phimoe`                                                                                                                                                                                                                 | Splits transposed fused expert `gate_up_proj` tensors into per-expert `gate_proj` + `up_proj`, preserves expert bias when present, and converts expert tensors into numbered expert `nn.Linear` modules. |
 | Flattened expert layout | `dbrx`                                                                                                                                                                                                                              | Rebuilds the flattened DBRX expert FFN weights into numbered expert `gate_proj`, `up_proj`, and `down_proj` `nn.Linear` modules. |

{defuser-0.0.22 → defuser-0.0.24}/README.md

@@ -66,7 +66,7 @@ Defuser currently supports the following `transformers>=5.3.0` `model_type` valu
 
 | Pattern | Supported model types                                                                                                                                                                                                               | Defused op performed ⚙️ |
 | --- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- |
-| Standard routed expert tensors 🧱 | `deepseek_v2`, `dots1`, `ernie4_5_moe`, `ernie4_5_vl_moe`, `exaone_moe`, `flex_olmo`, `glm4_moe_lite`, `glm4v_moe`, `hunyuan_v1_moe`, `jamba`, `laguna`, `lfm2_moe`, `minimax`, `minimax_m2`, `olmoe`, `qwen3_vl_moe`, `solar_open` | Splits fused expert tensors or registered expert buffers into numbered expert `nn.Linear` modules with per-expert `gate_proj`, `up_proj`, and `down_proj`. |
+| Standard routed expert tensors 🧱 | `deepseek_v2`, `dots1`, `ernie4_5_moe`, `ernie4_5_vl_moe`, `exaone_moe`, `flex_olmo`, `glm4_moe_lite`, `glm4v_moe`, `hunyuan_v1_moe`, `jamba`, `laguna`, `lfm2_moe`, `minimax`, `minimax_m2`, `olmoe`, `qwen3_vl_moe`, `solar_open`, `solar_open2` | Splits fused expert tensors or registered expert buffers into numbered expert `nn.Linear` modules with per-expert `gate_proj`, `up_proj`, and `down_proj`. |
 | Mixed sparse and shared experts | `deepseek_v3`, deepseek_v4`, `glm_moe_dsa`, `qwen3_5_moe`, `qwen3_5_moe_text`                                                                                                                                                       | Runtime expert tensor defusion for routed experts while preserving the model's shared-expert path. |
 | Transposed or packed expert tensors | `gpt_oss`, `phimoe`                                                                                                                                                                                                                 | Splits transposed fused expert `gate_up_proj` tensors into per-expert `gate_proj` + `up_proj`, preserves expert bias when present, and converts expert tensors into numbered expert `nn.Linear` modules. |
 | Flattened expert layout | `dbrx`                                                                                                                                                                                                                              | Rebuilds the flattened DBRX expert FFN weights into numbered expert `gate_proj`, `up_proj`, and `down_proj` `nn.Linear` modules. |

{defuser-0.0.22 → defuser-0.0.24}/defuser/model_registry.py

@@ -16,12 +16,21 @@ class PATCH(str, Enum):
 
 
 MODEL_CONFIG = {
+    "cohere2_moe": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
     "dbrx": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
     "deepseek_v2": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
+    "deepseek_ocr2": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
+    "inkling_mm_model": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
     "deepseek_v3": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
@@ -62,8 +71,12 @@ MODEL_CONFIG = {
         ],
         "checkpoint_mapping": [
             WeightRenaming(".block_sparse_moe.", ".mlp."),
-            WeightRenaming(r".experts.(\d+).w1.weight", r".experts.\1.gate_proj.weight"),
-            WeightRenaming(r".experts.(\d+).w2.weight", r".experts.\1.down_proj.weight"),
+            WeightRenaming(
+                r".experts.(\d+).w1.weight", r".experts.\1.gate_proj.weight"
+            ),
+            WeightRenaming(
+                r".experts.(\d+).w2.weight", r".experts.\1.down_proj.weight"
+            ),
             WeightRenaming(r".experts.(\d+).w3.weight", r".experts.\1.up_proj.weight"),
             WeightConverter(
                 source_patterns=".experts.gate_up_proj",
@@ -124,7 +137,7 @@ MODEL_CONFIG = {
             (
                 "transformers.models.qwen3_omni_moe.modeling_qwen3_omni_moe.Qwen3OmniMoeTalkerTextSparseMoeBlock",
                 "defuser.modeling.unfused_moe.qwen3_omni_moe.LinearQwen3OmniMoeTalkerTextSparseMoeBlock",
-            )
+            ),
         ],
     },
     "glm4_moe": {
@@ -168,6 +181,9 @@ MODEL_CONFIG = {
     "glm4v_moe": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
+    "glm4v_moe_text": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
     "glm_image": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
@@ -192,6 +208,9 @@ MODEL_CONFIG = {
     "hunyuan_v1_moe": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
+    "hy_v3": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
     "jamba": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
@@ -210,6 +229,15 @@ MODEL_CONFIG = {
             }
         ],
     },
+    "llama4_text": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+        PATCH.EXPERTS_DEFUSE: [
+            {
+                "module_class": "transformers.models.llama4.modeling_llama4.Llama4TextExperts",
+                "forward_impl": "batched_input",
+            }
+        ],
+    },
     "lfm2_moe": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
@@ -222,6 +250,9 @@ MODEL_CONFIG = {
     "minimax_m2": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
+    "minimax_m3_vl": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
     "nemotron_h": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
@@ -243,6 +274,9 @@ MODEL_CONFIG = {
     "solar_open": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },
+    "solar_open2": {
+        "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
+    },
     "zamba2": {
         "min_transformers_version": MIN_SUPPORTED_TRANSFORMERS_VERSION,
     },

{defuser-0.0.22 → defuser-0.0.24}/pyproject.toml

@@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "Defuser"
-version = "0.0.22"
+version = "0.0.24"
 description = "Model defuser helper for HF Transformers."
 readme = "README.md"
 requires-python = ">=3.9"

{defuser-0.0.22 → defuser-0.0.24}/tests/test_candidate_coverage.py

@@ -351,6 +351,22 @@ STANDARD_MOE_CASES = [
         "route_indices": [[0], [1], [2], [3], [0]],
         "expert_attrs": ("gate_proj", "up_proj", "down_proj"),
     },
+    {
+        "model_type": "solar_open2",
+        "module_path": "transformers.models.solar_open2.modeling_solar_open2",
+        "class_name": "SolarOpen2Experts",
+        "config_module": "transformers.models.solar_open2.configuration_solar_open2",
+        "config_name": "SolarOpen2Config",
+        "config_updates": {
+            "hidden_size": 64,
+            "moe_intermediate_size": 32,
+            "n_routed_experts": 4,
+            "hidden_act": "silu",
+        },
+        "input_dim": 64,
+        "route_indices": [[0], [1], [2], [3], [0]],
+        "expert_attrs": ("gate_proj", "up_proj", "down_proj"),
+    },
     {
         "model_type": "dbrx",
         "module_path": "transformers.models.dbrx.modeling_dbrx",
@@ -571,6 +587,7 @@ ALL_CANDIDATE_MODEL_TYPES = {
     "qwen3_omni_moe",
     "qwen3_vl_moe",
     "solar_open",
+    "solar_open2",
     "zamba2",
 }
 

{defuser-0.0.22 → defuser-0.0.24}/tests/test_meta_model_defusion.py

@@ -779,6 +779,17 @@ META_MODEL_CASES = [
         "target_class_paths": ("transformers.models.solar_open.modeling_solar_open.SolarOpenNaiveMoe",),
         "validator": "experts",
     },
+    {
+        "model_type": "solar_open2",
+        "mode": "convert",
+        "model_module": "transformers.models.solar_open2.modeling_solar_open2",
+        "model_class": "SolarOpen2ForCausalLM",
+        "config_module": "transformers.models.solar_open2.configuration_solar_open2",
+        "config_class": "SolarOpen2Config",
+        "target_class_paths": ("transformers.models.solar_open2.modeling_solar_open2.SolarOpen2Experts",),
+        "validator": "experts",
+        "min_targets": 2,
+    },
     {
         "model_type": "zamba2",
         "mode": "convert",