fusion-bench 0.2.24__py3-none-any.whl → 0.2.26__py3-none-any.whl

{fusion_bench-0.2.24.dist-info → fusion_bench-0.2.26.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fusion_bench
-Version: 0.2.24
+Version: 0.2.26
 Summary: A Comprehensive Benchmark of Deep Model Fusion
 Author-email: Anke Tang <tang.anke@foxmail.com>
 Project-URL: Repository, https://github.com/tanganke/fusion_bench
@@ -82,7 +82,13 @@ Model merging has emerged as a promising approach for multi-task learning (MTL),
 </details>
 
 <details>
-  <summary>Anke Tang, et al. Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging. Jan 2025. https://arxiv.org/pdf/2501.09522</summary>
+  <summary>Enneng Yang, et al. Continual Model Merging without Data: Dual Projections for Balancing Stability and Plasticity. NeurIPS 2025. https://github.com/EnnengYang/DOP</summary>
+
+Model merging integrates multiple expert models with diverse capabilities into a unified framework, facilitating collaborative learning. However, most existing methods assume simultaneous access to all models, which is often impractical in real-world scenarios where models are received sequentially. While some studies have investigated continual model merging (CMM)--which involves sequentially merging multiple models--the challenge of balancing prior knowledge (stability) and incorporating new tasks (plasticity) remains unresolved. This paper, for the first time, formally defines the stability and plasticity of CMM from the perspective of orthogonal projection. Subsequently, we analyze the relationships among the spaces spanned by task data, historical gradients, and accumulated gradients. Building on this, we propose a data-free Dual Orthogonal Projection (DOP) method, which eliminates data dependence and mitigates interference between the merged model and models for old and new tasks by projecting their parameter differences onto their respective approximate data spaces. Finally, to solve potential conflicts between stability and plasticity, we reformulate DOP as a multi-objective optimization problem and employ a multi-gradient descent algorithm to obtain a Pareto-optimal solution. Extensive experiments across multiple architectures and task configurations validate that our approach significantly outperforms state-of-the-art CMM methods.
+</details>
+
+<details>
+  <summary>Anke Tang, et al. Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging. NeurIPS 2025. Jan 2025. https://arxiv.org/pdf/2501.09522</summary>
 
 Deep model merging represents an emerging research direction that combines multiple fine-tuned models to harness their specialized capabilities across different tasks and domains. Current model merging techniques focus on merging all available models simultaneously, with weight interpolation-based methods being the predominant approaches. However, these conventional approaches are not well-suited for scenarios where models become available sequentially, and they often suffer from high memory requirements and potential interference between tasks. In this study, we propose a training-free projection-based continual merging method that processes models sequentially through orthogonal projections of weight matrices and adaptive scaling mechanisms. Our method operates by projecting new parameter updates onto subspaces orthogonal to existing merged parameter updates while using an adaptive scaling mechanism to maintain stable parameter distances, enabling efficient sequential integration of task-specific knowledge. Our approach maintains constant memory complexity to the number of models, minimizes interference between tasks through orthogonal projections, and retains the performance of previously merged models through adaptive task vector scaling. Extensive experiments on CLIP-ViT models demonstrate that our method achieves a 5-8% average accuracy improvement while maintaining robust performance in different task orderings.
 </details>

{fusion_bench-0.2.24.dist-info → fusion_bench-0.2.26.dist-info}/RECORD

@@ -1,4 +1,4 @@
-fusion_bench/__init__.py,sha256=Ha-mkRETS7qxHPdHHgu8bRA3kTvQ64P6tWnx5mGDDA4,2472
+fusion_bench/__init__.py,sha256=Rw9sT2ZegKMxZAG7FBDgqVOqBGlJ-43C5p_EarRHd1M,5816
 fusion_bench/__main__.py,sha256=weUjxpP3ULnDgUxCehdbmoCM9cqfkhDhGB85tAF5qoE,81
 fusion_bench/_get_started/__init__.py,sha256=Ht6OK6Luei2kdY9jRZzRQfzBlm3Yfm64BkXxpzeRg9Q,40
 fusion_bench/_get_started/greeting_program.py,sha256=wvVsPa7Djwx5Z5spAI6F9Kvv9KwfNkjIgJVH8oXR3Bo,1233
@@ -18,8 +18,8 @@ fusion_bench/constants/banner.py,sha256=fuIO36ETKlS6a3wbwZn-rA2OswSCfOYyyhZ0Fnal
 fusion_bench/constants/clip_vision.py,sha256=qOHlYZYSOqpOO4-cfwUUhbv7qyr5IuUAW3yWjqjbJBo,1430
 fusion_bench/constants/paths.py,sha256=1xLaZ2J3B3d0bo2ndubawaOjiFMJDAK6TjF685HlCM0,719
 fusion_bench/constants/runtime.py,sha256=UWhUwjfXgaHkcyxSqkkrcmrMVZ_HxR4VVfUz_ewnw4M,1838
-fusion_bench/dataset/__init__.py,sha256=OJiYmcqz0Vm5O7mE4PB5QFJeL_KjrsseQTRsQATGTm4,1050
-fusion_bench/dataset/clip_dataset.py,sha256=hLL7NyzOIt0gNT1kzjrexFISbj-B0KdlgtyGf6K8NjI,3143
+fusion_bench/dataset/__init__.py,sha256=2b4UGemg_F1I5cXkAzNMm12XmlP9-06DH8cW1V6ugwo,1495
+fusion_bench/dataset/clip_dataset.py,sha256=Y27odUQWiUOb-WdJnorhcxccDvzrHISxisxFbiRrQHs,3185
 fusion_bench/dataset/fer2013.py,sha256=Lub_xVhHfqaiPprvOsDVspJNioh1FjSrkhn3gL_UXDA,404
 fusion_bench/dataset/gpt2_glue.py,sha256=UvNWKAAMnKMNjF0BCpwwc7Nz0SI7KacxRR6SDm9Mn0s,8869
 fusion_bench/dataset/gsm8k.py,sha256=2OkDGDebZ295vkne2Ni4bhs6GbOIt4Vxx2F1315jsyk,2235
@@ -48,7 +48,7 @@ fusion_bench/dataset/llama/stanford_shp.py,sha256=6ueXKnFXIBBobacU1h5WxGLZrSOtBk
 fusion_bench/dataset/llama/ultrachat.py,sha256=Go7WvrDAYnm184fdazHGRYLbSY6Xd7jrESyQeUJtOww,1736
 fusion_bench/dataset/llama/wikitext.py,sha256=9ZHR-nMfXRumd3o-PIj3n7B83YlVeqpGkZ2zJs2B-9Y,2883
 fusion_bench/dataset/llama/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-fusion_bench/method/__init__.py,sha256=-d5WMlvY3kHYSUeompoG71T6fSttXPDjPf6X4TxNkqY,8986
+fusion_bench/method/__init__.py,sha256=-r9Sz5haSc5X4eNvxGvHwowQfS2sLfZ21orUC5ae7ws,9454
 fusion_bench/method/base_algorithm.py,sha256=OnKSNPQ_nIdIWxryyblW_sko7uoEBN4lGh-eLkJ4kh4,9004
 fusion_bench/method/dummy.py,sha256=hb1y6LR_geRZ5eRgGwt5zJUcHYorCeIbs5i76CvurUc,1031
 fusion_bench/method/ensemble.py,sha256=Bjzqxt-tUp5cawT1jIhqKswN5QH3bkYbmuI4LS4uTG0,3619
@@ -76,9 +76,10 @@ fusion_bench/method/bitdelta/bitdelta_utils/__init__.py,sha256=47DEQpj8HBSa-_TIm
 fusion_bench/method/bitdelta/bitdelta_utils/binary_gemm_kernel.py,sha256=zC0w5cwr-o8cE63kpBzHUA3S0FeJPX-Xf3mIS5ziIos,15546
 fusion_bench/method/bitdelta/bitdelta_utils/data.py,sha256=LGEgv8o8glyyLLYh6Ur5h_sulxPFmy6i-xi-Ap1G-Wc,1052
 fusion_bench/method/bitdelta/bitdelta_utils/diff.py,sha256=o3ib5sgGDYLgnL8YTfX0YDc4Md6W9_gb03jzftTn5s4,4075
-fusion_bench/method/classification/__init__.py,sha256=emB06UOMDHK5pfQ1WuvLG9Fm0aEEtZxSjpVw8fVE0fM,167
-fusion_bench/method/classification/clip_finetune.py,sha256=QNOw9O-BTOVOsW7lzRu8L-UfbiBpsT_8tS6i6BpbVyA,15726
+fusion_bench/method/classification/__init__.py,sha256=byVJ574JQ_DUvsDv8S6ZM6BKAv4ZZ964Ej4btm0aC7k,867
+fusion_bench/method/classification/clip_finetune.py,sha256=5q5Sr3eVVh8DfYdeSoGjwaKDksC8F2dY2r8Dl-wRaDg,15844
 fusion_bench/method/classification/continual_clip_finetune.py,sha256=OLhZKS-6aCnafevZkZYcNMKTWDDj3DATB27eZl_i8EY,11530
+fusion_bench/method/classification/image_classification_finetune.py,sha256=CPMpZvaULWaim01EvJJHlU4C6HQ16OCqZGoMvPBEWtY,8157
 fusion_bench/method/concrete_subspace/__init__.py,sha256=jJoFcjnQe-jvccsm9DuCXna378m9XBT9vV1fEZbdfR0,464
 fusion_bench/method/concrete_subspace/clip_concrete_adamerging.py,sha256=UkLOkaa_Dzlb4Q5ES69Y9GV1bodTnD7DzZFreykt65s,24706
 fusion_bench/method/concrete_subspace/clip_concrete_task_arithmetic.py,sha256=Nx-3AiAeIt5zmcC21Ta2_-4cAQg9hOWvThurXNZzA-w,10580
@@ -100,6 +101,10 @@ fusion_bench/method/doge_ta/__init__.py,sha256=dixO0i5fmhgC_W2_DAQ4PzYnkMCZX5D8t
 fusion_bench/method/doge_ta/clip_layer_wise_adamerging.py,sha256=4WPG2fhFw-u6oSoT-fBrP2K9YpX-MH-AotBL1DknfpA,1304
 fusion_bench/method/doge_ta/doge_ta.py,sha256=jrJF52JUBdrB3EGWaXJMFZE-v8syzZGr4smG6rEO74c,13790
 fusion_bench/method/doge_ta/layer_wise_adamerging.py,sha256=rLk3Nep5d6wMUNCp6q7pC7L0pfBvUwGBIuiGM7CQOf4,9780
+fusion_bench/method/dop/__init__.py,sha256=MD8c44ovLLJX_-v9t2SdLrvKLxVf8PijzFFNjJfvhpE,37
+fusion_bench/method/dop/dop.py,sha256=_wNjN1DSK27aKEyWVay61fqc7prwJ1uiv_3618_bQ20,14160
+fusion_bench/method/dop/min_norm_solvers.py,sha256=a7n2X0BE_YajlaUygyHV0yqW6-x5dTyZ5V0mt_Q69qE,8291
+fusion_bench/method/dop/utils.py,sha256=_q7yy3ENNFUh1qUd5J5DThRL4J1tIxEcknCO2AKmeYM,2102
 fusion_bench/method/expert_sparsity/__init__.py,sha256=nt7k5cKqA2Bax1aM93ODwsEuibZ_hdFgQsUos_8h2v8,271
 fusion_bench/method/expert_sparsity/mixtral/__init__.py,sha256=FyKDZIyYUnqvGIdJ5BS639UpzSBj11g28ATHs1Yczdk,545
 fusion_bench/method/expert_sparsity/mixtral/dynamic_skipping.py,sha256=zZa4IAKimFZMoxoQ_Oi7z2R9o5H6kxV2QTb0e-t9kDY,5665
@@ -157,7 +162,7 @@ fusion_bench/method/moe_pruner/utils/layerwrapper.py,sha256=6ahiuzw00qtbpmJg11Yq
 fusion_bench/method/moe_pruner/utils/prune.py,sha256=U0cX5RgyAezS7C4jnlfGwjZhMSLKhDvq3hZZGrzJVfM,10609
 fusion_bench/method/moe_pruner/utils/score.py,sha256=AVWOwsu6CGBHnO7S1JnJNqZVMMTfSj5QQNAPQXI59no,1177
 fusion_bench/method/opcm/__init__.py,sha256=0QcltOnjIYV1XEPDEagChLixLAhjiBnYwfWK00am29k,202
-fusion_bench/method/opcm/opcm.py,sha256=fIZtR8KZCUKTxo5URMZIVmDdb2Y6vugJaUvmg5tswdc,11655
+fusion_bench/method/opcm/opcm.py,sha256=m12JanlpfL4udUVhRnYt5RRchGdq1e8L91r1mNVNVqw,11733
 fusion_bench/method/opcm/task_arithmetic.py,sha256=YvtsWkjtnk7E3C4_xNr--uQWjQhoDZZB-klSx81_tGw,4824
 fusion_bench/method/opcm/ties_merging.py,sha256=-N3i7eMbhK95qyJsmmNMKNmPCkgGHGFa423a52cgi6g,6868
 fusion_bench/method/opcm/utils.py,sha256=_q7yy3ENNFUh1qUd5J5DThRL4J1tIxEcknCO2AKmeYM,2102
@@ -181,7 +186,7 @@ fusion_bench/method/pruning/wanda_utils/prune_opt.py,sha256=onfIRAF0yFi9b1GNDS9D
 fusion_bench/method/pruning/wanda_utils/sparsegpt.py,sha256=V1FEIGgSFbPT5YPrYXCWhz1lLXaor6RwfNund7EEIWM,5434
 fusion_bench/method/pwe_moe/__init__.py,sha256=gZUhbqCtCeVSip3nyt5rNSrEDqtByl2ILcWrD4Z3jx4,124
 fusion_bench/method/pwe_moe/clip_pwe_moe.py,sha256=aIKWwVRIvFm0_-6nTUMBuJA5ptUSFDBpcujvOOwKVD0,10253
-fusion_bench/method/pwe_moe/module.py,sha256=mQbVbE6y-Q2zxifF1_k13UGGx_I725V9aUkRumjIDHI,12251
+fusion_bench/method/pwe_moe/module.py,sha256=KJ1biDPPLYq5eOaF-XGtW_ZcaB8G7vs9ZMo1Hig6L-4,12209
 fusion_bench/method/pwe_moe/openclip_pwe_moe.py,sha256=xhQsFt8FwK_obd3u3FQsBpH1o5XaLCsHohjWOEd7lJc,18354
 fusion_bench/method/pwe_moe/utils.py,sha256=K9BeVMrhYv7GNlJO76eoQbkI1dOO7XF18yK06WUh9ZA,1336
 fusion_bench/method/pwe_moe/phn/__init__.py,sha256=PXX-hb_bd7GdtLHcAcnGGsW_Wbg8g2YlRZMTCk3axUw,78
@@ -220,7 +225,7 @@ fusion_bench/method/sparselo/sparselo.py,sha256=U3eIjLcz484Tq7kbQry_U7YFiTx3ECOe
 fusion_bench/method/surgery/__init__.py,sha256=6sRKWeL8cx6Jy2aC9tRL78irNTJnp9w75K2dAxBxhho,88
 fusion_bench/method/surgery/clip_layer_wise_adamerging_surgery.py,sha256=Cc8LiAqkQzJwQJsyuazG5wgq6kghVcyL8rGkhPcBVoU,5936
 fusion_bench/method/tall_mask/__init__.py,sha256=XINPP8PqGQ01he9p2RyHaKGyrcYoJuYwIzvwkrr0ILY,61
-fusion_bench/method/tall_mask/task_arithmetic.py,sha256=c-5ehKV_t46ljvKTBDr-eA3-FbSD_UNXlza4cOqK5aI,4371
+fusion_bench/method/tall_mask/task_arithmetic.py,sha256=RX_JgEPwG52EPYGXWYGuq0LBeyJHMbVZn7Qy_4QmSsQ,4373
 fusion_bench/method/tall_mask/utils.py,sha256=Wlp8WcPwR_lCaBIZ9rgG6ewLfSzz3G7kPk9yj13pvls,8817
 fusion_bench/method/task_arithmetic/__init__.py,sha256=pSx_NV5Ra_6UXpyYWCi6ANQoAnEtymZt_X1dDN9wT4Y,96
 fusion_bench/method/task_arithmetic/task_arithmetic.py,sha256=KsSBshf04MUwIjoc0HAAmY6cWMqjZwZOYXbUuU4EaL0,6320
@@ -261,28 +266,30 @@ fusion_bench/metrics/text_to_image_generation/__init__.py,sha256=OEIxpKmyy6-3iWy
 fusion_bench/metrics/text_to_image_generation/aesthetic_scorer.py,sha256=-ZaD84ENPITh_K0Fe9OKYYoiGnPhlSE9gTbBqrtnqqA,4487
 fusion_bench/metrics/text_to_image_generation/compressibility.py,sha256=x4dNTFnAN4naChBDZBO-jUghnHAyobRVOupctKYRg1w,1656
 fusion_bench/metrics/text_to_image_generation/pickscore_scorer.py,sha256=aSWzl8k7z80Cirg5qdfkPsp3sMFEv_PjA1NJv3PPWXY,3115
-fusion_bench/mixins/__init__.py,sha256=yjRvcB9Mn-c0g8tXmoBf2Dn8gyc-Na6dyhc4r674asM,1213
+fusion_bench/mixins/__init__.py,sha256=2_mAT0VHiUYGyWJyiDSxcFmI4Qt64Y2qlNu1Z11fgyY,1320
 fusion_bench/mixins/clip_classification.py,sha256=8dqJuI3AVetFZKuzTp1SR2kGQ-vGvfbcmwfnzuUiwfI,10096
 fusion_bench/mixins/fabric_training.py,sha256=ZmycEhCaNCgVi5oM9m0q6msxgk3quowmFvDAcvskFrg,13017
 fusion_bench/mixins/hydra_config.py,sha256=rfT-XPUKV_U3nvuTVsKLmSmEiieoSIsbhxE5_-E0er0,5508
 fusion_bench/mixins/lightning_fabric.py,sha256=5iamAL7YV6lEm_-8NuzFjfIy1vslwKthSpCSWLLhlCM,7506
 fusion_bench/mixins/openclip_classification.py,sha256=O45HzgLXNvlQr5RVpfIGsYdIQ0tY5g_68KB0MTqsZWU,290
+fusion_bench/mixins/pyinstrument.py,sha256=I8CLVRUK6G_U8S5x-netmtAcy6m9uLB0UGB1AokbheU,5108
 fusion_bench/mixins/rich_live.py,sha256=j7wNgrgwfdpl6nCXZGF_2DLtNq2aqCb_52Qhe9QSltc,495
 fusion_bench/mixins/serialization.py,sha256=z73Mmq952TIdPwwZ8cRdl3n0_uc9lqylFI9fxKesREs,13260
-fusion_bench/mixins/simple_profiler.py,sha256=czWMl6p9PoxbQ5A8Uifwleaq5QPGEn0qMc8MXu9dSZM,2200
+fusion_bench/mixins/simple_profiler.py,sha256=QA4fZhD-uL06fZaoqBQowI0c_qrAUhWszFteyznFfUw,5391
 fusion_bench/mixins/optim/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 fusion_bench/mixins/optim/adamw_with_warmup.py,sha256=qTnRl8GVVIfaplOFBHnJFuZUbxPZRWRGHGNzm_EDhDE,1421
 fusion_bench/modelpool/PeftModelForSeq2SeqLM.py,sha256=rxPKTTWno3KAcTTEfydPpXx1b0EJa8PLbqrberweFF8,2108
-fusion_bench/modelpool/__init__.py,sha256=WFDdiwPqdkzVsDYguWPPGGX_ZTRZhUCK8WMuhZpjKCg,1512
-fusion_bench/modelpool/base_pool.py,sha256=u2ahVkurq60yH86LMw1Rw98cpcgDTRiVeICpm9jewNI,9785
+fusion_bench/modelpool/__init__.py,sha256=wKAkEgit_1ZtDAOKOntzrUKdCjOFIxnPMYN02B970Wg,1671
+fusion_bench/modelpool/base_pool.py,sha256=5snzTmqn1Xs_dy0Ws5QWxs9uCAXMwIuclrwfikKPh9o,12298
 fusion_bench/modelpool/huggingface_automodel.py,sha256=OJ6EyYyjNv1_Bhjn-zli-e__BJ0xVa4Fx9lhXVb-DJo,552
 fusion_bench/modelpool/huggingface_gpt2_classification.py,sha256=j8nicVwtoLXY4RPE2dcepeEB3agBKkkH-xA3yMj1czw,2014
 fusion_bench/modelpool/lazy_state_dict_pool.py,sha256=HtEA85rqSCHfsIddI5sKDcZf5kSuHNwrb8fF1TUSTr0,652
 fusion_bench/modelpool/nyuv2_modelpool.py,sha256=btuXmYxwfjI6MnGakhoOf53Iyb9fxYH20CavGTrTcnA,1375
+fusion_bench/modelpool/resnet_for_image_classification.py,sha256=G72gRG6LzVWZcf_AvH5TKDVB2zyRrVG3RUe3WlU9_wE,7398
 fusion_bench/modelpool/causal_lm/__init__.py,sha256=F432-aDIgAbUITj4GNZS9dgUKKhaDMCbTeHB-9MecaQ,99
 fusion_bench/modelpool/causal_lm/causal_lm.py,sha256=gpUQqxZIuKoaQ-gvdPsLVxI7UifueR6k3YzbUV1i0lk,19902
 fusion_bench/modelpool/clip_vision/__init__.py,sha256=3b9gN2bWUsoA1EmpitnIMnIlX7nklxbkn4WJ0QJtS2c,43
-fusion_bench/modelpool/clip_vision/modelpool.py,sha256=e5t9olRMOj_SyGVy-gqn7RwC5FAqxNsJDongWIv2KFY,7108
+fusion_bench/modelpool/clip_vision/modelpool.py,sha256=ENQfAAwQ3NFEyDv0C313HA0h5yF6QyvT0_IOe9cDQ40,9250
 fusion_bench/modelpool/openclip_vision/__init__.py,sha256=QDmAitKqUwRygN9QncdS_kGWZdfTKL4uUifC8xh9c10,47
 fusion_bench/modelpool/openclip_vision/modelpool.py,sha256=2MieB4PMvg85DaiYu49m3BzuBjib1xozJHTpYyHhRTs,11102
 fusion_bench/modelpool/seq2seq_lm/__init__.py,sha256=FnfSMHcwNHDQEMdB2HdK4WphQ6MufsRLUkczuALjM4Q,57
@@ -290,7 +297,7 @@ fusion_bench/modelpool/seq2seq_lm/modelpool.py,sha256=yfa_B5TUIkuC1fTn4xD3HHnFPd
 fusion_bench/modelpool/seq_classification_lm/__init__.py,sha256=_VB9nlR_gm6IEXNMsNR3VnzFiCpxNGuAGF39rZ9DpBA,129
 fusion_bench/modelpool/seq_classification_lm/reward_model.py,sha256=NKf-eoei1GdU4ojKSpN5_kQwax4uUEStnlKyh8qOrNg,540
 fusion_bench/modelpool/seq_classification_lm/seq_classification_lm.py,sha256=t9wXHFwa7V2XC3ajxt4_bSsxMTDKW4nebvdxhG7VeLM,3435
-fusion_bench/models/__init__.py,sha256=LeLQw2Yphu4QKZxjws_7MCM50XvFP1rTrvJ_2SR5zIA,271
+fusion_bench/models/__init__.py,sha256=TURxx0Hnv3LBz2VFN36Y6ZfIOxvAGbKro5zhn6rtwP4,893
 fusion_bench/models/hf_clip.py,sha256=lL4LxbdwC_rDWRozdEJmRlzKaNcQMpWwCSMDE0tfZRM,7525
 fusion_bench/models/hf_utils.py,sha256=bfB3QAUqsG-TyUeOWrZt8V7GeWDhp-fKg3P0J3D_TbQ,5497
 fusion_bench/models/parameter_dict.py,sha256=HCkTJCz23pYN1_Hhegx8gglOtrnzVKJPMeg9_rUhe18,3630
@@ -369,16 +376,18 @@ fusion_bench/models/wrappers/ensemble.py,sha256=T-DAKrAm-ciZwV6Hbt8uASbjtoQpHTlv
 fusion_bench/models/wrappers/layer_wise_fusion.py,sha256=A7LjG0inL5oeEVOkJwEUDM15v4dpQnsCq2y9zA78R3k,11198
 fusion_bench/models/wrappers/layer_wise_fusion_doge_ta.py,sha256=q5Hc4BtLpAawMbxsWJRL-8OR-x7994Jhr9IyN7vKZ9o,16930
 fusion_bench/models/wrappers/task_wise_fusion.py,sha256=ROLANdDq0bZ3sIROqIv3udPN8lzDdEwxD0Jonx-5ycw,17465
-fusion_bench/optim/__init__.py,sha256=lemrcuiA6OLjQkpYm-RP-Ox2MgjngN1ywvCo0NgShlM,61
+fusion_bench/optim/__init__.py,sha256=JS7J2VjrM2LdkiFCxuQnIuFwBsWiPyFb7QuEU6V2bPY,845
 fusion_bench/optim/exception.py,sha256=fMgo1heiqfGhuI5RIbf30BwWSShn5RQiyeb30QtfTI0,1607
 fusion_bench/optim/mezo.py,sha256=Vm4vMGh10Fhe28_9L1MK8r_U7DrurA8Liprh2_gn4_U,3646
-fusion_bench/optim/lr_scheduler/__init__.py,sha256=W7CsdW4XKqXbNfzjvv2wmrvNWwfH_sQ-wiBViRPlP3U,29
+fusion_bench/optim/muon.py,sha256=ABrX-05EeIw-JPWaYqVKU9-ary8dhaKhLOykePF7Cxw,14178
+fusion_bench/optim/lr_scheduler/__init__.py,sha256=yhRN3ZylT5rx3b8-_KYt_aOBIlrscpa4eh-K5H8B-WY,613
 fusion_bench/optim/lr_scheduler/linear_warmup.py,sha256=Dvy_TCUuAQHlbDF2jo2_502Ae4JWXGrtZL3gwA_H6ZI,6566
 fusion_bench/optim/lr_scheduler/utils/__init__.py,sha256=GfZk9VYL3cFE1Qy2xQpGc1GCgnjySk5-D7EVRZ-C05Q,29
 fusion_bench/optim/lr_scheduler/utils/visualization.py,sha256=Ea1n9ElNizAe0iUnjynyfteuZunv2-UBMN_NfEU2imA,3490
-fusion_bench/programs/__init__.py,sha256=oGoRp2TMI6ELxyfkeTg2h27hZJEDz9x31AsmvwvNvJw,508
+fusion_bench/programs/__init__.py,sha256=YFlvpDC6y2Vm66VSlHKD1vu5nRDQRYNR_Nkn_61xqiI,605
 fusion_bench/programs/base_program.py,sha256=Bl_bv8SawEUc-GBTtZFMoii0y-r-0hOXBAJkQFexWCU,3475
-fusion_bench/programs/fabric_fusion_program.py,sha256=jt0_tlg37a2jBl2YikaC0N71Gmr4J340wkKAekyT180,12453
+fusion_bench/programs/fabric_fusion_program.py,sha256=wIHNpLUw6uAXpAasJRAMWut55hF_EGFShxn70zRRvfk,12449
+fusion_bench/programs/fusion_program.py,sha256=qLyA3FHJUMM1L3mlYn4jlnZzv9OKguWM5aGGIoLts2I,11309
 fusion_bench/scripts/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 fusion_bench/scripts/cli.py,sha256=kEWLEkZEBqUr1_-XTePzNC5NM8lwWvgUBf0Lcuk_FI8,2739
 fusion_bench/scripts/imgui.py,sha256=r9Glbfbwu3JCsX9TKQFwcHarvwA_G7ff0jWBUPW1S1U,7613
@@ -408,7 +417,7 @@ fusion_bench/taskpool/openclip_vision/openclip_taskpool.py,sha256=PtD_Y9CWzPI3WE
 fusion_bench/tasks/__init__.py,sha256=Z_ePIp4Xizkj78QopLg1dZkJAN_IF73MkbR_nkfHQ9Y,52
 fusion_bench/tasks/base_task.py,sha256=Fg_pdZhld-2KPKX0C1WrxaTz0EYWrvJerAHO-hA03GI,412
 fusion_bench/tasks/classification.py,sha256=i5sXVr2twzxzvRx4U4EmbJIYg-QbM-Kll5Ol0rsf4XU,2253
-fusion_bench/tasks/clip_classification/__init__.py,sha256=KiQlO2pqNdEOQUDO92lyHax8pZBFK6z0I5iClh6q9I0,8551
+fusion_bench/tasks/clip_classification/__init__.py,sha256=EltfO9NCVKPmvayop2fZtjPtGMM6Eajz5wGuPkVFRzE,8970
 fusion_bench/tasks/clip_classification/cifar10.py,sha256=Slx-A7I3CiEg0c4pIgy4OkoMD8MAHL8qCRtSQhYoYpE,968
 fusion_bench/tasks/clip_classification/cifar100.py,sha256=mzmdMxTuuAczJR10DX5FarJl3QRfgVee6cLJ3_faO1U,2750
 fusion_bench/tasks/clip_classification/clip_dataset.py,sha256=KVREQCAxAFnMeFgZzQbWaeLGhTB-XV4p1EuSsam3rMU,58
@@ -441,7 +450,7 @@ fusion_bench/tasks/flan_t5_text_generation/glue_evaluation.py,sha256=-B1wqVGp3wZ
 fusion_bench/tasks/flan_t5_text_generation/glue_load_dataset.py,sha256=sVihXHbqwi8IlDpiIxzvmDv-Ob7WKvi23GIRYbBUKOc,1833
 fusion_bench/tasks/flan_t5_text_generation/glue_preprocessors.py,sha256=GhRmGmcJGF4oVgZQarsBtx8GNKrNEZUkrillNz3iBuY,13183
 fusion_bench/tasks/flan_t5_text_generation/glue_prompt_templates.py,sha256=mKMTXIr5o-BqS_Hvv1bbMvvjQLLeKNVw7BKS9qgQ8Dw,1890
-fusion_bench/utils/__init__.py,sha256=wNAfpP-u_-8HGbLaBoHT_wriU_cNvY4M_UXdBv2kXhc,695
+fusion_bench/utils/__init__.py,sha256=b61bfpNY2FOm3QWdexEOMMv1Tcp8zz2pR6644r18RSM,4778
 fusion_bench/utils/auto.py,sha256=uACQLE62_kNyhl4BGduvcbyeTE61qXpIJx3Ccl8kh68,920
 fusion_bench/utils/cache_utils.py,sha256=-bTZijQgl4BuAx0VSJFD-bSDOXuq3o0NkrOaiLiyofU,4795
 fusion_bench/utils/data.py,sha256=aalB3kGbZUF-PZ_IaAhcXanRKhS-RNMT5mUrEBb4R3E,6722
@@ -454,17 +463,17 @@ fusion_bench/utils/functools.py,sha256=7_tYJ2WD88_2DDuOOj5aZz3cYuslYH5tsVyIgCeLt
 fusion_bench/utils/hydra_utils.py,sha256=TklUDKDEZlg4keI-TEZiqh4gFjr9-61Rt1RMlqkoSGk,1174
 fusion_bench/utils/instantiate_utils.py,sha256=OXkfhq_o3Sgy5n3Psf-HI-dIfbK9oD2GBdfcx3gT63Q,17526
 fusion_bench/utils/json.py,sha256=sVCqbm9mmyHybiui-O57KFt_ULrjLtN2wipSo6VDvqE,2533
-fusion_bench/utils/lazy_imports.py,sha256=dg4Uu8FaoEu0WGVTo5o_PbLZs3Ei_RG75Ta-Us1iPW4,3500
-fusion_bench/utils/lazy_state_dict.py,sha256=srEKyctbuBW3yrVFSG7Tki_XkBwoc6eUmDXLxHXqX0o,20328
-fusion_bench/utils/misc.py,sha256=93q0m-HYWkPK91Co5lll_J0Dxs6YahW2lD_X8fUAyTk,2420
+fusion_bench/utils/lazy_imports.py,sha256=s-1ABhPyyHs7gW4aodCzu3NySzILzTL7kVNZ0DZRXJA,6156
+fusion_bench/utils/lazy_state_dict.py,sha256=mJaiAtKB1vlNUAoQILnnCmU80FGJ8MSwmdPpmdhOyDE,22206
+fusion_bench/utils/misc.py,sha256=_7BaS9dNKyySGU0qmTmE0Tk8WK82TEm7IBJxVRkuEAw,5315
 fusion_bench/utils/modelscope.py,sha256=P8fV6Eff8oP0LVGIFGbLvuk8MBteysN438djZ6ZEfE4,10699
-fusion_bench/utils/packages.py,sha256=wKl-qtPjA61LrdgTTusuNyvs8jfUv4mA5IwPTFWyYtA,2139
+fusion_bench/utils/packages.py,sha256=m2E0ryIMI0NwWR9vUHkK9FtZEwA1G-A4dYOf87olli4,2217
 fusion_bench/utils/parameters.py,sha256=ufEDOYJwcQQxLfveK8hBAGwpu5J3LA_cTWiDgZ2zkJ0,11788
-fusion_bench/utils/path.py,sha256=qrfgar3b-6_2v032-2hTt97L6qdtG7zc3CFrGFyKSGE,2400
-fusion_bench/utils/pylogger.py,sha256=r2KXTvq-j8uHdjBBoVPOgkjv4c6pyhbX6xf1JbOsF4w,3335
-fusion_bench/utils/rich_utils.py,sha256=24RF-OHK6h9ggZ95csw_vMU8YtxYNOxlzjcH7dpuESY,5863
+fusion_bench/utils/path.py,sha256=piznok_znXkTY71VBwJrxBlXureYOdQnMfvqaZ26qvc,2643
+fusion_bench/utils/pylogger.py,sha256=1Uy_LkHkbrYdt1g5Ge_eAh2YoCJwn3U3Ndouz9sVA6g,3419
+fusion_bench/utils/rich_utils.py,sha256=3Z0di-1IOs3QoovF2frNA28ITVKWBLdm84zbXdTrM28,5924
 fusion_bench/utils/set.py,sha256=_43ZvGKJ_BK9sUslsSNhi7xEfuAQuyj3vViImnGpnCY,134
-fusion_bench/utils/state_dict_arithmetic.py,sha256=fczHDEpL2_UmxNIdvQtllXvBWBcmKpw-p6CIS_upjwI,11818
+fusion_bench/utils/state_dict_arithmetic.py,sha256=kETybQPAcmupmTuKYKFThBDU2WBlhFPdj_Qzv500cFg,38385
 fusion_bench/utils/tensorboard.py,sha256=9fkgNYR9LM38nPNkudcxL9TjLUseW-280M0k2nLff7o,1669
 fusion_bench/utils/timer.py,sha256=adBpA_XjpCuVvL6uyCtKhAFRzk4SXsr8T8P5kQNz0x8,5012
 fusion_bench/utils/type.py,sha256=2iu8PQzSzI2KopYwg4Pay7qpq7s_LKkl6Rhj-tjG3u0,630
@@ -475,13 +484,14 @@ fusion_bench/utils/plot/token_notebook.py,sha256=bsntXf46Zz_RavTxNiB9c3-KvHw7LFw
 fusion_bench/utils/strenum/__init__.py,sha256=id9ORi1uXrDxhbmVxitJ1KDwLS4H3AAwFpaK5h1cQzw,8531
 fusion_bench/utils/strenum/_name_mangler.py,sha256=o11M5-bURW2RBvRTYXFQIPNeqLzburdoWLIqk8X3ydw,3397
 fusion_bench/utils/strenum/_version.py,sha256=6JQRo9LcvODbCOeVFYQb9HNJ_J9XiG_Zbn8ws2A3BV8,18466
-fusion_bench-0.2.24.dist-info/licenses/LICENSE,sha256=nhnOJlw4CPuPVE0qvkGmxfFgHmKi-6nzXvTu8t0NUdg,1066
+fusion_bench-0.2.26.dist-info/licenses/LICENSE,sha256=nhnOJlw4CPuPVE0qvkGmxfFgHmKi-6nzXvTu8t0NUdg,1066
 fusion_bench_config/README.md,sha256=Lc8YSBJ5oxf9KV5kKDivJ9LRyGuraGQPmBbgbdVA-j4,703
 fusion_bench_config/clip-vit-base-patch32_robustness_corrupted.yaml,sha256=7IxLQoLRz-sRWyV8Vqc5kQcmYE_9YQz2_77pmvAkum8,1207
 fusion_bench_config/fabric_model_fusion.yaml,sha256=U8BxsaOvsg9bsEZcIpBE-feo9n9G7Y1kQDHqPVxUYAg,2601
 fusion_bench_config/llama_full_finetune.yaml,sha256=wmtslON9MTEp8L9Y6Wz3adqsZq_IFU1y6dCcxuikoEU,787
 fusion_bench_config/llama_magnitude_pruning.yaml,sha256=xFyDJpb8gyIjosteOpEW9eayONWhl0B763r1XmO-9w8,633
 fusion_bench_config/llama_model_fusion.yaml,sha256=KMMDFPAiiOU1vIMWw58FoMhi8-_SDImF4eqlg9ZoprY,586
+fusion_bench_config/model_fusion.yaml,sha256=hODoFHcSl4hla0X8lt2oXwUKwYS4V6aw-Sxhyv6j70M,2467
 fusion_bench_config/nyuv2_config.yaml,sha256=VtiqcyNwTxsiv8FFxdSBiUp0Qqtxig0j2bSZ8Faf4xA,540
 fusion_bench_config/nyuv2_mtl_train.yaml,sha256=VpQsJ9oheIlcbfU_vdmIVXJEESKG7GuftSmmoDptstE,609
 fusion_bench_config/_get_started/clip_evaluate_single_model.yaml,sha256=Bh448Jd_6OlldG6jo9LYZrx0U-xLZXtB8I6yxnFHM_I,630
@@ -593,14 +603,14 @@ fusion_bench_config/hydra/default.yaml,sha256=Fpi3pV1hqPoPk5QdBncse6NlNOAl2YHzD4
 fusion_bench_config/hydra/help/fusion_bench_help.yaml,sha256=v8s891Cr5wyxBXGDn_VBBwwRmb0JXOL874Sl-zNoCWA,1880
 fusion_bench_config/hydra/job_logging/rich_logging.yaml,sha256=_dYGeFTCqaPrRowLXBNMXwzYhw8ns1TkQFfALwK1aCw,441
 fusion_bench_config/method/clip_finetune.yaml,sha256=yWjcdKYaKvy53sGaygg2ElAjb9-YFCyCGE1s9aB_dPM,677
-fusion_bench_config/method/depth_upscaling.yaml,sha256=m2XUK8Znf8nnaPKMNH7Un19DQXJlFwpaHE02UId1nxY,632
-fusion_bench_config/method/dummy.yaml,sha256=5qs6OuIfriKOH7FgqvcMXMUoRLmXDZmjA4irpAsc5xo,45
+fusion_bench_config/method/depth_upscaling.yaml,sha256=86YqczaMzZftymLy_k2cb-GMy4C42yTxxP4c4htZTBs,1230
+fusion_bench_config/method/dummy.yaml,sha256=Pw2w6WQiw3K4_KH0THPs4NSM7lZoZLsNbB72iPSVsl8,427
 fusion_bench_config/method/mixtral_moe_merging.yaml,sha256=AdVhXD6Crw-B3QyNpP4ToHRSg-EeSCIGtazA7lQvPOU,148
 fusion_bench_config/method/mixtral_moe_upscaling.yaml,sha256=wYDRnWOpZ6SgvL2Fm9wIDomrN2x5Jaq5vg1hjh3druk,210
-fusion_bench_config/method/model_recombination.yaml,sha256=RGb4boklWcN_GnI-bB5o_qr5o5vGgfIfi_EnTqQ_qcA,195
-fusion_bench_config/method/simple_average.yaml,sha256=GtMNvt0-qWOevRX2V6fjiYUO2BwDvMw-EcxRMS_PhZQ,53
-fusion_bench_config/method/task_arithmetic.yaml,sha256=hqkbc8kbzEFPFmBIKbf-6-vT2ZsBYxhhlP7ZmNT13PM,74
-fusion_bench_config/method/ties_merging.yaml,sha256=0lsy-q-9SNY5xzfoAOFpeva2AqdwcbLwMxb0ZtTU2PA,292
+fusion_bench_config/method/model_recombination.yaml,sha256=DeyVPdDCL-eyJDlPZXLAIWfKi3p8nN0dLFRx5ydsERc,740
+fusion_bench_config/method/simple_average.yaml,sha256=uB51mNlFKb9S3Go1p6SLGgr3PWJFZs97Ccn1zZZkEug,577
+fusion_bench_config/method/task_arithmetic.yaml,sha256=zQmNpnQrZTHiRv_KmYnHPMScKf8MUMLbQYh9254_1Jg,580
+fusion_bench_config/method/ties_merging.yaml,sha256=c3BjnFo-ZU5hmCrfi-1VQPhd_EYGtftxxYDHTVCMy6s,501
 fusion_bench_config/method/ada_svd/clip_vision.yaml,sha256=3l0VKCL66rZNx020UKhf_UzXScZ5XZYOUeNm8mqo0So,183
 fusion_bench_config/method/adamerging/clip.yaml,sha256=NBJaK0a4RxV3D2LRciUeWmTqabRwu6OxZnT7u7iz6ug,753
 fusion_bench_config/method/adamerging/layer_wise_flan_t5.yaml,sha256=DxkZhcuu_-ErIUqBUmWKN5UXYYWKoKPX6IgjV-Txwv0,541
@@ -608,9 +618,11 @@ fusion_bench_config/method/adamerging/layer_wise_gpt2.yaml,sha256=bLz6zc5CofeUO2
 fusion_bench_config/method/adamerging/llama_sft.yaml,sha256=khKzfhvQ5oxBMH0d-YvyjN-qIgQNeevDodXngS5g9KY,1022
 fusion_bench_config/method/analysis/task_vector_cos_similarity.yaml,sha256=hxVA4deUr1go1RZl12qD8PekwydWJ9SBQowSqmo3A8I,139
 fusion_bench_config/method/analysis/task_vector_violin_plot.yaml,sha256=FmBGj0Ib2xYd-49x_xZSeVbExwL-A9-tHhHTMBrT_Fg,134
-fusion_bench_config/method/bitdelta/bitdelta.yaml,sha256=b92xQpufqrSHAiU0QFE8g0nQ7RGSowOubGrEz_KugsQ,231
+fusion_bench_config/method/bitdelta/bitdelta.yaml,sha256=uuR5x1IVTWyZjTSd5i1JXd_D8tG7tWBfOpgMBDCBgR0,436
 fusion_bench_config/method/classification/clip_continual_finetune.yaml,sha256=Ls63kdLb1bLwUEqzfyTtJcpFOdv3HmwzBML0V2JnnAs,791
 fusion_bench_config/method/classification/clip_finetune.yaml,sha256=yWjcdKYaKvy53sGaygg2ElAjb9-YFCyCGE1s9aB_dPM,677
+fusion_bench_config/method/classification/image_classification_finetune.yaml,sha256=fl60RFCYwmrwwu3QlaJTFiBLmSmnjHxl-xyq4Gb80iU,401
+fusion_bench_config/method/classification/image_classification_finetune_test.yaml,sha256=IxUbjeTSvpPZpZsRhOMlmrCALgWOSZjgeUjo1M41aCg,175
 fusion_bench_config/method/concrete_subspace/clip_concrete_layer_wise_adamerging.yaml,sha256=r0zR1WenY1fYba6mEBAoHJZKcx1x7L2cQmEA_54NTYM,739
 fusion_bench_config/method/concrete_subspace/clip_concrete_task_arithmetic.yaml,sha256=eNoqcY1iMbs0Y5kKi_ya3rmQQMHqU7ht3EU7G_xmwN0,746
 fusion_bench_config/method/concrete_subspace/clip_concrete_task_wise_adamerging.yaml,sha256=P3mwQQewFFiqZNYJp8c02Sf8zBuStKInr_Yn74OCOxI,738
@@ -623,9 +635,10 @@ fusion_bench_config/method/dare/task_arithmetic.yaml,sha256=cUAweNJ6p2aOv__0dvUL
 fusion_bench_config/method/dare/ties_merging.yaml,sha256=7gDW4XpezrsccsbJGqqKrbX26JnqAc85A-MY66DGvuE,416
 fusion_bench_config/method/dawe/dawe_for_clip.yaml,sha256=99P5xpp1YGvIwXGxDcxRtJMLE2FhvEFmFBQjOMEcGoc,1023
 fusion_bench_config/method/doge_ta/doge_ta.yaml,sha256=CtZI3YPMJNDy225yhOJbSiMKlsc-X5nCFzmVh0dvr-w,78
-fusion_bench_config/method/ensemble/max_model_predictor.yaml,sha256=khdpCvKMNytx4nZSgtUJFXv44MVytXu0aqUVd9TixXo,57
-fusion_bench_config/method/ensemble/simple_ensemble.yaml,sha256=RKa3IgN3DfFZVmeXVIdTt0NdPVV0jFkpQz6SxLs3Kso,124
-fusion_bench_config/method/ensemble/weighted_ensemble.yaml,sha256=2KD3PjFglqL7fjqhjXtOWxZ1mvmYodiNVroXsFd7EGE,261
+fusion_bench_config/method/dop/dop.yaml,sha256=ZgdjuVfTj83kAvrS4RrPgGX7d_QQ7d1lIMlzhjiVeUc,954
+fusion_bench_config/method/ensemble/max_model_predictor.yaml,sha256=ugO9FbEYqQk3RkX7wUDE9UOg-4D0F4Rezv0O-7hTeRg,476
+fusion_bench_config/method/ensemble/simple_ensemble.yaml,sha256=kfPAaPVQIet9dYThKNsEBfe9gHdeCREnsM-snSOPahM,546
+fusion_bench_config/method/ensemble/weighted_ensemble.yaml,sha256=LhlxU2P_inxR8MB0Z62phHWj5S4qxD7ITG4Ly-GUcQo,770
 fusion_bench_config/method/expert_sparsity/README.md,sha256=CLE0-XblXDWCUTHPaTNtBH-YquXn-uawwTJiYrgjMaA,239
 fusion_bench_config/method/expert_sparsity/mixtral.yaml,sha256=maFL3LM0zfnQ1eXoNXUslSjgZmpOdUJgl_a31dYUBbc,605
 fusion_bench_config/method/fisher_merging/clip_fisher_merging.yaml,sha256=-m5uDA9hfBg_8vF3s0MnUp0JTl3MqpB4-rlPEg9CHD4,569
@@ -638,23 +651,23 @@ fusion_bench_config/method/gossip/layer_wise_flan_t5.yaml,sha256=2yBqbhwz2vq65wT
 fusion_bench_config/method/isotropic_merging/iso_c.yaml,sha256=mn_5nyc7s_a7QH1MkEj9ZncjNHtZa0mzfXcUGRJOiAw,81
 fusion_bench_config/method/isotropic_merging/iso_cts.yaml,sha256=70BODJt69pZ_9xH7S_Z2Yzb299LFIGkXy1bQiHQad6A,110
 fusion_bench_config/method/linear/expo.yaml,sha256=St3NW6cKVRV3vCn8y0gxQ8k66VTdtsLTEWQTbO9wQ0Y,420
-fusion_bench_config/method/linear/linear_interpolation.yaml,sha256=chM6_HRKKcMleTeuKY3-YNI1qaMG2CfnsRwUxAlHsRw,66
+fusion_bench_config/method/linear/linear_interpolation.yaml,sha256=cAL_ekEIJhJD4cfAbKilV0k_lNNPoJqY4sABVEKcM7E,523
 fusion_bench_config/method/linear/llama_expo.yaml,sha256=SvqamjT06BMObQ58sks5x7Wv6kGpp3-Nlw3ihbD_kSA,621
 fusion_bench_config/method/linear/llama_expo_with_dare.yaml,sha256=Pp8s2xmEg5XSvaGKtwTYx_PzcGvwRh2gPpZ6u9as4_E,383
 fusion_bench_config/method/linear/simple_average_for_causallm.yaml,sha256=qqeIr61PJEcfZclZ5vV64GCzyt-8b1zB0FDZu8DsbXQ,322
 fusion_bench_config/method/linear/task_arithmetic_for_causallm.yaml,sha256=tJA0n0_XVvll4rZYVHQVqFCz8W3Bey6NjPKMIH3-P0U,142
 fusion_bench_config/method/linear/ties_merging_for_causallm.yaml,sha256=1oEIdxV0OqWjDQ9V_lmXEPUayp4KbKHE2SvpCLmiKOU,489
-fusion_bench_config/method/linear/weighted_average.yaml,sha256=uq2gHGCwVHHSa1H-hzcrSlumUTLJ50tfyiY1Mh1pFsk,186
-fusion_bench_config/method/linear/weighted_average_for_llama.yaml,sha256=se2aq6t5R1f-ZG6ubUyRr__DBe9BzXrgL81ua3DkQoM,498
+fusion_bench_config/method/linear/weighted_average.yaml,sha256=OjE4EdfDHPYx8PlBJ6xIpCz4ITu_65VsRyefioRXGQ8,408
+fusion_bench_config/method/linear/weighted_average_for_llama.yaml,sha256=886ZKr81gyN7DISqtbrM5WnjSXd_6AlakQyOJQagoYY,518
 fusion_bench_config/method/lm_finetune/bradley_terry_rm.yaml,sha256=QHsRfJK9K4KajsX3LBHG8cDt7ZLJWxOBnJjpHRQSB_s,1348
 fusion_bench_config/method/lm_finetune/fullfinetune_sft.yaml,sha256=c0rFqj2GV11X9RMraHXJtJ9OiMUzZtvDVsTn4tgAeco,1337
 fusion_bench_config/method/lm_finetune/peftfinetune_sft.yaml,sha256=LjGwfTiiC5iQKr62i22XopQTfSKbx9UbsDvEW-byneQ,1622
-fusion_bench_config/method/model_stock/model_stock.yaml,sha256=G92eRhG_Zsgi2R2FRnMViGC9QPvo7ge-o_eI4ZZLxao,321
+fusion_bench_config/method/model_stock/model_stock.yaml,sha256=4KHAFCjL4AQ5dxkv7IGkUTxE8g-GCoxDkA3BbnlzQC0,530
 fusion_bench_config/method/moe_pruner/moe_pruner.yaml,sha256=OYMYLKvLlNEht7BK9phaTEvAE1ySaVi-pvjYiT-OTGw,442
-fusion_bench_config/method/opcm/opcm.yaml,sha256=YkjAMVGFDj0xqqxA7XWNr0vmcRyxeYbV387nWe0cUbk,331
-fusion_bench_config/method/opcm/task_arithmetic.yaml,sha256=wc9Bz7K_u0feLZbhCBhAuwjeIQTSugJu0I0DCmRNY_c,326
-fusion_bench_config/method/opcm/ties_merging.yaml,sha256=XOE1XzSdYXYzqev9bFD4g4prcmE1OiVINkVXsquizAA,541
-fusion_bench_config/method/opcm/weight_average.yaml,sha256=SmhftSJ_YXN6tn-0GuzQgjbE2sOd7YXoPYjDWzpY_9E,304
+fusion_bench_config/method/opcm/opcm.yaml,sha256=7NBOGo6W1FDbqdkT8gfM5PI2kHfqB8ofMfgcxVI1suM,686
+fusion_bench_config/method/opcm/task_arithmetic.yaml,sha256=WL_nVXhZWV9fe_ttChShkjYZVJnOCzvZ3i7NBppYsxk,743
+fusion_bench_config/method/opcm/ties_merging.yaml,sha256=1-xR0dVEEFJue9r-oBk1ZfGmGM9vCu4cJBG5aZnJ3C8,917
+fusion_bench_config/method/opcm/weight_average.yaml,sha256=n-eyxVkpRanlRJdFWFK3kppiO_W1S99WNjyjdBLDnw0,668
 fusion_bench_config/method/pruning/llama_magnitude_pruning.yaml,sha256=Px8LU_UtDz-YHDFfqQ7scEPOproiFOaudKVshrhCTgc,483
 fusion_bench_config/method/pruning/llama_random_pruning.yaml,sha256=0RiZS8d42PXZzwncPG8zcbnyYJ9vtfr2sOSqS8oDyT4,325
 fusion_bench_config/method/pruning/llama_sparsegpt_pruning.yaml,sha256=gC6Ss0n2tKSb4gyVfx45BvsFbVBGN-om4-2S1sKS-_w,505
@@ -671,15 +684,15 @@ fusion_bench_config/method/regmean/clip_regmean.yaml,sha256=QfkCHCLK9wbyB1Tq1S7Y
 fusion_bench_config/method/regmean/gpt2_regmean.yaml,sha256=n94aTboDdwSA7Tki8l_o8tYQkhXxPV8lRf-dRNPIsOs,422
 fusion_bench_config/method/regmean/regmean.yaml,sha256=ZgVVLx-lHwVgjtjTl4VZUlthh8yyua87QvoJfmNHud4,101
 fusion_bench_config/method/regmean_plusplus/clip_regmean_plusplus.yaml,sha256=A034ryEwvosqyQzA3KWs7kdp-3CUnoJtCujVywV-uzA,434
-fusion_bench_config/method/slerp/slerp.yaml,sha256=xldDUULtfCdwzAkQUb0C8-TmbW7FqcAlIOsPX8p4n6w,116
-fusion_bench_config/method/slerp/slerp_lm.yaml,sha256=c5OQ0zD7e0lXQyec09joHOFNxV1LMT4bHuwgk9GWskc,114
+fusion_bench_config/method/slerp/slerp.yaml,sha256=XR3z6iqyHirkoFSdLAeV2bP1yyI25MoWG-LqdE-ypjA,719
+fusion_bench_config/method/slerp/slerp_lm.yaml,sha256=hO07n6elZg_FrqEfSfbdR-tb1hqwT7vaLgAZKdF8O1o,479
 fusion_bench_config/method/smile_upscaling/causal_lm_upscaling.yaml,sha256=skLwgu_VHShm4m0oEOkqKzcBS5Cz7J29xEj7pTaSm0k,916
 fusion_bench_config/method/smile_upscaling/error_accumulation.yaml,sha256=6Gui-OuQ3P_4TwO_syh9SWJCNeHiAQzS55aO-ByYKbQ,154
 fusion_bench_config/method/smile_upscaling/projected_energy.yaml,sha256=M_EBOC3B_pxaBO3tD6mnbXpvy6-EaegSsE-jdJs-HY0,114
 fusion_bench_config/method/smile_upscaling/singular_projection_merging.yaml,sha256=ZMn_ImRjjc2uozf7ocQIzbgvFDpBV7S-34KptbBXVGo,200
 fusion_bench_config/method/smile_upscaling/smile_mistral_upscaling.yaml,sha256=VFMrkbO69d0wCjTQCuKysYGVe6hEwNu792g1QkhU5Mk,383
 fusion_bench_config/method/smile_upscaling/smile_qwen2_upscaling.yaml,sha256=MfZ1u1HIJoy_csWiLzR4GLz-eiaVxo2gmNYre224yqo,433
-fusion_bench_config/method/smile_upscaling/smile_upscaling.yaml,sha256=G88mabTTniDUtiUC9Vg3cj_sw6D05mE4_ZdyYI4Omjk,477
+fusion_bench_config/method/smile_upscaling/smile_upscaling.yaml,sha256=38DGdOjpDo-dOMpfy807p3x-eAvibjED-BGtFGnaycA,689
 fusion_bench_config/method/sparselo_pruning/llama_iterative_sparselo.yaml,sha256=L-WgNhFjcp_2tocDxZi6STVTtoaSd1v9UOQaKO_QvHM,669
 fusion_bench_config/method/sparselo_pruning/llama_pcp_sparselo.yaml,sha256=prTEFH0eu7R_CVNQ0GPWL9QsOLFcT1uM12zZdi3qcFo,636
 fusion_bench_config/method/sparselo_pruning/llama_sparselo.yaml,sha256=Cmg8N4l--3C0qeSHG-HLOgjJZ954eWHoDNgRnx0pLK0,614
@@ -879,6 +892,12 @@ fusion_bench_config/modelpool/OpenCLIPVisionModelPool/ViT-B-32_TA_sun397_cars.ya
 fusion_bench_config/modelpool/OpenCLIPVisionModelPool/ViT-B-32_TA_sun397_dtd.yaml,sha256=JpPxNF-M5D179nEtfroM4xgEQHN4jHlj6qXXChKt0jg,701
 fusion_bench_config/modelpool/OpenCLIPVisionModelPool/ViT-B-32_individual.yaml,sha256=Edu1Ij1gXcRQs3REHkKvaBVZRXAYVUkdaahtuSnpkmc,225
 fusion_bench_config/modelpool/OpenCLIPVisionModelPool/ViT-L-14_TA8.yaml,sha256=pC28PjVeUkkR-Jd4l0vYh1BScrUn7DAgGIqfuK4wH_I,1185
+fusion_bench_config/modelpool/ResNetForImageClassfication/transformers/resnet152_cifar10.yaml,sha256=CLn-9uSMsQurlUL1uvfiXfPe7huQDTyehkhs4NJ5GOs,375
+fusion_bench_config/modelpool/ResNetForImageClassfication/transformers/resnet152_cifar100.yaml,sha256=_WD401eyMtDXdwse_nb3Y8ELCIG_EL2PpHuY40-y0pU,378
+fusion_bench_config/modelpool/ResNetForImageClassfication/transformers/resnet18_cifar10.yaml,sha256=DPsMtgtczaP8WOqADkjqbJUTnNk3ZOsHiCf6A4_oPu0,374
+fusion_bench_config/modelpool/ResNetForImageClassfication/transformers/resnet18_cifar100.yaml,sha256=VzCKLMbhNXxlqtKUEDpsiL-GVOeoG7iNsu5TZAWK51I,377
+fusion_bench_config/modelpool/ResNetForImageClassfication/transformers/resnet50_cifar10.yaml,sha256=rOOzE5KFamspHgBzAqQg5m0a4B22_uniK6WHhG6-kBA,374
+fusion_bench_config/modelpool/ResNetForImageClassfication/transformers/resnet50_cifar100.yaml,sha256=8t5OR0yhdnGsLgASVJwUbdUgsl4GHLS4HhDuXyxNERU,377
 fusion_bench_config/modelpool/Seq2SeqLMPool/_template.yaml,sha256=mRx-Xx4s6_IBoJJRogIBW4egmqW0wi1kGVWp_YwYVvQ,233
 fusion_bench_config/modelpool/Seq2SeqLMPool/flan-t5-base_glue.yaml,sha256=GK2ewBU0bZmWpaqtdl7zZDUuSmRH1Jz7CFfauYipodk,334
 fusion_bench_config/modelpool/Seq2SeqLMPool/flan-t5-base_glue_lora16.yaml,sha256=qs9LGlcfv-xDFPAhCCzaD5UVI4P8Mza57AqU0EQCeGo,1697
@@ -934,8 +953,8 @@ fusion_bench_config/taskpool/LMEvalHarnessTaskPool/lm_eval.yaml,sha256=3q-KMuFaM
 fusion_bench_config/taskpool/OpenCLIPVisionModelTaskPool/ViT-B-16_TA8.yaml,sha256=GjpiiRownrBCpl-TNwWRW2PYePbF-Cl99jlLNPrK5T4,1017
 fusion_bench_config/taskpool/OpenCLIPVisionModelTaskPool/ViT-B-32_TA8.yaml,sha256=WwiYMQKehtJixDPnu5o3vcWe4yJksXTWRqOzm3uVWXQ,1017
 fusion_bench_config/taskpool/OpenCLIPVisionModelTaskPool/ViT-L-14_TA8.yaml,sha256=xGRt0J9joXTzWUew6DvoYprAWlPXhaVFw5AX4im5VQw,1017
-fusion_bench-0.2.24.dist-info/METADATA,sha256=DllRpMnvVgyeqjN_YlNeo7IlqukzOjuYO_cWopOo1tA,22621
-fusion_bench-0.2.24.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-fusion_bench-0.2.24.dist-info/entry_points.txt,sha256=iUQ8MCJvda7HP4vYh2n1Teoapb4G9PBVYZkAfcc5SHU,116
-fusion_bench-0.2.24.dist-info/top_level.txt,sha256=BuO4TL6iHL_2yPBUX9-LlIrHRczA_BNMIFwweK0PQEI,13
-fusion_bench-0.2.24.dist-info/RECORD,,
+fusion_bench-0.2.26.dist-info/METADATA,sha256=BOHkLorLs0w_fgAtRz7tpYVExKxFiClGISLlsnW3BG8,24307
+fusion_bench-0.2.26.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+fusion_bench-0.2.26.dist-info/entry_points.txt,sha256=iUQ8MCJvda7HP4vYh2n1Teoapb4G9PBVYZkAfcc5SHU,116
+fusion_bench-0.2.26.dist-info/top_level.txt,sha256=BuO4TL6iHL_2yPBUX9-LlIrHRczA_BNMIFwweK0PQEI,13
+fusion_bench-0.2.26.dist-info/RECORD,,

fusion_bench_config/method/bitdelta/bitdelta.yaml

@@ -1,3 +1,6 @@
+# =============================================================================
+# FusionBench Method Configuration: BitDelta
+# =============================================================================
 _target_: fusion_bench.method.bitdelta.BitDeltaAlgorithm
 save_dir: null
 save_full_model: false

fusion_bench_config/method/classification/image_classification_finetune.yaml

@@ -0,0 +1,16 @@
+_target_: fusion_bench.method.classification.ImageClassificationFineTuning
+max_epochs: 10
+max_steps: null
+label_smoothing: 0
+optimizer:
+  _target_: torch.optim.SGD
+  lr: 0.001
+  momentum: 0.9
+  weight_decay: 1e-4
+lr_scheduler:
+  _target_: torch.optim.lr_scheduler.CosineAnnealingLR
+  T_max: ${..max_epochs}
+dataloader_kwargs:
+  batch_size: 256 # batch size per GPU
+  num_workers: 8
+  pin_memory: true

fusion_bench_config/method/classification/image_classification_finetune_test.yaml

@@ -0,0 +1,6 @@
+_target_: fusion_bench.method.classification.ImageClassificationFineTuning_Test
+checkpoint_path: null
+dataloader_kwargs:
+  batch_size: 256
+  num_workers: 4
+  pin_memory: true

fusion_bench_config/method/depth_upscaling.yaml

@@ -1,3 +1,12 @@
+# =============================================================================
+# FusionBench Method Configuration: Depth Upscaling
+# =============================================================================
+# Constructs a deeper model by stacking/selecting layers from existing models.
+#
+# - layer_indices: list[int | str] specifying which layers to use. Strings are Python
+#   expressions evaluated to lists, e.g., "range(6,12)".
+# - Example: [0, 2, 4, "range(6,12)"] selects 1st, 3rd, 5th, and 7th-12th layers.
+# =============================================================================
 _target_: DepthUpscalingAlgorithm
 # this should be a list of integers or string, indicating the sequence of layers. 
 # If the entry is an integer, it will use the n-th layer of the model. 

fusion_bench_config/method/dop/dop.yaml

@@ -0,0 +1,30 @@
+_target_: fusion_bench.method.dop.dop.ContinualDOPForCLIP
+
+# the random seed to use
+seed: null
+# shuffle the order of the models
+shuffle_order: true
+# save the merged model on every step
+save_on_every_step: false
+# evaluate the merged model on every step
+evaluate_on_every_step: true
+
+# optimizer (learning rate)
+lr: 1e-4
+# optimizer (num_steps)
+num_steps: 200
+
+# weighted loss
+# if mgda is true, use mgda to optimize the loss weights
+mgda: true
+# if mgda is false, this is the weight for the loss of the first task
+alpha: 0.8
+# if mgda is true and ema is ture, using exponential moving average (ema), alpha is the initial value
+ema: true
+# if mgda is true and ema is ture, using exponential moving average (ema), beta is the decay rate
+ema_beta: 0.999
+
+# epsilon for svd (the proportion of energy retained)
+svd_epsilon: 0.99999
+# the space to project the delta w (left singular vectors, right singular vectors, or both)
+svd_proj_space: uv # u or v or uv

fusion_bench_config/method/dummy.yaml

@@ -1 +1,7 @@
+# =============================================================================
+# FusionBench Method Configuration: Dummy
+# =============================================================================
+# No-op method for testing pipelines and wiring.
+# Instantiates and exits without modifying models.
+# =============================================================================
 _target_: fusion_bench.method.DummyAlgorithm

fusion_bench_config/method/ensemble/max_model_predictor.yaml

@@ -1 +1,7 @@
+# =============================================================================
+# FusionBench Method Configuration: Max Model Predictor
+# =============================================================================
+# Selects the model with maximum confidence or performance per example/task.
+# No additional hyperparameters are required.
+# =============================================================================
 _target_: fusion_bench.method.MaxModelPredictorAlgorithm

fusion_bench_config/method/ensemble/simple_ensemble.yaml

@@ -1,2 +1,9 @@
+# =============================================================================
+# FusionBench Method Configuration: Simple Ensemble
+# =============================================================================
+# Averages model predictions uniformly.
+#
+# device_map: leave null for single device or provide a mapping for multi-device setups.
+# =============================================================================
 _target_: fusion_bench.method.SimpleEnsembleAlgorithm
-device_map: null  # Set to null for single device, or specify mapping
+device_map: null # Set to null for single device, or specify mapping

fusion_bench_config/method/ensemble/weighted_ensemble.yaml

@@ -1,3 +1,11 @@
+# =============================================================================
+# FusionBench Method Configuration: Weighted Ensemble
+# =============================================================================
+# Ensembles model predictions using specified per-model weights.
+#
+# - Set normalize=true to rescale weights to sum to 1.
+# - weights: one float per model in the pool (order-sensitive). If null, uses equal weights.
+# =============================================================================
 _target_: fusion_bench.method.WeightedEnsembleAlgorithm
 normalize: true
 # this should be a list of floats, one for each model in the ensemble

fusion_bench_config/method/linear/linear_interpolation.yaml

@@ -1,2 +1,10 @@
+# =============================================================================
+# FusionBench Method Configuration: Linear Interpolation
+# =============================================================================
+# Interpolates between two models: (1 - t) * model0 + t * model1
+#
+# - t in [0,1]: 0 returns model0; 1 returns model1.
+# - Only meaningful for two-model pools.
+# =============================================================================
 _target_: fusion_bench.method.LinearInterpolationAlgorithm
 t: 0.5

fusion_bench_config/method/linear/weighted_average.yaml

@@ -1,3 +1,6 @@
+# =============================================================================
+# FusionBench Method Configuration: Weighted Average (Linear)
+# =============================================================================
 _target_: fusion_bench.method.WeightedAverageAlgorithm
 normalize: true # if true, the weights will be normalized before merging
 weights: # List of weights for each model

fusion_bench_config/method/linear/weighted_average_for_llama.yaml

@@ -1,4 +1,4 @@
-_target_: WeightedAverageForLLama
+_target_: fusion_bench.method.WeightedAverageForLLama
 normalize: true # if true, the weights will be normalized before merging
 weights: # List of weights for each model
   - 0.5

fusion_bench_config/method/model_recombination.yaml

@@ -1,3 +1,11 @@
+# =============================================================================
+# FusionBench Method Configuration: Model Recombination
+# =============================================================================
+# Recombines submodules/layers from multiple models to form a new model.
+#
+# - return_modelpool: override run() argument to return model pool instead of merged model.
+#   Set to null to respect runtime argument; set to true/false to force behavior.
+# =============================================================================
 _target_: fusion_bench.method.ModelRecombinationAlgorithm
 # if `return_model_pool` is not null, the argument `return_modelpool` passed to the `run` method will be ignored.
 return_modelpool: null

fusion_bench_config/method/model_stock/model_stock.yaml

@@ -1,3 +1,6 @@
+# =============================================================================
+# FusionBench Method Configuration: Model Stock
+# =============================================================================
 _target_: fusion_bench.method.model_stock.ModelStock
 ignore_keys:
   [
@@ -9,4 +12,4 @@ ignore_keys:
     "model.ln_final.bias",
   ]
 model_save_path: ${path.log_dir}/checkpoint
-model_save_kwargs: null
+model_save_kwargs: null

fusion_bench_config/method/opcm/opcm.yaml

@@ -1,3 +1,8 @@
+# =============================================================================
+# FusionBench Method Configuration: OPCM
+# =============================================================================
+# Incrementally merges models via SVD projection and evaluation per step.
+# =============================================================================
 _target_: fusion_bench.method.opcm.opcm.OPCMForCLIP
 # shuffle the order of the models
 shuffle_order: true

fusion_bench_config/method/opcm/task_arithmetic.yaml

@@ -1,3 +1,9 @@
+# =============================================================================
+# FusionBench Method Configuration: Continual Task Arithmetic
+# =============================================================================
+# Applies task arithmetic incrementally across a stream of models.
+# Maintains per-step save/eval similar to OPCM.
+# =============================================================================
 _target_: fusion_bench.method.opcm.task_arithmetic.ContinualTaskArithmeticForCLIP
 scaling_factor: 0.3
 # shuffle the order of the models

fusion_bench_config/method/opcm/ties_merging.yaml

@@ -1,3 +1,8 @@
+# =============================================================================
+# FusionBench Method Configuration: Continual TIES Merging
+# =============================================================================
+# Continual variant of TIES merging with per-step save/eval instrumentation.
+# =============================================================================
 _target_: fusion_bench.method.opcm.ties_merging.ContinualTiesMergingForCLIP
 # Scaling factor $\lambda$
 scaling_factor: 0.5

fusion_bench_config/method/opcm/weight_average.yaml

@@ -1,3 +1,8 @@
+# =============================================================================
+# FusionBench Method Configuration: Continual Weighted Average
+# =============================================================================
+# Incrementally averages model weights as new models arrive.
+# =============================================================================
 _target_: fusion_bench.method.opcm.weight_average.ContinualWeightAverageForCLIP
 # shuffle the order of the models
 shuffle_order: true

fusion_bench_config/method/simple_average.yaml

@@ -1 +1,10 @@
+# =============================================================================
+# FusionBench Method Configuration: Simple Average
+# =============================================================================
+# Equally averages parameters of all models in the model pool.
+#
+# Usage notes
+# - No hyperparameters required; behavior is deterministic given model order.
+# - Ensure models are architecture-compatible (same shapes) before merging.
+# =============================================================================
 _target_: fusion_bench.method.SimpleAverageAlgorithm