fusion-bench 0.2.20__tar.gz → 0.2.21__tar.gz

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fusion_bench
-Version: 0.2.20
+Version: 0.2.21
 Summary: A Comprehensive Benchmark of Deep Model Fusion
 Author-email: Anke Tang <tang.anke@foxmail.com>
 License: MIT License
@@ -45,13 +45,17 @@ Requires-Dist: rich
 Requires-Dist: scipy
 Requires-Dist: h5py
 Requires-Dist: pytest
+Requires-Dist: transformers!=4.49
+Requires-Dist: pillow!=11.2.1
 Provides-Extra: lm-eval-harness
 Requires-Dist: lm-eval; extra == "lm-eval-harness"
+Requires-Dist: immutabledict; extra == "lm-eval-harness"
+Requires-Dist: langdetect; extra == "lm-eval-harness"
 Dynamic: license-file
 
 <div align='center'>
 
-# FusionBench: A Comprehensive Benchmark/ToolKit of Deep Model Fusion
+# FusionBench: A Comprehensive Benchmark/Toolkit of Deep Model Fusion
 
 [![arXiv](https://img.shields.io/badge/arXiv-2406.03280-b31b1b.svg)](http://arxiv.org/abs/2406.03280)
 [![GitHub License](https://img.shields.io/github/license/tanganke/fusion_bench)](https://github.com/tanganke/fusion_bench/blob/main/LICENSE)
@@ -75,7 +79,7 @@ Projects based on FusionBench and news from the community (descending order of d
 <details>
   <summary>The-Hai Nguyen, Dang Huu-Tien, Takeshi Suzuki, and Le-Minh Nguyen. RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging. Aug, 2025. https://www.arxiv.org/abs/2508.03121</summary>
 
-Regression Mean (RegMean), an approach that formulates model merging as a linear regression problem, aims to find the optimal weights for each linear layer in the merge model by minimizing the discrepancy in predictions between the merge and candidate models. RegMean provides a precise closed-form solution for the merging problem; therefore, it offers explainability and computational efficiency. However, RegMean merges each linear layer independently, overlooking how the features and information in the earlier layers propagate through the layers and influence the final prediction in the merge model. In this paper, we introduce RegMean++, a simple yet effective alternative to RegMean, that explicitly incorporates both intra- and cross-layer dependencies between merge models' layers into RegMean's objective. By accounting for these dependencies, RegMean++ better captures the behaviors of the merge model. Extensive experiments demonstrate that RegMean++ consistently outperforms RegMean across diverse settings, including in-domain (ID) and out-of-domain (OOD) generalization, sequential merging, large-scale tasks, and robustness under several types of distribution shifts. Furthermore, RegMean++ achieves competitive or state-of-the-art performance compared to various recent advanced model merging methods. 
+Regression Mean (RegMean), an approach that formulates model merging as a linear regression problem, aims to find the optimal weights for each linear layer in the merge model by minimizing the discrepancy in predictions between the merge and candidate models. RegMean provides a precise closed-form solution for the merging problem; therefore, it offers explainability and computational efficiency. However, RegMean merges each linear layer independently, overlooking how the features and information in the earlier layers propagate through the layers and influence the final prediction in the merge model. In this paper, we introduce RegMean++, a simple yet effective alternative to RegMean, that explicitly incorporates both intra- and cross-layer dependencies between merge models' layers into RegMean's objective. By accounting for these dependencies, RegMean++ better captures the behaviors of the merge model. Extensive experiments demonstrate that RegMean++ consistently outperforms RegMean across diverse settings, including in-domain (ID) and out-of-domain (OOD) generalization, sequential merging, large-scale tasks, and robustness under several types of distribution shifts. Furthermore, RegMean++ achieves competitive or state-of-the-art performance compared to various recent advanced model merging methods.
 
   <img width="1000" alt="image" src="docs/algorithms/images/regmean_vs_regmean_plusplus.png">
 </details>
@@ -89,7 +93,7 @@ Model merging has emerged as a promising approach for multi-task learning (MTL),
 <details>
   <summary>Daniel Marczak, et al. No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces. Feb 2025. https://arxiv.org/abs/2502.04959</summary>
 
-  Model merging integrates the weights of multiple task-specific models into a single multi-task model. Despite recent interest in the problem, a significant performance gap between the combined and single-task models remains. In this paper, we investigate the key characteristics of task matrices -- weight update matrices applied to a pre-trained model -- that enable effective merging. We show that alignment between singular components of task-specific and merged matrices strongly correlates with performance improvement over the pre-trained model. Based on this, we propose an isotropic merging framework that flattens the singular value spectrum of task matrices, enhances alignment, and reduces the performance gap. Additionally, we incorporate both common and task-specific subspaces to further improve alignment and performance. Our proposed approach achieves state-of-the-art performance across multiple scenarios, including various sets of tasks and model scales. This work advances the understanding of model merging dynamics, offering an effective methodology to merge models without requiring additional training. 
+  Model merging integrates the weights of multiple task-specific models into a single multi-task model. Despite recent interest in the problem, a significant performance gap between the combined and single-task models remains. In this paper, we investigate the key characteristics of task matrices -- weight update matrices applied to a pre-trained model -- that enable effective merging. We show that alignment between singular components of task-specific and merged matrices strongly correlates with performance improvement over the pre-trained model. Based on this, we propose an isotropic merging framework that flattens the singular value spectrum of task matrices, enhances alignment, and reduces the performance gap. Additionally, we incorporate both common and task-specific subspaces to further improve alignment and performance. Our proposed approach achieves state-of-the-art performance across multiple scenarios, including various sets of tasks and model scales. This work advances the understanding of model merging dynamics, offering an effective methodology to merge models without requiring additional training.
 </details>
 
 <details>
@@ -107,12 +111,12 @@ Merging multiple expert models offers a promising approach for performing multi-
 <details>
   <summary>Hongling Zheng, Li Shen, Anke Tang, Yong Luo et al. Learn From Model Beyond Fine-Tuning: A Survey. Nature Machine Intelligence. Jan, 2025. https://www.nature.com/articles/s42256-024-00961-0</summary>
 
-  > Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primarily attributed to their ability to comprehend instructions and access extensive, high-quality data. This not only showcases their current effectiveness but also sets a promising trajectory towards the development of artificial general intelligence. Unfortunately, due to multiple constraints, the raw data of the model used for large model training are often inaccessible, so the use of end-to-end models for downstream tasks has become a new research trend, which we call Learn From Model (LFM) in this article. LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. Each category encompasses a repertoire of methods and strategies that aim to enhance the capabilities and performance of FM. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM, in order to help readers better understand the current research status and ideas. To conclude, we summarize the survey by highlighting several critical areas for future exploration and addressing open issues that require further attention from the research community. The relevant papers we investigated in this article can be accessed at https://github.com/ruthless-man/Awesome-Learn-from-Model.
+  > Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primarily attributed to their ability to comprehend instructions and access extensive, high-quality data. This not only showcases their current effectiveness but also sets a promising trajectory towards the development of artificial general intelligence. Unfortunately, due to multiple constraints, the raw data of the model used for large model training are often inaccessible, so the use of end-to-end models for downstream tasks has become a new research trend, which we call Learn From Model (LFM) in this article. LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. Each category encompasses a repertoire of methods and strategies that aim to enhance the capabilities and performance of FM. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM, in order to help readers better understand the current research status and ideas. To conclude, we summarize the survey by highlighting several critical areas for future exploration and addressing open issues that require further attention from the research community. The relevant papers we investigated in this article can be accessed at <https://github.com/ruthless-man/Awesome-Learn-from-Model>.
 </details>
 
 <details>
   <summary>Li Shen, Anke Tang, Enneng Yang et al. Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging. Oct, 2024. https://github.com/EnnengYang/Efficient-WEMoE</summary>
- 
+
   <img width="1018" alt="image" src="https://github.com/user-attachments/assets/b7e1279e-87fc-4016-8867-1bff7700e271">
 
 </details>
@@ -138,7 +142,7 @@ Install from PyPI:
 pip install fusion-bench
 ```
 
-or install the latest version in development from github repository
+or install the latest version in development from the GitHub repository
 
 ```bash
 git clone https://github.com/tanganke/fusion_bench.git
@@ -155,7 +159,6 @@ pip install -e . # install the package in editable mode
 
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10256836.svg)](https://doi.org/10.5281/zenodo.10256836)
 
-
 ```bash
 pip install "fusion-bench[lm-eval-harness]"
 ```
@@ -205,8 +208,8 @@ The project is structured as follows:
 
 ## A Unified Command Line Interface
 
-The `fusion_bench` command-line interface is a powerful tool for researchers and practitioners in the field of model fusion. It provides a streamlined way to experiment with various fusion algorithms, model combinations, and evaluation tasks. 
-By leveraging Hydra's configuration management, fusion_bench offers flexibility in setting up experiments and reproducibility in results. 
+The `fusion_bench` command-line interface is a powerful tool for researchers and practitioners in the field of model fusion. It provides a streamlined way to experiment with various fusion algorithms, model combinations, and evaluation tasks.
+By leveraging Hydra's configuration management, fusion_bench offers flexibility in setting up experiments and reproducibility in results.
 The CLI's design allows for easy extension to new fusion methods, model types, and tasks, making it a versatile platform for advancing research in model fusion techniques.
 
 Read the [CLI documentation](https://tanganke.github.io/fusion_bench/cli/fusion_bench/) for more information.
@@ -245,7 +248,7 @@ class DerivedModelFusionAlgorithm(BaseModelFusionAlgorithm):
         )
 ```
 
-A corresponding configuration file should be created to specify the class and hyperparameters of the algorithm. 
+A corresponding configuration file should be created to specify the class and hyperparameters of the algorithm.
 Here we assume the configuration file is placed at `config/method/your_algorithm_config.yaml`.
 
 > [!NOTE]
@@ -280,7 +283,7 @@ Click on [<kbd>Use this template</kbd>](https://github.com/fusion-bench/fusion-b
 
 ### FusionBench Command Generator WebUI (for v0.1.x)
 
-FusionBench Command Generator is a user-friendly web interface for generating FusionBench commands based on configuration files. 
+FusionBench Command Generator is a user-friendly web interface for generating FusionBench commands based on configuration files.
 It provides an interactive way to select and customize FusionBench configurations, making it easier to run experiments with different settings.
 [Read more here](https://tanganke.github.io/fusion_bench/cli/fusion_bench_webui/).
 
@@ -291,18 +294,14 @@ It provides an interactive way to select and customize FusionBench configuration
 If you find this benchmark useful, please consider citing our work:
 
 ```bibtex
-@misc{tangFusionBenchComprehensiveBenchmark2024,
-  title = {{{FusionBench}}: {{A Comprehensive Benchmark}} of {{Deep Model Fusion}}},
-  shorttitle = {{{FusionBench}}},
-  author = {Tang, Anke and Shen, Li and Luo, Yong and Hu, Han and Du, Bo and Tao, Dacheng},
-  year = {2024},
-  month = jun,
-  number = {arXiv:2406.03280},
-  eprint = {2406.03280},
-  publisher = {arXiv},
-  url = {http://arxiv.org/abs/2406.03280},
-  archiveprefix = {arxiv},
-  langid = {english},
-  keywords = {Computer Science - Artificial Intelligence,Computer Science - Computation and Language,Computer Science - Machine Learning}
+@article{tang2024fusionbench,
+  title={Fusionbench: A comprehensive benchmark of deep model fusion},
+  author={Tang, Anke and Shen, Li and Luo, Yong and Hu, Han and Du, Bo and Tao, Dacheng},
+  journal={arXiv preprint arXiv:2406.03280},
+  year={2024}
 }
 ```
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=tanganke/fusion_bench&type=Date)](https://www.star-history.com/#tanganke/fusion_bench&Date)

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/README.md

@@ -1,6 +1,6 @@
 <div align='center'>
 
-# FusionBench: A Comprehensive Benchmark/ToolKit of Deep Model Fusion
+# FusionBench: A Comprehensive Benchmark/Toolkit of Deep Model Fusion
 
 [![arXiv](https://img.shields.io/badge/arXiv-2406.03280-b31b1b.svg)](http://arxiv.org/abs/2406.03280)
 [![GitHub License](https://img.shields.io/github/license/tanganke/fusion_bench)](https://github.com/tanganke/fusion_bench/blob/main/LICENSE)
@@ -24,7 +24,7 @@ Projects based on FusionBench and news from the community (descending order of d
 <details>
   <summary>The-Hai Nguyen, Dang Huu-Tien, Takeshi Suzuki, and Le-Minh Nguyen. RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging. Aug, 2025. https://www.arxiv.org/abs/2508.03121</summary>
 
-Regression Mean (RegMean), an approach that formulates model merging as a linear regression problem, aims to find the optimal weights for each linear layer in the merge model by minimizing the discrepancy in predictions between the merge and candidate models. RegMean provides a precise closed-form solution for the merging problem; therefore, it offers explainability and computational efficiency. However, RegMean merges each linear layer independently, overlooking how the features and information in the earlier layers propagate through the layers and influence the final prediction in the merge model. In this paper, we introduce RegMean++, a simple yet effective alternative to RegMean, that explicitly incorporates both intra- and cross-layer dependencies between merge models' layers into RegMean's objective. By accounting for these dependencies, RegMean++ better captures the behaviors of the merge model. Extensive experiments demonstrate that RegMean++ consistently outperforms RegMean across diverse settings, including in-domain (ID) and out-of-domain (OOD) generalization, sequential merging, large-scale tasks, and robustness under several types of distribution shifts. Furthermore, RegMean++ achieves competitive or state-of-the-art performance compared to various recent advanced model merging methods. 
+Regression Mean (RegMean), an approach that formulates model merging as a linear regression problem, aims to find the optimal weights for each linear layer in the merge model by minimizing the discrepancy in predictions between the merge and candidate models. RegMean provides a precise closed-form solution for the merging problem; therefore, it offers explainability and computational efficiency. However, RegMean merges each linear layer independently, overlooking how the features and information in the earlier layers propagate through the layers and influence the final prediction in the merge model. In this paper, we introduce RegMean++, a simple yet effective alternative to RegMean, that explicitly incorporates both intra- and cross-layer dependencies between merge models' layers into RegMean's objective. By accounting for these dependencies, RegMean++ better captures the behaviors of the merge model. Extensive experiments demonstrate that RegMean++ consistently outperforms RegMean across diverse settings, including in-domain (ID) and out-of-domain (OOD) generalization, sequential merging, large-scale tasks, and robustness under several types of distribution shifts. Furthermore, RegMean++ achieves competitive or state-of-the-art performance compared to various recent advanced model merging methods.
 
   <img width="1000" alt="image" src="docs/algorithms/images/regmean_vs_regmean_plusplus.png">
 </details>
@@ -38,7 +38,7 @@ Model merging has emerged as a promising approach for multi-task learning (MTL),
 <details>
   <summary>Daniel Marczak, et al. No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces. Feb 2025. https://arxiv.org/abs/2502.04959</summary>
 
-  Model merging integrates the weights of multiple task-specific models into a single multi-task model. Despite recent interest in the problem, a significant performance gap between the combined and single-task models remains. In this paper, we investigate the key characteristics of task matrices -- weight update matrices applied to a pre-trained model -- that enable effective merging. We show that alignment between singular components of task-specific and merged matrices strongly correlates with performance improvement over the pre-trained model. Based on this, we propose an isotropic merging framework that flattens the singular value spectrum of task matrices, enhances alignment, and reduces the performance gap. Additionally, we incorporate both common and task-specific subspaces to further improve alignment and performance. Our proposed approach achieves state-of-the-art performance across multiple scenarios, including various sets of tasks and model scales. This work advances the understanding of model merging dynamics, offering an effective methodology to merge models without requiring additional training. 
+  Model merging integrates the weights of multiple task-specific models into a single multi-task model. Despite recent interest in the problem, a significant performance gap between the combined and single-task models remains. In this paper, we investigate the key characteristics of task matrices -- weight update matrices applied to a pre-trained model -- that enable effective merging. We show that alignment between singular components of task-specific and merged matrices strongly correlates with performance improvement over the pre-trained model. Based on this, we propose an isotropic merging framework that flattens the singular value spectrum of task matrices, enhances alignment, and reduces the performance gap. Additionally, we incorporate both common and task-specific subspaces to further improve alignment and performance. Our proposed approach achieves state-of-the-art performance across multiple scenarios, including various sets of tasks and model scales. This work advances the understanding of model merging dynamics, offering an effective methodology to merge models without requiring additional training.
 </details>
 
 <details>
@@ -56,12 +56,12 @@ Merging multiple expert models offers a promising approach for performing multi-
 <details>
   <summary>Hongling Zheng, Li Shen, Anke Tang, Yong Luo et al. Learn From Model Beyond Fine-Tuning: A Survey. Nature Machine Intelligence. Jan, 2025. https://www.nature.com/articles/s42256-024-00961-0</summary>
 
-  > Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primarily attributed to their ability to comprehend instructions and access extensive, high-quality data. This not only showcases their current effectiveness but also sets a promising trajectory towards the development of artificial general intelligence. Unfortunately, due to multiple constraints, the raw data of the model used for large model training are often inaccessible, so the use of end-to-end models for downstream tasks has become a new research trend, which we call Learn From Model (LFM) in this article. LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. Each category encompasses a repertoire of methods and strategies that aim to enhance the capabilities and performance of FM. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM, in order to help readers better understand the current research status and ideas. To conclude, we summarize the survey by highlighting several critical areas for future exploration and addressing open issues that require further attention from the research community. The relevant papers we investigated in this article can be accessed at https://github.com/ruthless-man/Awesome-Learn-from-Model.
+  > Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primarily attributed to their ability to comprehend instructions and access extensive, high-quality data. This not only showcases their current effectiveness but also sets a promising trajectory towards the development of artificial general intelligence. Unfortunately, due to multiple constraints, the raw data of the model used for large model training are often inaccessible, so the use of end-to-end models for downstream tasks has become a new research trend, which we call Learn From Model (LFM) in this article. LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. Each category encompasses a repertoire of methods and strategies that aim to enhance the capabilities and performance of FM. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM, in order to help readers better understand the current research status and ideas. To conclude, we summarize the survey by highlighting several critical areas for future exploration and addressing open issues that require further attention from the research community. The relevant papers we investigated in this article can be accessed at <https://github.com/ruthless-man/Awesome-Learn-from-Model>.
 </details>
 
 <details>
   <summary>Li Shen, Anke Tang, Enneng Yang et al. Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging. Oct, 2024. https://github.com/EnnengYang/Efficient-WEMoE</summary>
- 
+
   <img width="1018" alt="image" src="https://github.com/user-attachments/assets/b7e1279e-87fc-4016-8867-1bff7700e271">
 
 </details>
@@ -87,7 +87,7 @@ Install from PyPI:
 pip install fusion-bench
 ```
 
-or install the latest version in development from github repository
+or install the latest version in development from the GitHub repository
 
 ```bash
 git clone https://github.com/tanganke/fusion_bench.git
@@ -104,7 +104,6 @@ pip install -e . # install the package in editable mode
 
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10256836.svg)](https://doi.org/10.5281/zenodo.10256836)
 
-
 ```bash
 pip install "fusion-bench[lm-eval-harness]"
 ```
@@ -154,8 +153,8 @@ The project is structured as follows:
 
 ## A Unified Command Line Interface
 
-The `fusion_bench` command-line interface is a powerful tool for researchers and practitioners in the field of model fusion. It provides a streamlined way to experiment with various fusion algorithms, model combinations, and evaluation tasks. 
-By leveraging Hydra's configuration management, fusion_bench offers flexibility in setting up experiments and reproducibility in results. 
+The `fusion_bench` command-line interface is a powerful tool for researchers and practitioners in the field of model fusion. It provides a streamlined way to experiment with various fusion algorithms, model combinations, and evaluation tasks.
+By leveraging Hydra's configuration management, fusion_bench offers flexibility in setting up experiments and reproducibility in results.
 The CLI's design allows for easy extension to new fusion methods, model types, and tasks, making it a versatile platform for advancing research in model fusion techniques.
 
 Read the [CLI documentation](https://tanganke.github.io/fusion_bench/cli/fusion_bench/) for more information.
@@ -194,7 +193,7 @@ class DerivedModelFusionAlgorithm(BaseModelFusionAlgorithm):
         )
 ```
 
-A corresponding configuration file should be created to specify the class and hyperparameters of the algorithm. 
+A corresponding configuration file should be created to specify the class and hyperparameters of the algorithm.
 Here we assume the configuration file is placed at `config/method/your_algorithm_config.yaml`.
 
 > [!NOTE]
@@ -229,7 +228,7 @@ Click on [<kbd>Use this template</kbd>](https://github.com/fusion-bench/fusion-b
 
 ### FusionBench Command Generator WebUI (for v0.1.x)
 
-FusionBench Command Generator is a user-friendly web interface for generating FusionBench commands based on configuration files. 
+FusionBench Command Generator is a user-friendly web interface for generating FusionBench commands based on configuration files.
 It provides an interactive way to select and customize FusionBench configurations, making it easier to run experiments with different settings.
 [Read more here](https://tanganke.github.io/fusion_bench/cli/fusion_bench_webui/).
 
@@ -240,18 +239,14 @@ It provides an interactive way to select and customize FusionBench configuration
 If you find this benchmark useful, please consider citing our work:
 
 ```bibtex
-@misc{tangFusionBenchComprehensiveBenchmark2024,
-  title = {{{FusionBench}}: {{A Comprehensive Benchmark}} of {{Deep Model Fusion}}},
-  shorttitle = {{{FusionBench}}},
-  author = {Tang, Anke and Shen, Li and Luo, Yong and Hu, Han and Du, Bo and Tao, Dacheng},
-  year = {2024},
-  month = jun,
-  number = {arXiv:2406.03280},
-  eprint = {2406.03280},
-  publisher = {arXiv},
-  url = {http://arxiv.org/abs/2406.03280},
-  archiveprefix = {arxiv},
-  langid = {english},
-  keywords = {Computer Science - Artificial Intelligence,Computer Science - Computation and Language,Computer Science - Machine Learning}
+@article{tang2024fusionbench,
+  title={Fusionbench: A comprehensive benchmark of deep model fusion},
+  author={Tang, Anke and Shen, Li and Luo, Yong and Hu, Han and Du, Bo and Tao, Dacheng},
+  journal={arXiv preprint arXiv:2406.03280},
+  year={2024}
 }
 ```
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=tanganke/fusion_bench&type=Date)](https://www.star-history.com/#tanganke/fusion_bench&Date)

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/fusion_bench/__init__.py

@@ -20,6 +20,7 @@ from . import (
     utils,
 )
 from .method import BaseAlgorithm, BaseModelFusionAlgorithm
+from .mixins import auto_register_config
 from .modelpool import BaseModelPool
 from .models import separate_io
 from .taskpool import BaseTaskPool

fusion_bench-0.2.21/fusion_bench/_get_started/__init__.py

@@ -0,0 +1,3 @@
+"""
+Tutorial module for FusionBench
+"""

fusion_bench-0.2.21/fusion_bench/_get_started/greeting_program.py

@@ -0,0 +1,49 @@
+import logging
+from typing import Optional
+
+from omegaconf import DictConfig
+
+from fusion_bench.programs import BaseHydraProgram
+
+log = logging.getLogger(__name__)
+
+
+class GreetingProgram(BaseHydraProgram):
+    """
+    A simple program that greets users with a custom message.
+    """
+
+    _config_mapping = BaseHydraProgram._config_mapping | {
+        "message": "message",
+        "name": "name",
+        "repeat_count": "repeat_count",
+    }
+
+    def __init__(
+        self,
+        message: str = "Hello",
+        name: str = "World",
+        repeat_count: int = 1,
+        **kwargs,
+    ):
+        self.message = message
+        self.name = name
+        self.repeat_count = repeat_count
+        super().__init__(**kwargs)
+
+    def run(self):
+        """Execute the greeting workflow."""
+        log.info("Starting greeting program")
+
+        # Create the greeting
+        greeting = f"{self.message}, {self.name}!"
+
+        # Print the greeting multiple times
+        for i in range(self.repeat_count):
+            if self.repeat_count > 1:
+                print(f"[{i+1}/{self.repeat_count}] {greeting}")
+            else:
+                print(greeting)
+
+        log.info("Greeting program completed")
+        return greeting

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/fusion_bench/compat/method/base_algorithm.py

@@ -36,6 +36,20 @@ class ModelFusionAlgorithm(ABC):
             algorithm_config = DictConfig({})
         self.config = algorithm_config
 
+    def on_run_start(self):
+        """
+        Hook method called at the start of the run.
+        Can be overridden by subclasses to perform initialization tasks.
+        """
+        pass
+
+    def on_run_end(self):
+        """
+        Hook method called at the end of the run.
+        Can be overridden by subclasses to perform cleanup tasks.
+        """
+        pass
+
     @abstractmethod
     def run(self, modelpool):
         """

fusion_bench-0.2.21/fusion_bench/constants/__init__.py

@@ -0,0 +1,7 @@
+# flake8: noqa F401
+import importlib.metadata
+
+from .paths import *
+
+# fusionbench version
+FUSION_BENCH_VERSION = importlib.metadata.version("fusion-bench")

fusion_bench-0.2.21/fusion_bench/constants/clip_vision.py

@@ -0,0 +1,46 @@
+"Constants for CLIP Vision Model Merging"
+
+TASK_NAMES_TA8 = [
+    "sun397",
+    "stanford-cars",
+    "resisc45",
+    "eurosat",
+    "svhn",
+    "gtsrb",
+    "mnist",
+    "dtd",
+]
+"The 8 tasks used in the Task Arithmetic paper."
+TASK_NAMES_TALL8 = TASK_NAMES_TA8
+"The 8 tasks used in the Tall Mask paper"
+TASK_NAMES_TALL10 = TASK_NAMES_TA8 + ["oxford_flowers102", "pcam"]
+TASK_NAMES_TALL12 = TASK_NAMES_TALL10 + [
+    "fer2013",
+    "oxford-iiit-pet",
+]
+TASK_NAMES_TALL14 = TASK_NAMES_TALL12 + [
+    "stl10",
+    "cifar100",
+]
+"The 14 tasks used in the TALL mask paper"
+TASK_NAMES_TALL16 = TASK_NAMES_TALL14 + ["cifar10", "food101"]
+TASK_NAMES_TALL18 = TASK_NAMES_TALL16 + ["fashion_mnist", "emnist_letters"]
+TASK_NAMES_TALL20 = TASK_NAMES_TALL18 + ["kmnist", "rendered-sst2"]
+"The 20 tasks used in the TALL mask paper"
+TASK_NAMES_TA8_CAP = [
+    "SUN397",
+    "Cars",
+    "RESISC45",
+    "EuroSAT",
+    "SVHN",
+    "GTSRB",
+    "MNIST",
+    "DTD",
+]
+TASK_NAMES_TALL8_CAP = TASK_NAMES_TA8_CAP
+TASK_NAMES_TALL10_CAP = TASK_NAMES_TALL8_CAP + ["Flowers102", "PCAM"]
+TASK_NAMES_TALL12_CAP = TASK_NAMES_TALL10_CAP + ["FER2013", "OxfordIIITPet"]
+TASK_NAMES_TALL14_CAP = TASK_NAMES_TALL12_CAP + ["STL10", "CIFAR100"]
+TASK_NAMES_TALL16_CAP = TASK_NAMES_TALL14_CAP + ["CIFAR10", "Food101"]
+TASK_NAMES_TALL18_CAP = TASK_NAMES_TALL16_CAP + ["FashionMNIST", "EMNIST"]
+TASK_NAMES_TALL20_CAP = TASK_NAMES_TALL18_CAP + ["KMNIST", "RenderedSST2"]

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/fusion_bench/constants/paths.py

@@ -7,10 +7,14 @@ log = logging.getLogger(__name__)
 __all__ = ["LIBRARY_PATH", "PROJECT_ROOT_PATH", "DEFAULT_CONFIG_PATH"]
 
 LIBRARY_PATH = Path(importlib.import_module("fusion_bench").__path__[0])
+"""Path to the library directory."""
+
 PROJECT_ROOT_PATH = LIBRARY_PATH.parent
+"""Path to the project root directory."""
 
 if (PROJECT_ROOT_PATH / "config").is_dir():
     DEFAULT_CONFIG_PATH = PROJECT_ROOT_PATH / "config"
+    """Path to the default config directory."""
 elif (PROJECT_ROOT_PATH / "fusion_bench_config").is_dir():
     DEFAULT_CONFIG_PATH = PROJECT_ROOT_PATH / "fusion_bench_config"
 else:

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/fusion_bench/dataset/clip_dataset.py

@@ -5,6 +5,7 @@ This module provides a class to convert a dataset whose object is a list of dict
 from typing import Optional, Tuple
 
 import torch
+from torch.utils.data import Dataset
 from transformers import CLIPProcessor, ProcessorMixin
 
 __all__ = ["CLIPDataset"]
@@ -28,7 +29,7 @@ class CLIPDataset(torch.utils.data.Dataset):
         processor (CLIPProcessor): The CLIP processor used for image preprocessing.
     """
 
-    def __init__(self, dataset, processor: Optional[CLIPProcessor] = None):
+    def __init__(self, dataset: Dataset, processor: Optional[CLIPProcessor] = None):
         self.dataset = dataset
         self.processor = processor
 

{fusion_bench-0.2.20 → fusion_bench-0.2.21}/fusion_bench/dataset/gpt2_glue.py

@@ -16,7 +16,7 @@ from functools import partial
 from pathlib import Path
 from typing import Literal
 
-from datasets import load_dataset, load_from_disk
+from datasets import Dataset, load_dataset, load_from_disk
 from transformers import PreTrainedTokenizer
 
 
@@ -147,7 +147,7 @@ class TokenizedGLUE:
         return glue_dataset_loaders[name]()
 
     @cache_dataset
-    def load_mrpc_dataset(self):
+    def load_mrpc_dataset(self) -> Dataset:
         """
         Load and tokenize the MRPC dataset.
 
@@ -166,7 +166,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_rte_dataset(self):
+    def load_rte_dataset(self) -> Dataset:
         """
         Load and tokenize the RTE dataset.
 
@@ -186,7 +186,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_wnli_dataset(self):
+    def load_wnli_dataset(self) -> Dataset:
         """
         Load and tokenize the WNLI dataset.
 
@@ -205,7 +205,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_qqp_dataset(self):
+    def load_qqp_dataset(self) -> Dataset:
         """
         Load and tokenize the QQP dataset.
 
@@ -224,7 +224,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_mnli_dataset(self):
+    def load_mnli_dataset(self) -> Dataset:
         """
         Load and tokenize the MNLI dataset.
 
@@ -243,7 +243,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_cola_dataset(self):
+    def load_cola_dataset(self) -> Dataset:
         """
         Load and tokenize the CoLA dataset.
 
@@ -262,7 +262,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_sst2_dataset(self):
+    def load_sst2_dataset(self) -> Dataset:
         """
         Load and tokenize the SST-2 dataset.
 
@@ -281,7 +281,7 @@ class TokenizedGLUE:
         return dataset
 
     @cache_dataset
-    def load_qnli_dataset(self):
+    def load_qnli_dataset(self) -> Dataset:
         """
         Load and tokenize the QNLI dataset.