fusion-bench 0.2.28__tar.gz → 0.2.30__tar.gz

{fusion_bench-0.2.28 → fusion_bench-0.2.30}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fusion-bench
-Version: 0.2.28
+Version: 0.2.30
 Summary: A Comprehensive Benchmark of Deep Model Fusion
 Author-email: Anke Tang <tang.anke@foxmail.com>
 Project-URL: Repository, https://github.com/tanganke/fusion_bench
@@ -44,12 +44,14 @@ Dynamic: license-file
 
 [![arXiv](https://img.shields.io/badge/arXiv-2406.03280-b31b1b.svg)](http://arxiv.org/abs/2406.03280)
 [![GitHub License](https://img.shields.io/github/license/tanganke/fusion_bench)](https://github.com/tanganke/fusion_bench/blob/main/LICENSE)
-[![PyPI - Version](https://img.shields.io/pypi/v/fusion-bench)](https://pypi.org/project/fusion-bench/)
-[![Downloads](https://static.pepy.tech/badge/fusion-bench/month)](https://pepy.tech/project/fusion-bench)
 [![Static Badge](https://img.shields.io/badge/doc-mkdocs-blue)](https://tanganke.github.io/fusion_bench/)
 [![Static Badge](https://img.shields.io/badge/code%20style-black-black)](https://github.com/psf/black)
 [![Static Badge](https://img.shields.io/badge/code%20style-yamlfmt-black)](https://github.com/google/yamlfmt)
 
+[![CodeFactor](https://www.codefactor.io/repository/github/tanganke/fusion_bench/badge/main)](https://www.codefactor.io/repository/github/tanganke/fusion_bench/overview/main)
+[![PyPI - Version](https://img.shields.io/pypi/v/fusion-bench)](https://pypi.org/project/fusion-bench/)
+[![Downloads](https://static.pepy.tech/badge/fusion-bench/month)](https://pepy.tech/project/fusion-bench)
+
 </div>
 
 > [!TIP]
@@ -205,15 +207,57 @@ The CLI's design allows for easy extension to new fusion methods, model types, a
 
 Read the [CLI documentation](https://tanganke.github.io/fusion_bench/cli/fusion_bench/) for more information.
 
+## The FusionBench Workflow
+
+FusionBench follows a three-component architecture to perform model fusion experiments:
+
+```mermaid
+graph LR
+    CLI[fusion_bench CLI] --> Hydra[Hydra Config]
+    Hydra --> Program[Program]
+    
+    Program --> MP[ModelPool<br/>Manages Models<br/>& Datasets]
+    Program --> Method[Method<br/>Fusion Algorithm]
+    Program --> TP[TaskPool<br/>Evaluation Tasks]
+    
+    MP --> Method
+    Method --> Merged[Merged Model]
+    Merged --> TP
+    TP --> Report[Evaluation Report]
+    
+    style CLI fill:#e1f5e1
+    style Hydra fill:#f0e1ff
+    style Method fill:#ffe1f0
+    style Merged fill:#fff4e1
+    style Report fill:#e1f0ff
+```
+
+**Key Components:**
+
+1. **CLI**: Entry point using Hydra for configuration management
+2. **Program**: Orchestrates the fusion workflow (e.g., `FabricModelFusionProgram`)
+3. **ModelPool**: Manages task-specific models and their datasets
+4. **Method**: Implements the fusion algorithm (e.g., Simple Average, Task Arithmetic, AdaMerging)
+5. **TaskPool**: Evaluates the merged model on benchmark tasks
+
+**Workflow Steps:**
+
+1. User runs `fusion_bench` with config overrides
+2. Hydra loads YAML configs for method, modelpool, and taskpool
+3. Program instantiates all three components
+4. Method executes fusion algorithm on ModelPool
+5. TaskPool evaluates the merged model
+6. Results are saved and reported
+
 ## Implement your own model fusion algorithm
 
 First, create a new Python file for the algorithm in the `fusion_bench/method` directory.
 Following the naming convention, the file should be named `{method_name_or_class}/{variant}.py`.
 
 ```python
-from fusion_bench import BaseModelFusionAlgorithm, BaseModelPool
+from fusion_bench import BaseAlgorithm, BaseModelPool
 
-class DerivedModelFusionAlgorithm(BaseModelFusionAlgorithm):
+class DerivedModelFusionAlgorithm(BaseAlgorithm):
     """
     An example of a derived model fusion algorithm.
     """
@@ -221,7 +265,7 @@ class DerivedModelFusionAlgorithm(BaseModelFusionAlgorithm):
     # _config_mapping maps the attribution to the corresponding key in the configuration file.
     # this is optional and can be used to serialize the object to a configuration file.
     # `self.config.hyperparam_1` will be mapped to the attribute `hyperparam_attr_1`.
-    _config_mapping = BaseModelFusionAlgorithm._config_mapping | {
+    _config_mapping = BaseAlgorithm._config_mapping | {
         "hyperparam_attr_1": "hyperparam_1",
         "hyperparam_attr_2": "hyperparam_2",
     }
@@ -272,11 +316,24 @@ Click on [<kbd>Use this template</kbd>](https://github.com/fusion-bench/fusion-b
 
 </div>
 
-### FusionBench Command Generator WebUI (for v0.1.x)
+### FusionBench Command Generator WebUI
+
+> [!NOTE]
+> Requires `gradio` package. Install with `pip install gradio`.
+
+For users who prefer a graphical interface, FusionBench provides an interactive web UI for generating commands:
+
+```bash
+fusion_bench_webui
+```
+
+This launches a browser-based interface where you can:
 
-FusionBench Command Generator is a user-friendly web interface for generating FusionBench commands based on configuration files.
-It provides an interactive way to select and customize FusionBench configurations, making it easier to run experiments with different settings.
-[Read more here](https://tanganke.github.io/fusion_bench/cli/fusion_bench_webui/).
+- Select root configurations and components through dropdowns
+- Adjust hyperparameters interactively
+- View real-time YAML configuration updates
+
+The WebUI is particularly useful for exploring available configurations, experimenting with different parameter combinations, and learning the FusionBench configuration structure. [Learn more about the WebUI](https://tanganke.github.io/fusion_bench/cli/fusion_bench_webui/).
 
 ![FusionBench Command Generator Web Interface](docs/cli/images/fusion_bench_webui.png)
 
@@ -287,12 +344,14 @@ If you find this benchmark useful, please consider citing our work:
 ```bibtex
 @article{tang2024fusionbench,
   title={Fusionbench: A comprehensive benchmark of deep model fusion},
-  author={Tang, Anke and Shen, Li and Luo, Yong and Hu, Han and Du, Bo and Tao, Dacheng},
-  journal={arXiv preprint arXiv:2406.03280},
-  year={2024}
+  author={Tang, Anke and Shen, Li and Luo, Yong and Yang, Enneng and Hu, Han and Zhang, Lefei and Du, Bo and Tao, Dacheng},
+  journal={Journal of Machine Learning Research},
+  year={2025}
 }
 ```
 
 ## Star History
 
 [![Star History Chart](https://api.star-history.com/svg?repos=tanganke/fusion_bench&type=Date)](https://www.star-history.com/#tanganke/fusion_bench&Date)
+
+![Alt](https://repobeats.axiom.co/api/embed/83f1f046562e4a4787bdd6ed1190856f9f30bd9f.svg "Repobeats analytics image")

{fusion_bench-0.2.28 → fusion_bench-0.2.30}/README.md

@@ -4,12 +4,14 @@
 
 [![arXiv](https://img.shields.io/badge/arXiv-2406.03280-b31b1b.svg)](http://arxiv.org/abs/2406.03280)
 [![GitHub License](https://img.shields.io/github/license/tanganke/fusion_bench)](https://github.com/tanganke/fusion_bench/blob/main/LICENSE)
-[![PyPI - Version](https://img.shields.io/pypi/v/fusion-bench)](https://pypi.org/project/fusion-bench/)
-[![Downloads](https://static.pepy.tech/badge/fusion-bench/month)](https://pepy.tech/project/fusion-bench)
 [![Static Badge](https://img.shields.io/badge/doc-mkdocs-blue)](https://tanganke.github.io/fusion_bench/)
 [![Static Badge](https://img.shields.io/badge/code%20style-black-black)](https://github.com/psf/black)
 [![Static Badge](https://img.shields.io/badge/code%20style-yamlfmt-black)](https://github.com/google/yamlfmt)
 
+[![CodeFactor](https://www.codefactor.io/repository/github/tanganke/fusion_bench/badge/main)](https://www.codefactor.io/repository/github/tanganke/fusion_bench/overview/main)
+[![PyPI - Version](https://img.shields.io/pypi/v/fusion-bench)](https://pypi.org/project/fusion-bench/)
+[![Downloads](https://static.pepy.tech/badge/fusion-bench/month)](https://pepy.tech/project/fusion-bench)
+
 </div>
 
 > [!TIP]
@@ -165,15 +167,57 @@ The CLI's design allows for easy extension to new fusion methods, model types, a
 
 Read the [CLI documentation](https://tanganke.github.io/fusion_bench/cli/fusion_bench/) for more information.
 
+## The FusionBench Workflow
+
+FusionBench follows a three-component architecture to perform model fusion experiments:
+
+```mermaid
+graph LR
+    CLI[fusion_bench CLI] --> Hydra[Hydra Config]
+    Hydra --> Program[Program]
+    
+    Program --> MP[ModelPool<br/>Manages Models<br/>& Datasets]
+    Program --> Method[Method<br/>Fusion Algorithm]
+    Program --> TP[TaskPool<br/>Evaluation Tasks]
+    
+    MP --> Method
+    Method --> Merged[Merged Model]
+    Merged --> TP
+    TP --> Report[Evaluation Report]
+    
+    style CLI fill:#e1f5e1
+    style Hydra fill:#f0e1ff
+    style Method fill:#ffe1f0
+    style Merged fill:#fff4e1
+    style Report fill:#e1f0ff
+```
+
+**Key Components:**
+
+1. **CLI**: Entry point using Hydra for configuration management
+2. **Program**: Orchestrates the fusion workflow (e.g., `FabricModelFusionProgram`)
+3. **ModelPool**: Manages task-specific models and their datasets
+4. **Method**: Implements the fusion algorithm (e.g., Simple Average, Task Arithmetic, AdaMerging)
+5. **TaskPool**: Evaluates the merged model on benchmark tasks
+
+**Workflow Steps:**
+
+1. User runs `fusion_bench` with config overrides
+2. Hydra loads YAML configs for method, modelpool, and taskpool
+3. Program instantiates all three components
+4. Method executes fusion algorithm on ModelPool
+5. TaskPool evaluates the merged model
+6. Results are saved and reported
+
 ## Implement your own model fusion algorithm
 
 First, create a new Python file for the algorithm in the `fusion_bench/method` directory.
 Following the naming convention, the file should be named `{method_name_or_class}/{variant}.py`.
 
 ```python
-from fusion_bench import BaseModelFusionAlgorithm, BaseModelPool
+from fusion_bench import BaseAlgorithm, BaseModelPool
 
-class DerivedModelFusionAlgorithm(BaseModelFusionAlgorithm):
+class DerivedModelFusionAlgorithm(BaseAlgorithm):
     """
     An example of a derived model fusion algorithm.
     """
@@ -181,7 +225,7 @@ class DerivedModelFusionAlgorithm(BaseModelFusionAlgorithm):
     # _config_mapping maps the attribution to the corresponding key in the configuration file.
     # this is optional and can be used to serialize the object to a configuration file.
     # `self.config.hyperparam_1` will be mapped to the attribute `hyperparam_attr_1`.
-    _config_mapping = BaseModelFusionAlgorithm._config_mapping | {
+    _config_mapping = BaseAlgorithm._config_mapping | {
         "hyperparam_attr_1": "hyperparam_1",
         "hyperparam_attr_2": "hyperparam_2",
     }
@@ -232,11 +276,24 @@ Click on [<kbd>Use this template</kbd>](https://github.com/fusion-bench/fusion-b
 
 </div>
 
-### FusionBench Command Generator WebUI (for v0.1.x)
+### FusionBench Command Generator WebUI
+
+> [!NOTE]
+> Requires `gradio` package. Install with `pip install gradio`.
+
+For users who prefer a graphical interface, FusionBench provides an interactive web UI for generating commands:
+
+```bash
+fusion_bench_webui
+```
+
+This launches a browser-based interface where you can:
 
-FusionBench Command Generator is a user-friendly web interface for generating FusionBench commands based on configuration files.
-It provides an interactive way to select and customize FusionBench configurations, making it easier to run experiments with different settings.
-[Read more here](https://tanganke.github.io/fusion_bench/cli/fusion_bench_webui/).
+- Select root configurations and components through dropdowns
+- Adjust hyperparameters interactively
+- View real-time YAML configuration updates
+
+The WebUI is particularly useful for exploring available configurations, experimenting with different parameter combinations, and learning the FusionBench configuration structure. [Learn more about the WebUI](https://tanganke.github.io/fusion_bench/cli/fusion_bench_webui/).
 
 ![FusionBench Command Generator Web Interface](docs/cli/images/fusion_bench_webui.png)
 
@@ -247,12 +304,14 @@ If you find this benchmark useful, please consider citing our work:
 ```bibtex
 @article{tang2024fusionbench,
   title={Fusionbench: A comprehensive benchmark of deep model fusion},
-  author={Tang, Anke and Shen, Li and Luo, Yong and Hu, Han and Du, Bo and Tao, Dacheng},
-  journal={arXiv preprint arXiv:2406.03280},
-  year={2024}
+  author={Tang, Anke and Shen, Li and Luo, Yong and Yang, Enneng and Hu, Han and Zhang, Lefei and Du, Bo and Tao, Dacheng},
+  journal={Journal of Machine Learning Research},
+  year={2025}
 }
 ```
 
 ## Star History
 
 [![Star History Chart](https://api.star-history.com/svg?repos=tanganke/fusion_bench&type=Date)](https://www.star-history.com/#tanganke/fusion_bench&Date)
+
+![Alt](https://repobeats.axiom.co/api/embed/83f1f046562e4a4787bdd6ed1190856f9f30bd9f.svg "Repobeats analytics image")

fusion_bench-0.2.30/fusion_bench/constants/__init__.py

@@ -0,0 +1,12 @@
+# flake8: noqa F401
+import importlib.metadata
+
+from .paths import *
+from .runtime import RuntimeConstants
+
+# fusionbench version
+try:
+    FUSION_BENCH_VERSION = importlib.metadata.version("fusion-bench")
+except importlib.metadata.PackageNotFoundError:
+    # Fallback when package is not installed (e.g., during development)
+    FUSION_BENCH_VERSION = "0.0.0.dev"

fusion_bench-0.2.30/fusion_bench/constants/runtime.py

@@ -0,0 +1,161 @@
+"""
+Runtime Constants Module.
+
+This module provides a thread-safe singleton class for managing runtime configuration
+and constants across the Fusion Bench framework. It centralizes access to runtime
+settings like cache directories, debug flags, and logging preferences.
+
+Example:
+    ```python
+    from fusion_bench.constants.runtime import RuntimeConstants
+
+    # Get the singleton instance
+    runtime = RuntimeConstants()
+
+    # Configure cache directory
+    runtime.cache_dir = "/custom/cache/path"
+
+    # Enable debug mode
+    runtime.debug = True
+
+    # Control function call logging
+    runtime.print_function_call = True
+    ```
+"""
+
+import os
+import threading
+from pathlib import Path
+from typing import Optional, Union
+
+
+class RuntimeConstants:
+    """
+    Thread-safe singleton for managing runtime configuration and constants.
+
+    This class provides centralized access to runtime settings that affect the
+    behavior of the entire Fusion Bench framework. It ensures consistent
+    configuration across all modules and supports thread-safe access in
+    multi-threaded environments.
+
+    Attributes:
+        debug: Global debug flag for enabling verbose logging and debugging features.
+
+    Example:
+        ```python
+        runtime = RuntimeConstants()
+
+        # Configure caching
+        runtime.cache_dir = Path.home() / ".cache" / "fusion_bench"
+
+        # Enable debugging
+        runtime.debug = True
+        runtime.print_function_call = True
+        ```
+
+    Note:
+        This class implements the singleton pattern with thread-safe initialization.
+        Multiple calls to the constructor will return the same instance.
+    """
+
+    _instance: Optional["RuntimeConstants"] = None
+    _lock = threading.Lock()
+
+    def __new__(cls) -> "RuntimeConstants":
+        """
+        Create or return the singleton instance with thread safety.
+
+        Uses double-check locking pattern to ensure thread-safe singleton creation
+        while minimizing synchronization overhead.
+
+        Returns:
+            The singleton RuntimeConstants instance.
+        """
+        with cls._lock:
+            # Double-check locking pattern
+            if cls._instance is None:
+                cls._instance = super(RuntimeConstants, cls).__new__(cls)
+                cls._instance._initialized = False
+            return cls._instance
+
+    def __init__(self):
+        """
+        Initialize the singleton instance only once.
+
+        Subsequent calls to __init__ are no-ops to maintain singleton behavior.
+        """
+        if not self._initialized:
+            # Initialize default values
+            self._initialized = True
+
+    debug = False
+    """Global debug flag for enabling verbose logging and debugging features."""
+
+    @property
+    def cache_dir(self) -> Path:
+        """
+        Get the default cache directory for models and datasets.
+
+        Returns:
+            Path object pointing to the cache directory.
+
+        Example:
+            ```python
+            runtime = RuntimeConstants()
+            print(f"Cache directory: {runtime.cache_dir}")
+            ```
+        """
+        from fusion_bench.utils.cache_utils import DEFAULT_CACHE_DIR
+
+        return DEFAULT_CACHE_DIR
+
+    @cache_dir.setter
+    def cache_dir(self, path: Union[str, Path]) -> None:
+        """
+        Set the default cache directory for models and datasets.
+
+        Args:
+            path: New cache directory path as string or Path object.
+
+        Example:
+            ```python
+            runtime = RuntimeConstants()
+            runtime.cache_dir = "/data/fusion_bench_cache"
+            ```
+        """
+        from fusion_bench.utils.cache_utils import set_default_cache_dir
+
+        set_default_cache_dir(path)
+
+    @property
+    def print_function_call(self) -> bool:
+        """
+        Get whether function calls are printed during instantiation.
+
+        Returns:
+            True if function call printing is enabled, False otherwise.
+        """
+        from fusion_bench.utils.instantiate_utils import PRINT_FUNCTION_CALL
+
+        return PRINT_FUNCTION_CALL
+
+    @print_function_call.setter
+    def print_function_call(self, enable: bool) -> None:
+        """
+        Set whether to print function calls during instantiation.
+
+        Useful for debugging to see which functions are being called
+        when instantiating objects from configuration.
+
+        Args:
+            enable: True to enable printing, False to disable.
+
+        Example:
+            ```python
+            runtime = RuntimeConstants()
+            runtime.print_function_call = True  # Enable verbose logging
+            ```
+        """
+        from fusion_bench.utils.instantiate_utils import set_print_function_call
+
+        set_print_function_call(enable)

{fusion_bench-0.2.28 → fusion_bench-0.2.30}/fusion_bench/dataset/gsm8k.py

@@ -13,8 +13,12 @@ def load_gsm8k_question_label_data(
 
     An example in the dataset:
 
-    {'question': 'Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?',
-     'answer': 'Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72'}
+    ```python
+    {
+        'question': 'Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?',
+        'answer': 'Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72'
+    }
+    ```
 
     Args:
         dataset_name (Literal["train", "test", "train_socratic", "test_socratic"]): The name of the dataset to load.