xfuser 0.2__tar.gz → 0.3__tar.gz

{xfuser-0.2/xfuser.egg-info → xfuser-0.3}/PKG-INFO

@@ -1,23 +1,23 @@
 Metadata-Version: 2.1
 Name: xfuser
-Version: 0.2
+Version: 0.3
 Summary: xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters
 Home-page: https://github.com/xdit-project/xDiT.
-Author: Jiannan Wang, Jiarui Fang, Jinzhe Pan, Aoyu Li, Pengcheng Yang
+Author: Jiannan Wang, Jiarui Fang, Jinzhe Pan, Aoyu Li
 Author-email: fangjiarui123@gmail.com
 Classifier: Programming Language :: Python :: 3
 Classifier: Operating System :: OS Independent
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
-Requires-Dist: torch>=2.2
+Requires-Dist: torch>=2.3.0
 Requires-Dist: diffusers>=0.30.0
-Requires-Dist: transformers
-Requires-Dist: sentencepiece
+Requires-Dist: transformers>=4.39.1
+Requires-Dist: sentencepiece>=0.1.99
 Requires-Dist: accelerate==0.33.0
-Requires-Dist: beautifulsoup4
+Requires-Dist: beautifulsoup4>=4.12.3
 Requires-Dist: distvae
-Requires-Dist: ftfy
+Requires-Dist: ftfy>=6.2.0
 
 <div align="center">
   <!-- <h1>KTransformers</h1> -->
@@ -42,7 +42,7 @@ Requires-Dist: ftfy
 - [🚀 QuickStart](#QuickStart)
 - [✨ the xDiT's secret weapons](#secrets)
   - [1. PipeFusion](#PipeFusion)
-  - [2. USP](#USP)
+  - [2. Unified Sequence Parallel](#USP)
   - [3. Hybrid Parallel](#hybrid_parallel)
   - [4. CFG Parallel](#cfg_parallel)
   - [5. Parallel VAE](#parallel_vae)
@@ -71,9 +71,12 @@ xDiT provides a suite of efficient parallel inference approaches for Diffusion M
 
 The four parallel methods in xDiT can be configured in a hybrid manner, optimizing communication patterns to best suit the underlying network hardware.
 
-xDiT offers a set of APIs to adapt DiT models in [huggingface/diffusers](https://github.com/huggingface/diffusers) to hybrid parallel implementation through simple wrappers. 
+As shown in the following picture, xDiT offers a set of APIs to adapt DiT models in [huggingface/diffusers](https://github.com/huggingface/diffusers) to hybrid parallel implementation through simple wrappers. 
 If the model you require is not available in the model zoo, developing it yourself is straightforward; please refer to our [Dev Guide](#dev-guide).
 
+<div align="center">
+    <img src="assets/methods/xdit_method.png" alt="xdit methods">
+</div>
 
 We also have implemented the following parallel stategies for reference:
 
@@ -109,6 +112,7 @@ As we can see, PipeFusion and Sequence Parallel achieve lowest communication cos
 
 <h2 id="updates">📢 Updates</h2>
 
+* 🎉**August 9, 2024**: Support Latte sequence parallel version. The inference scripts are [examples/latte_example](examples/latte_example.py).
 * 🎉**August 8, 2024**: Support Flux sequence parallel version. The inference scripts are [examples/flux_example](examples/flux_example.py).
 * 🎉**August 2, 2024**: Support Stable Diffusion 3 hybrid parallel version. The inference scripts are [examples/sd3_example](examples/sd3_example.py).
 * 🎉**July 18, 2024**: Support PixArt-Sigma and PixArt-Alpha. The inference scripts are [examples/pixartsigma_example.py](examples/pixartsigma_example.py), [examples/pixartalpha_example.py](examples/pixartalpha_example.py).
@@ -120,13 +124,18 @@ As we can see, PipeFusion and Sequence Parallel achieve lowest communication cos
 
 <h2 id="support-dits">🎯 Supported DiTs</h2>
 
--  [🟠 Flux](https://huggingface.co/black-forest-labs/FLUX.1-schnell)
--  [🔴 PixArt-Sigma](https://huggingface.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS)
--  [🔵 HunyuanDiT-v1.2-Diffusers](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers)
--  [🟢 PixArt-alpha](https://huggingface.co/PixArt-alpha/PixArt-alpha)
--  [🟠 Stable Diffusion 3](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers)
--  [🔴 DiT-XL](https://huggingface.co/facebook/DiT-XL-2-256)
+| Model Name | CFG | SP | PipeFusion |
+| --- | --- | --- | --- |
+| [🎬 Latte](https://huggingface.co/maxin-cn/Latte-1) | ❎ | ✔️ | ❎ | 
+| [🔵 HunyuanDiT-v1.2-Diffusers](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers) | ✔️ | ✔️ | ✔️ |
+| [🟠 Flux](https://huggingface.co/black-forest-labs/FLUX.1-schnell) | NA | ✔️ | ❎ |
+| [🔴 PixArt-Sigma](https://huggingface.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS) | ✔️ | ✔️ | ✔️ |
+| [🟢 PixArt-alpha](https://huggingface.co/PixArt-alpha/PixArt-alpha) | ✔️ | ✔️ | ✔️ |
+| [🟠 Stable Diffusion 3](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers) | ✔️ | ✔️ | ✔️ |
 
+### Supported by legacy version only:
+
+-  [🔴 DiT-XL](https://huggingface.co/facebook/DiT-XL-2-256)
 
 <h2 id="perf">📈 Performance</h2>
 
@@ -162,18 +171,26 @@ Here are the benchmark results for Pixart-Alpha using the 20-step DPM solver as
 
 <h2 id="QuickStart">🚀 QuickStart</h2>
 
-1. Install yunchang for sequence parallel.
+### 1. Install from pip
+
+```
+pip install xfuser
+```
+
+### 2. Install from source 
+
+#### 2.1 Install yunchang for sequence parallel.
 
 Install yunchang from [feifeibear/long-context-attention](https://github.com/feifeibear/long-context-attention).
 Please note that it has a dependency on flash attention and specific GPU model requirements. We recommend installing yunchang from the source code rather than using `pip install yunchang==0.2.0`.
 
-2. Install xDiT
+#### 2.2 Install xDiT
 
 ```
 python setup.py install
 ```
 
-3. Usage
+### 2. Usage
 
 We provide examples demonstrating how to run models with xDiT in the [./examples/](./examples/) directory. 
 You can easily modify the model type, model directory, and parallel options in the [examples/run.sh](examples/run.sh) within the script to run some already supported DiT models.
@@ -255,7 +272,7 @@ examples/pixartalpha_example.py \
 --num_inference_steps 20 \
 --warmup_steps 0 \
 --prompt "A small dog" \
---use_split_batch
+--use_cfg_parallel
 ```
 
 
@@ -318,5 +335,12 @@ We also welcome developers to join and contribute more features and models to th
       archivePrefix={arXiv},
       primaryClass={cs.CV}
 }
+
+@article{fang2024unified,
+  title={USP: a Unified Sequence Parallelism Approach for Long Context Generative AI},
+  author={Fang, Jiarui and Zhao, Shangchun},
+  journal={arXiv preprint arXiv:2405.07719},
+  year={2024}
+}
 ```
 

xfuser-0.2/PKG-INFO → xfuser-0.3/README.md

@@ -1,24 +1,3 @@
-Metadata-Version: 2.1
-Name: xfuser
-Version: 0.2
-Summary: xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters
-Home-page: https://github.com/xdit-project/xDiT.
-Author: Jiannan Wang, Jiarui Fang, Jinzhe Pan, Aoyu Li, Pengcheng Yang
-Author-email: fangjiarui123@gmail.com
-Classifier: Programming Language :: Python :: 3
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE.txt
-Requires-Dist: torch>=2.2
-Requires-Dist: diffusers>=0.30.0
-Requires-Dist: transformers
-Requires-Dist: sentencepiece
-Requires-Dist: accelerate==0.33.0
-Requires-Dist: beautifulsoup4
-Requires-Dist: distvae
-Requires-Dist: ftfy
-
 <div align="center">
   <!-- <h1>KTransformers</h1> -->
   <p align="center">
@@ -42,7 +21,7 @@ Requires-Dist: ftfy
 - [🚀 QuickStart](#QuickStart)
 - [✨ the xDiT's secret weapons](#secrets)
   - [1. PipeFusion](#PipeFusion)
-  - [2. USP](#USP)
+  - [2. Unified Sequence Parallel](#USP)
   - [3. Hybrid Parallel](#hybrid_parallel)
   - [4. CFG Parallel](#cfg_parallel)
   - [5. Parallel VAE](#parallel_vae)
@@ -71,9 +50,12 @@ xDiT provides a suite of efficient parallel inference approaches for Diffusion M
 
 The four parallel methods in xDiT can be configured in a hybrid manner, optimizing communication patterns to best suit the underlying network hardware.
 
-xDiT offers a set of APIs to adapt DiT models in [huggingface/diffusers](https://github.com/huggingface/diffusers) to hybrid parallel implementation through simple wrappers. 
+As shown in the following picture, xDiT offers a set of APIs to adapt DiT models in [huggingface/diffusers](https://github.com/huggingface/diffusers) to hybrid parallel implementation through simple wrappers. 
 If the model you require is not available in the model zoo, developing it yourself is straightforward; please refer to our [Dev Guide](#dev-guide).
 
+<div align="center">
+    <img src="assets/methods/xdit_method.png" alt="xdit methods">
+</div>
 
 We also have implemented the following parallel stategies for reference:
 
@@ -109,6 +91,7 @@ As we can see, PipeFusion and Sequence Parallel achieve lowest communication cos
 
 <h2 id="updates">📢 Updates</h2>
 
+* 🎉**August 9, 2024**: Support Latte sequence parallel version. The inference scripts are [examples/latte_example](examples/latte_example.py).
 * 🎉**August 8, 2024**: Support Flux sequence parallel version. The inference scripts are [examples/flux_example](examples/flux_example.py).
 * 🎉**August 2, 2024**: Support Stable Diffusion 3 hybrid parallel version. The inference scripts are [examples/sd3_example](examples/sd3_example.py).
 * 🎉**July 18, 2024**: Support PixArt-Sigma and PixArt-Alpha. The inference scripts are [examples/pixartsigma_example.py](examples/pixartsigma_example.py), [examples/pixartalpha_example.py](examples/pixartalpha_example.py).
@@ -120,13 +103,18 @@ As we can see, PipeFusion and Sequence Parallel achieve lowest communication cos
 
 <h2 id="support-dits">🎯 Supported DiTs</h2>
 
--  [🟠 Flux](https://huggingface.co/black-forest-labs/FLUX.1-schnell)
--  [🔴 PixArt-Sigma](https://huggingface.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS)
--  [🔵 HunyuanDiT-v1.2-Diffusers](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers)
--  [🟢 PixArt-alpha](https://huggingface.co/PixArt-alpha/PixArt-alpha)
--  [🟠 Stable Diffusion 3](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers)
--  [🔴 DiT-XL](https://huggingface.co/facebook/DiT-XL-2-256)
+| Model Name | CFG | SP | PipeFusion |
+| --- | --- | --- | --- |
+| [🎬 Latte](https://huggingface.co/maxin-cn/Latte-1) | ❎ | ✔️ | ❎ | 
+| [🔵 HunyuanDiT-v1.2-Diffusers](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers) | ✔️ | ✔️ | ✔️ |
+| [🟠 Flux](https://huggingface.co/black-forest-labs/FLUX.1-schnell) | NA | ✔️ | ❎ |
+| [🔴 PixArt-Sigma](https://huggingface.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS) | ✔️ | ✔️ | ✔️ |
+| [🟢 PixArt-alpha](https://huggingface.co/PixArt-alpha/PixArt-alpha) | ✔️ | ✔️ | ✔️ |
+| [🟠 Stable Diffusion 3](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers) | ✔️ | ✔️ | ✔️ |
 
+### Supported by legacy version only:
+
+-  [🔴 DiT-XL](https://huggingface.co/facebook/DiT-XL-2-256)
 
 <h2 id="perf">📈 Performance</h2>
 
@@ -162,18 +150,26 @@ Here are the benchmark results for Pixart-Alpha using the 20-step DPM solver as
 
 <h2 id="QuickStart">🚀 QuickStart</h2>
 
-1. Install yunchang for sequence parallel.
+### 1. Install from pip
+
+```
+pip install xfuser
+```
+
+### 2. Install from source 
+
+#### 2.1 Install yunchang for sequence parallel.
 
 Install yunchang from [feifeibear/long-context-attention](https://github.com/feifeibear/long-context-attention).
 Please note that it has a dependency on flash attention and specific GPU model requirements. We recommend installing yunchang from the source code rather than using `pip install yunchang==0.2.0`.
 
-2. Install xDiT
+#### 2.2 Install xDiT
 
 ```
 python setup.py install
 ```
 
-3. Usage
+### 2. Usage
 
 We provide examples demonstrating how to run models with xDiT in the [./examples/](./examples/) directory. 
 You can easily modify the model type, model directory, and parallel options in the [examples/run.sh](examples/run.sh) within the script to run some already supported DiT models.
@@ -255,7 +251,7 @@ examples/pixartalpha_example.py \
 --num_inference_steps 20 \
 --warmup_steps 0 \
 --prompt "A small dog" \
---use_split_batch
+--use_cfg_parallel
 ```
 
 
@@ -318,5 +314,12 @@ We also welcome developers to join and contribute more features and models to th
       archivePrefix={arXiv},
       primaryClass={cs.CV}
 }
+
+@article{fang2024unified,
+  title={USP: a Unified Sequence Parallelism Approach for Long Context Generative AI},
+  author={Fang, Jiarui and Zhao, Shangchun},
+  journal={arXiv preprint arXiv:2405.07719},
+  year={2024}
+}
 ```
 

{xfuser-0.2 → xfuser-0.3}/setup.py

@@ -10,18 +10,18 @@ if __name__ == "__main__":
 
     setup(
         name="xfuser",
-        author="Jiannan Wang, Jiarui Fang, Jinzhe Pan, Aoyu Li, Pengcheng Yang",
+        author="Jiannan Wang, Jiarui Fang, Jinzhe Pan, Aoyu Li",
         author_email="fangjiarui123@gmail.com",
         packages=find_packages(),
         install_requires=[
-            "torch>=2.2",
+            "torch>=2.3.0",
             "diffusers>=0.30.0",
-            "transformers",
-            "sentencepiece",
+            "transformers>=4.39.1",
+            "sentencepiece>=0.1.99",
             "accelerate==0.33.0",
-            "beautifulsoup4",
+            "beautifulsoup4>=4.12.3",
             "distvae",
-            "ftfy",
+            "ftfy>=6.2.0",
         ],
         url="https://github.com/xdit-project/xDiT.",
         description="xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters",

{xfuser-0.2 → xfuser-0.3}/xfuser/__init__.py

@@ -2,7 +2,9 @@ from xfuser.model_executor.pipelines import (
     xFuserPixArtAlphaPipeline,
     xFuserPixArtSigmaPipeline,
     xFuserStableDiffusion3Pipeline,
-    xFuserFluxPipeline
+    xFuserFluxPipeline,
+    xFuserLattePipeline,
+    xFuserHunyuanDiTPipeline,
 )
 from xfuser.config import xFuserArgs, EngineConfig
 
@@ -11,6 +13,8 @@ __all__ = [
     "xFuserPixArtSigmaPipeline",
     "xFuserStableDiffusion3Pipeline",
     "xFuserFluxPipeline",
+    "xFuserLattePipeline",
+    "xFuserHunyuanDiTPipeline",
     "xFuserArgs",
     "EngineConfig",
 ]

xfuser-0.3/xfuser/__version__.py

@@ -0,0 +1 @@
+__version__ = "0.3"

xfuser-0.3/xfuser/config/args.py

@@ -0,0 +1,291 @@
+import sys
+import argparse
+import dataclasses
+from dataclasses import dataclass
+from typing import Optional, List, Tuple, Union
+
+import torch
+import torch.distributed
+
+from xfuser.logger import init_logger
+from xfuser.core.distributed import init_distributed_environment
+from xfuser.config.config import (
+    EngineConfig,
+    ParallelConfig,
+    TensorParallelConfig,
+    PipeFusionParallelConfig,
+    SequenceParallelConfig,
+    DataParallelConfig,
+    ModelConfig,
+    InputConfig,
+    RuntimeConfig,
+)
+
+logger = init_logger(__name__)
+
+
+class FlexibleArgumentParser(argparse.ArgumentParser):
+    """ArgumentParser that allows both underscore and dash in names."""
+
+    def parse_args(self, args=None, namespace=None):
+        if args is None:
+            args = sys.argv[1:]
+
+        # Convert underscores to dashes and vice versa in argument names
+        processed_args = []
+        for arg in args:
+            if arg.startswith("--"):
+                if "=" in arg:
+                    key, value = arg.split("=", 1)
+                    key = "--" + key[len("--") :].replace("-", "_")
+                    processed_args.append(f"{key}={value}")
+                else:
+                    processed_args.append("--" + arg[len("--") :].replace("-", "_"))
+            else:
+                processed_args.append(arg)
+
+        return super().parse_args(processed_args, namespace)
+
+
+def nullable_str(val: str):
+    if not val or val == "None":
+        return None
+    return val
+
+
+@dataclass
+class xFuserArgs:
+    """Arguments for xFuser engine."""
+
+    # Model arguments
+    model: str
+    download_dir: Optional[str] = None
+    trust_remote_code: bool = False
+    # Runtime arguments
+    warmup_steps: int = 1
+    # use_cuda_graph: bool = True
+    use_parallel_vae: bool = False
+    # use_profiler: bool = False
+    use_torch_compile: bool = False
+    # Parallel arguments
+    # data parallel
+    data_parallel_degree: int = 1
+    use_cfg_parallel: bool = False
+    # sequence parallel
+    ulysses_degree: Optional[int] = None
+    ring_degree: Optional[int] = None
+    # tensor parallel
+    tensor_parallel_degree: int = 1
+    split_scheme: Optional[str] = "row"
+    # pipefusion parallel
+    pipefusion_parallel_degree: int = 1
+    num_pipeline_patch: Optional[int] = None
+    attn_layer_num_for_pp: Optional[List[int]] = None
+    # Input arguments
+    height: int = 1024
+    width: int = 1024
+    num_inference_steps: int = 20
+    prompt: Union[str, List[str]] = ""
+    negative_prompt: Union[str, List[str]] = ""
+    no_use_resolution_binning: bool = False
+    seed: int = 42
+    output_type: str = "pil"
+
+    @staticmethod
+    def add_cli_args(parser: FlexibleArgumentParser):
+        """Shared CLI arguments for xFuser engine."""
+        # Model arguments
+        model_group = parser.add_argument_group("Model Options")
+        model_group.add_argument(
+            "--model",
+            type=str,
+            default="PixArt-alpha/PixArt-XL-2-1024-MS",
+            help="Name or path of the huggingface model to use.",
+        )
+        model_group.add_argument(
+            "--download-dir",
+            type=nullable_str,
+            default=xFuserArgs.download_dir,
+            help="Directory to download and load the weights, default to the default cache dir of huggingface.",
+        )
+        model_group.add_argument(
+            "--trust-remote-code",
+            action="store_true",
+            help="Trust remote code from huggingface.",
+        )
+
+        # Runtime arguments
+        runtime_group = parser.add_argument_group("Runtime Options")
+        runtime_group.add_argument(
+            "--warmup_steps", type=int, default=1, help="Warmup steps in generation."
+        )
+        # runtime_group.add_argument("--use_cuda_graph", action="store_true")
+        runtime_group.add_argument("--use_parallel_vae", action="store_true")
+        # runtime_group.add_argument("--use_profiler", action="store_true")
+        runtime_group.add_argument(
+            "--use_torch_compile",
+            action="store_true",
+            help="Enable torch.compile to accelerate inference in a single card",
+        )
+
+        # Parallel arguments
+        parallel_group = parser.add_argument_group("Parallel Processing Options")
+        parallel_group.add_argument(
+            "--use_cfg_parallel",
+            action="store_true",
+            help="Use split batch in classifier_free_guidance. cfg_degree will be 2 if set",
+        )
+        parallel_group.add_argument(
+            "--data_parallel_degree", type=int, default=1, help="Data parallel degree."
+        )
+        parallel_group.add_argument(
+            "--ulysses_degree",
+            type=int,
+            default=None,
+            help="Ulysses sequence parallel degree. Used in attention layer.",
+        )
+        parallel_group.add_argument(
+            "--ring_degree",
+            type=int,
+            default=None,
+            help="Ring sequence parallel degree. Used in attention layer.",
+        )
+        parallel_group.add_argument(
+            "--pipefusion_parallel_degree",
+            type=int,
+            default=1,
+            help="Pipefusion parallel degree. Indicates the number of pipeline stages.",
+        )
+        parallel_group.add_argument(
+            "--num_pipeline_patch",
+            type=int,
+            default=None,
+            help="Number of patches the feature map should be segmented in pipefusion parallel.",
+        )
+        parallel_group.add_argument(
+            "--attn_layer_num_for_pp",
+            default=None,
+            nargs="*",
+            type=int,
+            help="List representing the number of layers per stage of the pipeline in pipefusion parallel",
+        )
+        parallel_group.add_argument(
+            "--tensor_parallel_degree",
+            type=int,
+            default=1,
+            help="Tensor parallel degree.",
+        )
+        parallel_group.add_argument(
+            "--split_scheme",
+            type=str,
+            default="row",
+            help="Split scheme for tensor parallel.",
+        )
+
+        # Input arguments
+        input_group = parser.add_argument_group("Input Options")
+        input_group.add_argument(
+            "--height", type=int, default=1024, help="The height of image"
+        )
+        input_group.add_argument(
+            "--width", type=int, default=1024, help="The width of image"
+        )
+        input_group.add_argument(
+            "--prompt", type=str, nargs="*", default="", help="Prompt for the model."
+        )
+        input_group.add_argument("--no_use_resolution_binning", action="store_true")
+        input_group.add_argument(
+            "--negative_prompt",
+            type=str,
+            nargs="*",
+            default="",
+            help="Negative prompt for the model.",
+        )
+        input_group.add_argument(
+            "--num_inference_steps",
+            type=int,
+            default=20,
+            help="Number of inference steps.",
+        )
+        runtime_group.add_argument(
+            "--seed", type=int, default=42, help="Random seed for operations."
+        )
+        runtime_group.add_argument(
+            "--output_type",
+            type=str,
+            default="pil",
+            help="Output type of the pipeline.",
+        )
+
+        return parser
+
+    @classmethod
+    def from_cli_args(cls, args: argparse.Namespace):
+        # Get the list of attributes of this dataclass.
+        attrs = [attr.name for attr in dataclasses.fields(cls)]
+        # Set the attributes from the parsed arguments.
+        engine_args = cls(**{attr: getattr(args, attr) for attr in attrs})
+        return engine_args
+
+    def create_config(
+        self,
+    ) -> Tuple[EngineConfig, InputConfig]:
+        if not torch.distributed.is_initialized():
+            logger.warning(
+                "Distributed environment is not initialized. " "Initializing..."
+            )
+            init_distributed_environment()
+
+        model_config = ModelConfig(
+            model=self.model,
+            download_dir=self.download_dir,
+            trust_remote_code=self.trust_remote_code,
+        )
+
+        runtime_config = RuntimeConfig(
+            warmup_steps=self.warmup_steps,
+            # use_cuda_graph=self.use_cuda_graph,
+            use_parallel_vae=self.use_parallel_vae,
+            use_torch_compile=self.use_torch_compile,
+            # use_profiler=self.use_profiler,
+        )
+
+        parallel_config = ParallelConfig(
+            dp_config=DataParallelConfig(
+                dp_degree=self.data_parallel_degree,
+                use_cfg_parallel=self.use_cfg_parallel,
+            ),
+            sp_config=SequenceParallelConfig(
+                ulysses_degree=self.ulysses_degree,
+                ring_degree=self.ring_degree,
+            ),
+            tp_config=TensorParallelConfig(
+                tp_degree=self.tensor_parallel_degree,
+                split_scheme=self.split_scheme,
+            ),
+            pp_config=PipeFusionParallelConfig(
+                pp_degree=self.pipefusion_parallel_degree,
+                num_pipeline_patch=self.num_pipeline_patch,
+                attn_layer_num_for_pp=self.attn_layer_num_for_pp,
+            ),
+        )
+
+        engine_config = EngineConfig(
+            model_config=model_config,
+            runtime_config=runtime_config,
+            parallel_config=parallel_config,
+        )
+
+        input_config = InputConfig(
+            height=self.height,
+            width=self.width,
+            use_resolution_binning=not self.no_use_resolution_binning,
+            batch_size=len(self.prompt) if isinstance(self.prompt, list) else 1,
+            prompt=self.prompt,
+            negative_prompt=self.negative_prompt,
+            num_inference_steps=self.num_inference_steps,
+            seed=self.seed,
+            output_type=self.output_type,
+        )
+
+        return engine_config, input_config