mmgp 1.0.5__py3-none-any.whl → 1.1.0__py3-none-any.whl

{mmgp-1.0.5.dist-info → mmgp-1.1.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mmgp
-Version: 1.0.5
+Version: 1.1.0
 Summary: Memory Management for the GPU Poor
 Author-email: deepbeepmeep <deepbeepmeep@yahoo.com>
 License:                     GNU GENERAL PUBLIC LICENSE
@@ -683,7 +683,11 @@ License-File: LICENSE.md
 Requires-Dist: torch>=2.1.0
 Requires-Dist: optimum-quanto
 
-**------------------ Memory Management for the GPU Poor by DeepBeepMeep ------------------**
+
+<p align="center">
+  <H2>Memory Management for the GPU Poor by DeepBeepMeep</H2>	
+</p>
+
 
 This module contains multiples optimisations so that models such as Flux (and derived), Mochi, CogView, HunyuanVideo, ...  can run smoothly on a 24 GB GPU limited card. 
 This a replacement for the accelerate library that should in theory manage offloading, but doesn't work properly with models that are loaded / unloaded several
@@ -693,31 +697,50 @@ Requirements:
 - GPU: RTX 3090/ RTX 4090 (24 GB of VRAM)
 - RAM: minimum 48 GB, recommended 64 GB 
 
+First you need to install the module in your current project with:
+```shell
+pip install mmgp
+```
+
 It is almost plug and play and just needs to be invoked from the main app just after the model pipeline has been created.
-1) First make sure that the pipeline explictly loads the models in the CPU device 
-  for instance: pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+1) First make sure that the pipeline explictly loads the models in the CPU device, for instance:
+```
+  pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
+```
+
 2) Once every potential Lora has been loaded and merged, add the following lines:
 
-  *from mmgp import offload*
-  *offload.me(pipe)*
-  
-The 'transformer' model that contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
+```
+  from mmgp import offload
+  offload.me(pipe)
+```  
+The 'transformer' model in the pipe contains usually the video or image generator is quantized on the fly by default to 8 bits. If you want to save time on disk and reduce the loading time, you may want to load directly a prequantized model. In that case you need to set the option *quantizeTransformer* to *False* to turn off on the fly quantization.
 
 If you have more than 64GB RAM you may want to enable RAM pinning with the option *pinInRAM = True*. You will get in return super fast loading / unloading of models
 (this can save significant time if the same pipeline is run multiple times in a row)
 
-Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.
-
+Sometime there isn't an explicit pipe object as each submodel is loaded separately in the main app. If this is the case, you need to create a dictionary that manually maps all the models.\
 For instance :
-for flux derived models: *pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }*
-for mochi: *pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }*
+
+
+- for flux derived models: 
+```
+pipe = { "text_encoder": clip, "text_encoder_2": t5, "transformer": model, "vae":ae }
+```
+- for mochi: 
+```
+pipe = { "text_encoder": self.text_encoder, "transformer": self.dit, "vae":self.decoder }
+```
+
 
 Please note that there should be always one model whose Id is 'transformer'. It corresponds to the main image / video model which usually needs to be quantized (this is done on the fly by default when loading the model).
 
 Becareful, lots of models use the T5 XXL as a text encoder. However, quite often their corresponding pipeline configurations point at the official Google T5 XXL repository 
 where there is a huge 40GB model to download and load. It is cumbersorme as it is a 32 bits model and contains the decoder part of T5 that is not used. 
 I suggest you use instead one of the 16 bits encoder only version available around, for instance:
-*text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)*
+```
+text_encoder_2 = T5EncoderModel.from_pretrained("black-forest-labs/FLUX.1-dev", subfolder="text_encoder_2", torch_dtype=torch.float16)
+```
 
 Sometime just providing the pipe won't be sufficient as you will need to change the content of the core model: 
 - For instance you may need to disable an existing CPU offload logic that already exists (such as manual calls to move tensors between cuda and the cpu)
@@ -727,6 +750,6 @@ You are free to use my module for non commercial use as long you give me proper
 
 Thanks to
 ---------
-Huggingface / accelerate for the hooking examples
-Huggingface / quanto for their very useful quantizer
-gau-nernst for his Pinnig RAM samples
+- Huggingface / accelerate for the hooking examples
+- Huggingface / quanto for their very useful quantizer
+- gau-nernst for his Pinnig RAM samples

mmgp-1.1.0.dist-info/RECORD

@@ -0,0 +1,7 @@
+__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+mmgp.py,sha256=SNnSSAtSoavzBgJQqp_O-9JlgsmI7pjboF3aO6MGZNE,20770
+mmgp-1.1.0.dist-info/LICENSE.md,sha256=ixuiBLtpoK3iv89l7ylKkg9rs2GzF9ukPH7ynZYzK5s,35148
+mmgp-1.1.0.dist-info/METADATA,sha256=RrKUBzntWC8fFolmAzRgakswejR0D4W715AeI49QHj8,44527
+mmgp-1.1.0.dist-info/WHEEL,sha256=PZUExdf71Ui_so67QXpySuHtCi3-J3wvF4ORK6k_S8U,91
+mmgp-1.1.0.dist-info/top_level.txt,sha256=waGaepj2qVfnS2yAOkaMu4r9mJaVjGbEi6AwOUogU_U,14
+mmgp-1.1.0.dist-info/RECORD,,

{mmgp-1.0.5.dist-info → mmgp-1.1.0.dist-info}/top_level.txt

@@ -1,3 +1,2 @@
 __init__
-_version
 mmgp

_version.py

@@ -1,16 +0,0 @@
-# file generated by setuptools_scm
-# don't change, don't track in version control
-TYPE_CHECKING = False
-if TYPE_CHECKING:
-    from typing import Tuple, Union
-    VERSION_TUPLE = Tuple[Union[int, str], ...]
-else:
-    VERSION_TUPLE = object
-
-version: str
-__version__: str
-__version_tuple__: VERSION_TUPLE
-version_tuple: VERSION_TUPLE
-
-__version__ = version = '1.0.5'
-__version_tuple__ = version_tuple = (1, 0, 5)

mmgp-1.0.5.dist-info/RECORD

@@ -1,8 +0,0 @@
-__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-_version.py,sha256=ClFIlbf5O23dVWC6oD6t3m2n6nB22IOt3V9t-YX_mG0,411
-mmgp.py,sha256=SNnSSAtSoavzBgJQqp_O-9JlgsmI7pjboF3aO6MGZNE,20770
-mmgp-1.0.5.dist-info/LICENSE.md,sha256=ixuiBLtpoK3iv89l7ylKkg9rs2GzF9ukPH7ynZYzK5s,35148
-mmgp-1.0.5.dist-info/METADATA,sha256=alsu9xZpEgqcSz-xLg3k2B5eF-3u03el7-nOaLu0HZo,44381
-mmgp-1.0.5.dist-info/WHEEL,sha256=PZUExdf71Ui_so67QXpySuHtCi3-J3wvF4ORK6k_S8U,91
-mmgp-1.0.5.dist-info/top_level.txt,sha256=gf5FPNwF128kswDDAg8wKlyHQtBbgTfA37hYVvlA7mI,23
-mmgp-1.0.5.dist-info/RECORD,,